Everyone wants to pull them out

The origin of conspiratorial objects is not known. It's not known where they came from. It's unknown if their actions are motivated by any objectives. It's not known how they came into being or how they were created. Due to their mysterious abnormalities — they don't lend themselves to be classified; due to their unnatural powers — they actively prevent classification.

Conspiratorial objects endeavour to destroy of the Foundation in a very unpleasant way — through indirect influence. They understand the concept of "destruction" both as the physical collapse of every Site as well as global exposure of the Foundation. Even though the organization seems to be helpless in face of their actions, after all is said, it still knows they exist.

That might turn out to be its greatest achievement.

But they always come back

Introductory Tale: What are conspiratorial objects? by Dr JamesH
Conspirational objects achieved surprising proficiency in tricking the Foundation, but they seem to not know about the particular organization. They don't know what is our organization, what are our IDs, how do we communicate, where do we privately live, how many Class D personnel or council members we have. They are aware, maybe intelligent, but even though it's like someone programmed them to just destroy us, just because, without a reason.