An agent can write one good note. How do you keep 1,000 notes coherent?

I’ve been experimenting with using LLM agents to maintain a fairly large knowledge base over time.
What I’ve found is that getting an agent to write one good note isn’t particularly difficult. The real problem starts when the knowledge base grows to hundreds or thousands of files.

At that point, every individual edit can look reasonable while the corpus as a whole slowly becomes less coherent.

The same concept starts appearing on multiple pages with slightly different explanations. A canonical page gets updated, but its summaries and indexes don’t. Links, metadata, and terminology gradually drift apart.

There’s also a more frustrating failure mode: the agent says the task is complete, and the validation script reports success, but only part of the intended corpus was processed—or the script checked the wrong directory entirely.

So I’m starting to think this is less of a content-generation problem and more of a closed-loop corpus management problem.

How does an agent know:
which files are actually in scope;
which pages are canonical and which are derived;
what else needs to be updated after a change;
whether important knowledge was accidentally removed;
and whether the task is genuinely complete, rather than merely producing plausible output?

I’m curious whether anyone here is maintaining an AI-managed knowledge base with more than 1,000 files.
How are you handling it?

Do you use dependency graphs, manifests, human review, periodic full rebuilds, transactional batches, or separate agents for planning, writing, and validation? Or have you found a completely different approach?

I recently built my own attempt at this, called Cambium:
https://github.com/KimGLee/Cambium

It isn’t a knowledge-base application or a RAG framework. It’s closer to a set of governance rules and deterministic checks for agents maintaining a corpus over time.

The main idea is to separate:
a stable set of domain-independent maintenance rules;

from a user-defined Profile describing the structure and requirements of a specific knowledge base.
It also uses deterministic scripts to check things like links, vocabulary, task scope, residual content, and completion evidence, rather than trusting the agent to decide for itself that the work is finished.

The project is still early.
So far, I’ve mainly tested it on a knowledge base about LLMs and agent systems. I don’t yet have enough evidence to say that the same structure works well for legal knowledge, scientific research, software documentation, education, operations, or other domains.

It also currently assumes one coordinating agent. The rules allow bounded parallel work, but actual multi-agent orchestration still needs to be implemented by the host system.

I’m mainly interested in two things:
First, have you seen the same kind of corpus-level drift—where individual pages still look fine, but the knowledge base gradually loses consistency?
Second, what other maintenance loops or governance models are people using?

Blunt criticism of Cambium is also welcome, especially around whether separating the Kernel from the Profile is actually useful, and which parts of knowledge-base governance should never be left to an agent in the first place.