The next bottleneck in RAG may not be search latency, but data iteration speed

I talk with developers building RAG systems pretty often, and one thing keeps coming up.

Most teams start by optimizing retrieval latency. That makes sense. If search is slow, the whole app feels slow.

But once the first version works, another problem shows up: improving the data behind retrieval is often slower than retrieval itself.

That is why the idea behind Vector Lakebase is interesting to me.

The demo pipeline is simple: chunk documents, embed them, index them, retrieve top-k results.

Production RAG is less clean. Teams keep changing chunks, metadata, embeddings, evals, and indexes.

The painful part is that online serving and offline data work are often separate. The search index serves traffic, while deduplication, clustering, re-embedding, quality analysis, and cold data search happen somewhere else.

So data gets exported, processed, copied back, re-indexed, and kept in sync by glue code.

Vector Lakebase is trying to close that gap: keep low-latency retrieval, but bring serving and discovery closer to the same logical data, with different compute modes for online, on-demand, and offline work.

I do not see this as replacing vector databases. It is more like recognizing that RAG is becoming a loop: serve, observe, improve the data, rebuild, evaluate, repeat.

Latency still matters. But once search is fast enough, I’m curious where others feel the bigger pain: query latency or data iteration?