How to reduce latency when injecting CRM context into live voice agents?

Running into something annoying and curious how others are handling it.

For inbound voice calls, we look up CRM data before the first LLM response - stuff like last interaction summary, open tickets, account state.

• Call connects
• Caller ID - CRM lookup
• Pull structured fields
• Inject into system prompt
• First model response

Even with fast queries, that adds ~400–600ms. The agent feels slightly slow on the first turn.

Feels like a tradeoff between responsiveness and intelligence.

Curious how people are solving this without degrading UX.