LLM Analytics in Enterprises?

Hi folks

Im curious to understand if and how teams are building their LLM analytics for internal usage across different organisations. Additionally, how would you test to ensure theres low hallucinations etc.

For example in my team (small organisation <50 people), we built an MCP server that runs on Cloudflare workers. We then have our main MCP client which is Claude that connects to that MCP. We have developed many skills and amongst it is a data warehouse skill which contains knowledge.md and skills.md files to describe the data warehouse. Those md files essentially are our semantic layer. We have some test coverage by domain which we try to evaluate desired sql outputs based on sample questions but its really rudimentary at the moment.

This was meant to help 'democratise' data but without proper testing and a robust evaluation infrastructure, it has really led exposing a lot of the key gaps, data quality and documentation issues.

I'm keen to understand how people are tackling this across organisations of varying sizes!