Sql querying

I am building a chatbot for one of my use case where I have my db information in the form of JSON data. Now to provide the semantic search using rag I need to chunk them . But in my use case the json are nested jsons having table , column , relationship and index information along with business description.

Chunking strategy: I applied hybrid chunking process like column level chunking and table level chunking and then combine them with medata information . But I see poor results as it is giving better with hardcoded rule mapping than semantic one.

Can anyone help me with the right set of chunking strategy as I need to identify the right column and tatable for given query .

Thanks