Local PDF Database searchable with ollama - best setup?

Hi all,

I recently digitized most of my documents as PDFs and set up Paperless-NGX to provide a searchable web-based database. Since the documents are analyzed with OCR, the search in Paperless-NGX for certain keywords works really well to identify documents related to e.g. car repairs.

Now I'm trying to use a local LLM to ask it questions about my said car repairs, e.g. "which parts were repaired in 2024 and how much did they cost each? what is the phone number of the shop?"

I'm running ollama locally on a Win11 host machine an can run e.g. qwen3:32b or mixtral:8x7b at sufficient speeds.
I want this local LLM to search my PDFs and provide me with answers.

I tried setting up paperless-ai which kinda works for "tagging" the documents, but the RAG chat function really is all over the place. It analyzes each and every document before giving an answer but the answers are all over the place. When asked about car repairs it also gives information about my motorcycle or random stuff about my telecom provider...

Second i tried setting up AnythingLLM where documents can be loaded into the "Workspace" and this works well when only single documents are loaded. However, if I load 300 PDFs (which are not OCR scanned because they're not based on paperless-ngx) it also fails miserably.

Is there any recommened setup for my use case?

The goal is to ask questions about car/bike repairs, internet/power contracts or concert tickets.

Tx in advance for any help.