True Local AI capabilities - model selection - prompt finess...

Question | Help

Hello Guys,
I am experimenting with ollama and n8n for some automation.
The gig: I am pulling from the French piste.gouv.fr court decisions on a period of a month with n8n and the published API. Some processing is done and then I have a code node that is preparing the prompt to be passed to an http request to my local ollama server and then its output is also processed to build an email to be sent to me.
The goal is to have a summary of the decisions that are in my field of interest.
My server: Unraid, Hardware: i5-4570 + 16 Gb DDR + GTX 1060 6GB, and I have tested with a few models (qwen3:4b, phi3:mini, ministral-3:3b, ministral-3:8b, mistral-latestgemma3:4b and Llama3.1:8b
I could receive an output for like 2-3 decisions and the rest would be ignored.
Then I decided to try with my gamin PC (W11 + i5-13700 + 32 GB DDR5 + RTX 4070 Ti
with qwen2.5:14b, ministral-3:14b
Then with kids gaming PC (W11 + Ryzen 7800X3D + 32 GB DDR5 + RTX 4070 Ti Super 16 GB with mistral-small3.2:24b and qwen3:32b

My prompt goes: you are a paralegal and you have to summarize each decision reported below (in real it is a json passing the data) you have to produce a summary for each decision, with some formating etc... some keywords are implemented to short list some of the decisions only.
only one time my email was formated correctly with an short analysis for each decision.
All the other times, the model would limit itself to only 2-3 decisions, or would group them or would say it need to analyse the rest etc...
So my question: is my task too complex for so small models (max 32b parameters) ?
For now I am testing and i was hoping to have a solid result, expeting long execution time considering the low power machine (unraid server) but even with more modern platform, the model fails.
Do I need a much larger GPU VRAM like 24 GB minimum to run 70b models ?
Or is it a problem with my prompt? I have set the max_token to 25000 and timeout to 30 mn.
Before I crack the bank for a 3090 24 GB, I would love to read your thoughts on my problem...
Thank you for reading and maybe responding!!
AI Noob Inside

Posted Feb 19, 2026, 3:21 PM UTC

12
Loading comments…