llama.cpp or vllm for qwen3.5 9b serving.

Question | Help

I was using llama.cpp which I had compiled from source but I found my http connection was wasting time so I decided to go with a python wrapper and interface that way. I have had to recompile the world. I had to recompile cmake which is huge. Still not finished but almost there. Would vllm been a better way to go? I actually had better performance when I ran the model in lmstudio cli. Its almost done now so I am going to continue but I am thinking vllm on ubuntu if it isn't faster. I need speed to aggregate the results from a chromadb search into a response.

Any opinion on vllm for these models

Posted Mar 6, 2026, 4:08 AM UTC

66
Loading comments…