help with pi coding agent + vllm setup

Question

Hi, I am trying to get this setup working, but I simply cannot figure out what I am doing wrong.

I use docker.io/vllm/vllm-openai:latest for vllm + the containerization guide from the pi.dev website. (although i use podman)

This is my models.json file:

{
  "providers": {
    "vllm": {
      "baseUrl": "http://localhost:8010/v1",
      "api": "openai-completions",
      "apiKey": "vllm",
      "compat": {
        "supportsDeveloperRole": false,
        "supportsReasoningEffort": false
      },
      "models": [
            { "id": "Qwen/Qwen3-0.6B" }
      ]
    }
  }
}

My vllm endpoint is working, and i can reach it from inside the pi agent container, but when it try to interact with the model, i always get: Error: Connection error. I can see my listed model though.

Any suggestions would be really appreciated.

Posted Jun 15, 2026, 12:47 PM UTC

27
Loading comments…