I got tired of MCP servers failing silently, so I built a conformance + regression tester for them

I've been building a few MCP servers and kept running into the same annoying thing: the server doesn't crash when something's off. There's no error in the logs. The model on the other end just starts acting weird, skips a tool, or misreads a result, and I end up bisecting commits and re-reading the spec trying to figure out what I broke.

Got sick of debugging that by hand so I wrote a tool. It's called vexyo, MIT-ish (Apache-2.0), on npm.

It connects to your server like a client would and checks it against the spec (2025-11-25). 16 rules right now, across init, discovery, error handling, and transport. When something fails it tells you which rule and points at the spec section, instead of you guessing. Looks like this:

✗ discovery/tools-list-available
tools/list failed despite the tools capability being advertised
spec: Server Features §Tools / Listing Tools

15 pass · 1 fail · exit 1  

There's also a regression mode: it records what your tools return, then on later runs it flags when the output or schema changed, so you catch stuff that quietly drifted between commits.

It's meant to run in CI (exits non-zero, has a GitHub action + junit/json output), but you can just run it locally too. And it doesn't care what language your server is in since it only talks over stdio/http.

npm i -D @vexyo/cli 
npx vexyo init 
npx vexyo run

repo: https://github.com/vexyohq/vexyo

docs: https://vexyo.dev

It's early and it scratched my own itch first, so I'm mostly curious whether it catches anything real on other people's servers, or where it gets things wrong. If you try it lmk what happened.