10-15 agents a day meant I stopped being able to check anything, so I built this.

Nowadays my workflow is spawning 10-15 agents and having them do the work by theirselves; research, code, tests etc. And they do it pretty well. But there always has been one gap - they don’t see well what they’ve built.

Now focus on the “well”, of course we can wire up playwright with claude code and ask it to run it’s version and see if it works; but it almost never was able to catch any bugs that I later found from prod either by using the app myself, or an angry customer bashing me on email.

Here’s a few why’s:

• Claude is super optimistic - it sees builds pass, tests pass, lints pass, CI looking green and it feels it’s good to go.
• /pr-review-skill catches most of the code level bugs, giving Claude false confidence about the quality of PR.
• Claude knows what works, and tries to enact the exact steps that will make the UI test go green.

The last one is a real problem, since the model itself is not a bad tester, the harness is.

Now of course, we can fix most of them with super detailed prompting; but then there are a few more issues;

1. Bypassing auth; even on my app
2. Claude’s speed and linearity; each tests session blocks claude code
3. No evidence; you get the assertion of it working, but where’s the proof
4. No memory of app specific flaws, flows, and traits
5. No compounding memory of runs; it stays at the exact same quality throughout runs

So to fix all that, and to reclaim my cognitive bandwidth, I created this app that can run verification tests on my PR & Preview Deployments.

It provides me with visual evidence of things working / breaking and provides my agent the context required to fix those. More or less a complete loop where I am just a supervisor.

https://preview.redd.it/vt2fon0707jh1.png?width=813&format=png&auto=webp&s=f4677bbc2f4c172fa602c50fecb03a08225be228

This helped me calm my nerves about claude code PRs breaking my production and need to check everything manually, when we have the power of the tokens.

I tested this with ~15 apps over the weekends (3 mine, 12 others) and it found fewer but real issues compared to claude code, attached a sample here.

I have the code on Github, can share if anybody wants to try or contribute. Otherwise you can install it using `npx keryai`

What do you use for verifying claude code generated code works ?