Codex improved my agent harness by 40% overnight

Remember the first time you used Codex? That same jump is happening one level up. The community went from prompt engineering → context engineering → agent engineering → harness engineering.

I asked myself: what sits one level above the harness? Something that builds the harness.

So I built it. Autoharness lets Codex explore changes to your harness (e.g. prompts, hyperparameters, runtime context, scoring) run evals, and keep only the changes that actually improve the score. Inspired by Karpathy's autoresearch.

I pointed it at my own agent and let it run. On the tau2-airline benchmark, it autonomously found:

• +40.7% performance lift from adding best-of-N skillbook scoring with an LLM judge
• +24.1% performance lift from tightening reflector hyperparams (temperature + max subagent calls)
• +22.2% performance lift from injecting runtime context at every step (step budget, recent tool calls, recent results)

How it works:

1. One-line install
2. Point Codex at GUIDE.md
3. It proposes harness changes, evals each, keeps only the wins
4. Wake up to a better agent

TL;DR: Codex tunes my agent's prompts and configs for me. It tries a change, runs my eval, and keeps it only if the score went up.

Open-Source Repo: https://github.com/kayba-ai/autoharness