How I improved my Hermes agent's skill changes using simulation & introspection

How do we make an Agent *REALLY* self improve dramatically better? Self retrospection and self improvements are useful, but there's an approach that makes them dramatically more effective.

The story started when we reported a user bug to Tracey, our Superagent.

Tracey found the pattern that caused the issue but stopped short of doing the deep root cause analysis and implementing a PR for a fix for the systematic bug that caused the issue.

So, I forked the conversation:

1. Fix the systematic issue
2. This is the interesting part that triggered the post: I asked it to create a dump of this conversation up to this point.

I took the dump file and created a new session with Tracey. I asked it for its honest opinion of what it thought about the interaction. It correctly identified that that it stopped short of solving the systematic issue and figured out it was too focused on the customer request and customer communication.

Next, I took two steps:

1. I realigned its priorities. The real top priority is to identify systematic failures and issues so that it can fix them. We're using user bug reports as an instrument for the greater good. It's far more important to fix the systematic bug than explaining or remediating the bug for the reporting user.

2. I then asked Tracey to review the modified skills and changes it had made and to *run a simulation of the previous conversation* under the new rule changes.

In effect, I labeled what was wrong and asked it to loop until it verified that it got the solution right.

We usually just ask an agent to change its behavior and trust that when it says it modified some skills or files.

For some reason, we trust that when the agent claims it will now behave a certain way, that it actually will. It technically changes its own skills and instructions, but it doesn't test it against a real world case and it often forgets conflicting instructions or things that could make it non-optimal.

By forcing the agents to to validate the changes in its behavior through running a simulation, it loops until the results are optimal. This is now the SOP (Standard Operating Procedure) any time there's a skill change, they'll run validation and improvement loops.

In general, an agent should PROVE it IMPROVED, we shouldn't just trust that it has.

BTW, it even ran the simulation self check on the skill for self improvement self checks... The image is the results after a few loop runs

https://preview.redd.it/sd2ehx1fj4kh1.png?width=800&format=png&auto=webp&s=4cbbc8ac006526000d906ea2cfd5a2a92f4d8137