I spent weeks improving my LangGraph reviewer before realizing I was optimizing the wrong thing

A few months ago I started building an automated PR reviewer with LangGraph.

At first I focused on the usual things: review quality, prompt design, context selection, reducing false positives, and so on.

The feedback looked pretty good.

Then I noticed something odd.

Some review comments never appeared on the pull request.

At first I assumed it was a GitHub API issue.

After digging deeper, I found that the problem was much simpler: the model was occasionally attaching comments to diff locations that didn’t actually exist.

What’s interesting is that the feedback itself was often reasonable.

The failure wasn’t in the review.

It was in the coordinates.

I tried improving prompts and adding more instructions, but the issue never completely went away.

Eventually I stopped treating it as a generation problem and started treating it as a validation problem.

That led me to build a small validation layer that parses git diffs, constructs a map of valid coordinates, and checks every generated comment before it can be posted.

The idea is pretty simple, but it made the review pipeline much more reliable.

I ended up open-sourcing it as DiffLens:

https://github.com/ywu593412-afk/difflens

I’m curious whether others building review agents or coding workflows have run into similar issues.

Most discussions around hallucinations focus on the content itself.

I don’t see many people talking about hallucinated coordinates.