I got tired of writing the same code review comments over and over, so I automated it with Ai

I've been working on a side project for the past few days and wanted to share it for feedback.

Hey everyone, I've been working on a side project for the past few days and wanted to share it for early feedback.

The idea: I kept writing the same PR review comments over and over. I wanted to automate this without maintaining a complex bot — and I took inspiration from Codex's skill system to do it.

SkillFlow is an Azure DevOps pipeline task that runs LLM agents on every Pull Request. Each agent is defined by a plain .md file called a Skill — a small YAML frontmatter for configuration and a Markdown body that's the system prompt. Anyone on the team can write their own Skill and customize the agent behavior however they want.

markdown

---
name: security-scanner
provider: claude
model: claude-sonnet-4-6
api_key_var: ANTHROPIC_API_KEY
output: comments
tools:
  - get_pr_diff
  - get_file_content
  - post_inline_comment
---

You are a senior security engineer. Review every changed file
for OWASP Top 10 vulnerabilities and post inline comments...

Drop it in .azdo/skills/ and it runs automatically on every PR.

What a Skill can do:

• Post inline comments and code suggestions directly on the diff
• Commit changes directly to the PR branch (e.g. auto-update a changelog, add docstrings)
• Open a new Pull Request with the generated changes (e.g. generated tests)

Under the hood: built on LangChain + LangGraph with a ReAct agent loop. Supports Claude, OpenAI, Azure OpenAI, and Ollama — you pick the model per Skill.

It's early days and I'd love feedback, ideas, or just to know if this is something others would find useful!

GitHub: https://github.com/simonebittidev/azure-devops-skillflow