The test I use now: does it keep running, or does it just generate once?

One thing that keeps coming up in conversations with data teams: half of what’s labeled an AI agent for pipelines is really a code generator that goes quiet after one run.

Simple test one of my teammates here at Nexla swears by: pick a task, like onboarding a new source. Watch how many times a human has to click approve before it’s done. If it’s zero, you’ve got an agent. If it’s every step, you’ve got a well dressed copilot.

We actually built a side by side of this exact test in our latest piece, run the same onboarding task through both models and watch who's deciding at each step: https://nexla.com/ai-agents-for-data-engineering-what-they-actually-automate/

Two things I'm actually wondering:

• Which agent tools in your stack are actually autonomous versus just really good autocomplete?
• Has anyone let an agent handle schema drift end to end without a human checkpoint, how’d that go?