Built a fully autonomous Android agent — 30 tools, on-device model, and a policy gate that blocks every untraced action

I've been building an autonomous Android agent that controls the phone through the accessibility tree. It reads the

screen, decides what to do, does it, then verifies it worked — a full perceive-think-act-verify loop.

What it does:

30 tools — flashlight, clipboard, SMS, app launch, web search, navigation through any app. A spoken request through

the side button gets a spoken answer back, even with the app backgrounded.

The brain is a two-tier system: a Gemma 3 1B model runs on-device for simple commands (battery, toggle flashlight,

open an app) with zero latency. Anything harder escalates to a 70B cloud model. The router is deterministic — keyword

match + intent slot, no model call to decide which model to call.

The part I think is worth discussing here — the policy gate:

Every tool call passes through a manifest-checked gate before it touches the device. The gate doesn't log-and-allow —

it blocks. An operator tap resolves anything the gate flags. I ported this from a separate project where I was

measuring prompt injection defenses across 112 models. Same architecture, enforced on-device.

This is the part I'd push back on with most agent frameworks: autonomy without a hard gate isn't autonomy, it's a

liability. The agent can do anything the phone can do — which is exactly why it needs a layer that can say no.

Numbers:

- 134 unit tests, 8/8 device regression suite

- Release APK is ~8.7 MB (R8 minified)

- Hands-free assist works with the app backgrounded

- Named recipes: teach a multi-step task once, replay it later

- Deep perception reads structured data from apps (e.g., product + price from an Amazon search across 20 screens)

Native Kotlin, built solo.

Happy to go deeper on the gate architecture or the tool loop.