Is anyone else frustrated that Cursor sometimes loses visual context?

Thinking of building a cloud or local screen-watching voice agent for myself, and just wanted to see if anyone ever had this same issue.

Cursor is great for code context, but it can sometimes get cumbersome when you switch windows.

I’m thinking of building a WebRTC screen-watching voice agent with Gemini Live or a local model to sit alongside Cursor. Would this actually speed up your workflow or just be annoying?

I’m specifically trying to acknowledge the moments where you currently have to stop and take screenshots, or just inconveniently type out the issue:

- Figma/Localhost: Having Figma on one monitor, localhost on the other, and asking the agent what CSS you missed just by looking at the screen.

- Cloud Consoles: Staring at a messy AWS Console/GCP dashboard and having the agent correlate any visual logs with your codebase, or maybe even take control of the browser.

- UI Bugs: Debugging CSS/Framer Motion bugs where there’s no console error to copy-paste but just a visually broken UI.

- Un-indexed docs: Scrolling through obscure API docs in your browser when trying to integrate a new service/feature.

I know MCP exists and is great for pretty much anything, but I wanted to make this less as a substitute and more for convenience for those who use voice.

Has anyone tried making something similar or found an existing solution?