What matters most when choosing an LLM for an AI agent?

I've been experimenting with different LLMs for AI-agent workflows, and one thing I've noticed is that the "best" model isn't always the one with the highest benchmark scores.

For an agent, things like response speed, reliability, tool calling, context handling, and cost can make a pretty big difference depending on the workflow.

I'm curious how others here choose models for their agents.

Do you usually stick with one model, or do you switch between different models depending on the task? And which factor matters most to you: quality, speed, cost, or reliability?