Why is inference becoming so important for AI, and what happens if architectures move beyond transformers?

I’ve been trying to understand why so much of the AI hardware discussion is shifting from training to inference.

My understanding is that training creates the model, but inference is everything that happens when people actually use it and newer reasoning models may require far more inference compute because they spend more time “thinking” before answering.

does that make companies like Etched interesting because they are building chips specifically optimized for transformer inference rather than general purpose GPUs?

My questions are: Why is inference expected to become such a large part of total AI compute? Is increased inference time reasoning likely to be the main way AI systems continue improving? And if transformers are eventually replaced by a different architecture, how much of the current inference infrastructure including specialized chips like Etched’s would need to change?