My team has to stop this "let me grab this AI framework" mentality and think about overall system design

I think this might be a phenomenon in most places that are tinkering with AI, where the default is that "xyz AI framework has this functionality that can solve a said problem (e.g. guardrails, observability, etc.) so lets deploy that".

What grinds my gears is how this approach completely ignores the fundamental questions us senior devs should be asking when building AI solutions. Sure, a framework probably has some neat features, but have we considered how tightly coupled its low-level code is with our critical business logic (aka function/tools use and system prompt)? When it inevitably needs an update, are we ready for the ripple effect it'll have across our deployments? For example, how do I make a centrally update on rate limiting, or jailbreaking to all our AI apps if the core low-level functionality is baked into the application's core logic? What about dependency conflicts over time? Bloat, etc. etc.

We haven't seen enough maturity of AI systems to probably warrant an AI stack yet. But we should look at infrastructure building blocks for vector storage, proxying traffic (in and out of agents), memory and whatever set of primitives we need to build something that helps us move faster not just to POC but to production.

At the rate of which AI frameworks are being launched - they'll soon be deprecated. Presumably some of the infrastructure building blocks might get deprecated too but if I am building software that must be maintained and pushed to production I can't just whimsically leave everyone to their own devices. Its poor software design, and at the moment despite the copious amounts of code LLMs can generate humans have to apply judgement into what they must take in and how they architect their systems.

Disclaimer: I contribute to all projects above. I am a rust developer by trade with some skills in python.