open source models are getting cheap enough that your bottleneck isnt the model anymore

seeing a lot of hype posts about new open source models being free or 10x cheaper and yeah thats real

but once you actually try to ship something with them the thing that hurts usually isnt model quality or api price

its everything around it

context bloat is the first silent killer people get excited about huge context windows then they start stuffing whole docs and chat logs into every call it works in a demo then latency gets weird and costs creep back in and the model starts missing the one line that mattered

if youre building an agent or any workflow that calls the model more than once, the cheapest win is being ruthless about what you send not smaller for the sake of it, smaller because you know what the model needs to do next

a simple check thats saved me a bunch of time before you add more context, write down what the model is supposed to decide in the next step if you cant say it in one sentence, youre probably about to send a pile of text and hope it figures it out

second thing that breaks is evals when models get swapped weekly, you stop trusting your own system one day it feels smart, next day it starts hallucinating tool args or gets flaky on edge cases if you dont have a tiny set of real prompts you rerun every time you change a model or a prompt, youre flying blind

third is the glue code tool calling, retries, timeouts, rate limits, parsing this is where the hours go and its also where most of the user pain comes from because they dont care what model you used, they care that the workflow didnt finish

so yeah cheaper models are great but the teams that win with this stuff arent the ones chasing the newest weights theyre the ones who treat context like a budget, keep a small eval set, and make the boring reliability stuff solid

curious what part has been the real bottleneck for you when you tried to build on these newer cheap or open models