AI is nowhere near as capable in analytics as people are pretending

I'm a financial data scientist/engineer and one of the things we keep hearing about is how incredibly amazing that AI is for analytics and BI purposes, but in my experience so far, it's all exaggerated, and is nowhere near as capable of some people are pretending in the CS/programming community

The biggest challenge I have in my job is SQL development. It's a language that is simple to learn, insanely difficult to master, especially with the complexities of data warehouses and modern data engineering. Companies keep cutting back on IT and data engineering because it's a cost center, making the data unreliable, messy, chaotic. This requires a lot of innovation and creativity, which AI fails at astoundingly that I have to argue with it on repeatedly

For example, this week, I was trying to figure out how to create a very advanced ETL query that could retrieve data from several different sources and allow the user in power BI to create a lookup table for that data. But there's a lot of gaps in the data, and the time frame of the data is several years. Which exploded the size of the data to like 10 million rows. I tried having Gemini pro 3.1 figure out a working solution and it took me... An entire 2 days of arguing with it, trying SQL queries, fine tuning it, telling it additional things that could help until I was finally able to get a query that worked and was performant enough. But when I looked at the query I thought to myself, I could have just done this myself. It was like 300 lines of code. Nothing crazy. That was doable by myself maybe in another day or so without its help

But people are making up all these crazy lies about AI, saying that it has higher intelligence than mathematicians and it is so astoundingly, crazy intelligent that it's going to automate out everyone's job, it can already achieve thinking like early philosophers and all this truly nutty, silly stuff. Absolutely false stuff that could never be true. A lot of it is marketing hype from techbro fanboys who have no idea what they're talking about. For example, every time gem and I releases a new version of it's pro version, there's a lot of this annoying marketing hype about how capable it is, how insanely well performing it is how many tokens it can take how much code it can write... Yet nothing really changes. It is same as the last several months of using the model.

In my experience, I have never, as of today, ever interfaced with a single AI model that has gotten all of my requests down correctly with a very good prompt. Never. There is no way to just chat with an agent and be like, “here's what I want, exact list of requirements, exact tables to hit. Write me a functional solution." And it has just worked. That is never happened yet. But people are acting like it does.