LLM limitations

I was discussing LLM limitations recently with a colleague who is also an IT professional, probably more so than I.

And we came to a conclusion that one of the problems with AI code generators, and one huge advantage of humans in the field of complex text generation - humans can shut the f up.

LLM is a word predicting machine. It knows nothing, it thinks nothing, it has a context, a prompt and it slaps together the most likely words. And whatever suite of complex machinery you build around it, inside it's just that - a word predictor.

As such, LLM cannot not talk. It cannot shut up.

If you ask it to write a comment, it'll be a long comment, even if all that is needed is 3 words. If you ask it to write specification, it'll mix together bird view, specific details, irrelevant declarations - until the stream of words is exhausted.

Humans make specifications as short as they possibly can. If something can be explained in 3 sentences, human may write 4 or 6; but LLM happily churns 33 and a diagram; and another 33 if you ask it to improve it.

And hallucinations are just the ultimate example of that. If I don't know who wrote "Gone with the Wind", I will not try to guess or invent; I simply don't know. I have no answer, and that's it. LLM, on the other hand, can't have that. That's why we have these weird verbose replies to one word queries on Google. It's not that LLM is stupid, it's that it can't shut up when there's nothing to say.

It's quite a huge disadvantage in any knowledge-centric environment, including IT.