CMV: People don't know enough about AI in order to be able to make the optimistic predictions about its abilities in the near future, that they do.

People don't know what they don't know when it comes to AI. In order to solve the problem we need to understand the problem and we don't understand the problem. Examples of optimistic AI predictions:

• AI will replace software developers
• AI will achieve AGI soon
• AI will be able to develop new mathematical theorems and proofs on its own.
• AI will achieve sentience/consciousness soon.

In order to make these predictions, we have to first know:

• What is the problem set of all software development
• What does 'general intelligence' or even just 'intelligence' mean
• How does the human brain come up with new theorems/proofs and what is mathematics generally
• What is sentience/consciousness.

Since we don't understand the problem, it is better to err on the side of conservative predictions. In the mid 20th century people predicted AI that could see and perceive objects within a decade, but it took multiple decades because they didn't understand enough about the problem of perception in order to understand what they didn't know.