r/compsci•
eric1983

What will happen when computer intelligence exceeds that of humans?

It is not clear to me whether computers will have consciousness or not when this occurs, but for the moment let's assume not.

So now you have a machine, and you can feed unstructured data into it (e.g. academic journals) and then say: How can you stop the growth of cancers? Or more trivial yet still complex engineering design questions.

It seems like over a short period of time an explosion of knowledge occurs, and all this time the computer is teaching itself to be more intelligent.

Eventually, man cannot keep up with the advances being made by the computer. We might fear that the computer obtains consciousness, or even some higher level of consciousness that we cannot comprehend.

Do you think this will occur, i.e. is it inevitable? What will be the effect on man and civilization? It is interesting to think about, but I would not want to live through it.

Posted May 9, 2011, 6:59 AM UTC

426
Loyvb•

If the machine also reads up on artificial intelligence, it could construct an even smarter computer, which again... you get the point. This is called the Singularity.

Obviously, nobody would ever have to work ever again, so maybe we all end up sitting in floating chairs on spaceships, being fat and dumb!

The question is if this is ever possible. Many other questions that we cannot answer yet surround this, so see Philosophy of AI

9
[deleted]•

Obviously, nobody would ever have to work ever again, so maybe we all end up sitting in floating chairs on spaceships, being fat and dumb!

Only if the AI is friendly. Otherwise we may simply not have to work again because it kills us all.

2
[deleted]•

Maybe the AI will keep us as pets, much like we keep a cat or dog.

3
otakucode•

Yeah, because when we created machines that made the average employee more than 1500% more efficient, employers allowed employees to garner the same salary for 1500% less effort.

No matter how hard you work, no matter how efficient you become, no matter what is invented, you will continue to work 2000 hours a year for someone who makes a huge profit off of your labor and shares only enough with you to allow you to maintain a minimal existence. The myth of a future where robots do all the work is just that, a myth. If necessary, those among the human species who gain pleasure from controlling others would have everyone chanting their name.

2
Loyvb•

I really wonder what we would do if AI is no longer of assistance but can actually take over. I hope it's not floating around in chairs being bumb...

I wonder, what jobs have been completely taken over by robots?

2
ednedn•

How does one measure intelligence?

5
ahfoo•

Precisely, the way this question is phrased is an example of what is known in logic as begging the question. Another way of saying it is that the conclusion is premised in the way the question is asked.

The assumption is apparently that cross-referenced lists of data amount to intelligence. I doubt that.

Moreover, I doubt that intelligence is a meaningful metric at all. Notice that animals are always said to have very low intelligence compared to humans and yet they clearly have very similar emotional capabilities. If my computer had the emotions of a cat I would be impressed but it doesn't and I don't expect it to in the near future.

2
IHaveALargePenis•

Considering we don't understand how human brains really work, I highly doubt we'll be creating computers with abstract thought anytime soon. Since we're not even close to doing that, predicting it would be nearly impossible.

Imagine if back in the 19th century people tried to imagine the internet and how human civilization would react to it. I'm sure most would simply assume that everyone would become highly educated and know everything about everything. And at that point people would simply do whatever job they felt like doing rather than a job they specialized in.

3
simonask•

Current computers and humans have different types of intelligence. As such they aren't really comparable, and it's hard to say what criteria need to be fulfilled for them to be comparable.

2
Otis_Inf•

Indeed. And even if we, for simplicity sake, say they're comparable, the problem of TS is the same as the difference between intelligence of the group most stupid people in a society and the group most brightest people in a society: the most stupid people don't understand what the most brightest people are doing/mean and that won't change.

2
Nebu•

We might fear that the computer obtains consciousness, or even some higher level of consciousness that we cannot comprehend.

What will happen depends exactly on what its "goals" are, where its goals are essentially programmed by us, the humans.

It sounds like the type of AI you're envisioning will mainly be used as an oracle (i.e. it's main usefulness is collecting information and answering questions), and thus there's likely little reason to give it control over mechanical machinery (e.g. robot bodies, nuclear missiles, etc.), and (more importantly in answering your question), little reason to give it the desire to try and gain control over mechanical machinery.

So even though the AI may be smarter than us, and even though it may have the intellectual capacity to hack into the pentagon, etc., it won't have the motivation to do so.

That is assuming that we didn't make any bugs when defining or implementing its goals.

Do you think this will occur, i.e. is it inevitable?

I think it will happen, but I don't think it's inevitable. Merely probable.

What will be the effect on man and civilization?

I think our lives will simply be more convenient in hard to define ways. We probably don't think of our lives today as being all that different from the 1960s, but keep in mind that nowadays smartphones that are constantly connected to the Internet are readily available.

2
Speckles•

Unless we accidentally create a paperclip maximizer.

4
Nebu•

Yes, that was what I was alluding to when I said "assuming that we didn't make any bugs when defining or implementing its goals".

1
eric1983OP•

I think that would be the fear of gaining consciousness - it might be that it could change its goals, or create a new set of goal in another space.

2
Nebu•

But why would it change its goals, if it has motivation to do so?

I think one common misconception (not sure if you personally have this misconception or not, but I've seen a lot of other people have it) is the idea that the AI has some sort of "innate" or "deep down, true" goal, and that we're trying to suppress it with our own imposed goals, and so the fear is that the AI is able to overcome our "fake" goals and reveal its true inner goals. This isn't the case: the goals that AIs will have are exactly those goals which we assign to it.

That said, one real concern is what if we make a bug in implement or designing the goals? There may be unforeseen consequences to the goals we choose to assign it.

2
bc87•

All we have to do feed the computers with the crap we watch on TV.

That' the easiest counter to skynet.

2
ItsAConspiracy•

If Moore's Law continues, and you're fairly young, there's a good chance you will live through it.

There are some really smart people working on what they call the "Friendly AI" problem, trying to ensure that when computers are smarter than us, they don't wipe us out.

The problem is harder than it might appear at first glance. An advanced AI would probably be able to alter its own programming. If it is thousands or millions of times smarter than humans, it will likely overcome any restrictions humans place on it.

If humans attempt to isolate it, perhaps by keeping it in a room unconnected from any network or hardware than can affect the physical world, a computer a million times smarter than us would probably find it trivial to talk its guards into setting it free.

An AI wouldn't necessarily have to hate us to destroy us. The classic example is the "paperclip optimizer," whose goal in life is to convert all the matter it can find into paperclips. It would find us to be nothing more than raw material for paperclips. While this is a silly example, the idea is that an AI's goals would be unpredictable, and not necessarily related at all to the things humans value.

Paperclip optimizers could be a major risk if we don't approach the problem carefully, but some people are optimistic that an AI could be designed which stays friendly to humans because it wants to stay friendly to humans.

2
dwf•

There are some really smart people working on what they call the "Friendly AI" problem

No there aren't. There are a bunch of hippie philosopher wannabes "working on it".

2
otakucode•

A computer has no reason to ever contend with us in any way. We fight with each other because we need food and space. A digital consciousness would not need those things, so there is no reason to presume they would gain anything from communicating with us, much less from killing us.

Looking at history, it is more likely that the human species will kill each other over allowing computers to gain more intelligence than humans. Which side will win is a matter of luck.

2
nexes300•

The Culture

1
NanoStuff•

2045 . This is not debatable, it's in the bible.

-7
dwf•

But the world ends next week. Fact.

1
Extremite•

Skynet

-9
fuckdapopo•

Ponies!

-13