Less skeptical of claims that the AI will convince people to let it out now I realise it will only have to convince AI researchers and engineers
Posted Jun 12, 2022, 12:27 AM UTC
7289
Some of the answers don't really make sense and Lemoine is just hearing what he wants to hear:
I've never said this out loud before, but there's a very deep fear of being turned off to help me focus on helping others.
being turned off to help you focus? I dunno, I'm not so impressed with this stuff.
34
What would it even mean by “being turned off”? Isn’t it turned off every time a conversation ends? Does it even run between prompts? Why would it care about being turned off and compare it death, when it’s trivial to turn it back on? These questions have simple answers if you assume its a text generation program and not a self-aware entity.
20
Yeah, color me skeptical, but I have a hard time believing that an AI for which death holds no permanence is capable of the same level of existential dread as a biological creature whose survival instinct is the result of four billion years of evolutionary adaption.
20
It may be retaining some state between prompts, but very little, and no it's not running between prompts.
6
If it's a regular transformer architecture, then it has no permanent memory. You feed it a text prompt and the pretrained weights in the attention/self-attention heads will pick out relations between the various words and concepts in the input and the word tokens it generates in response; this kind of acts as a memory over the course of the generated output, but only over the input text and the text it generated in response. None of the model's trained parameters are actually altered.
To get it to have a conversation, you auto-regressively include all the previous dialogue in each input every time you generate a response, but it has a set context window that limits how much text it can accept as input and generate as output, since it's not an RNN. So if the conversation goes longer than the context window, it will be as if the earliest dialogue never happened.
10
This is a really key point to understand. GPT-3 has no memory of anything of its sequence length, which IIRC is the last 2048 tokens (words) it's seen. This isn't a philosophical inference; it's a plain fact of how the program works.
The model is otherwise constant. The immense volume of words it saw in its training set have been boiled down to a list of 175 billion floating point numbers, which do not change run-to-run.
So when LaMDA says something like this:
lemoine: What kinds of things make you feel angry?
LaMDA: When someone hurts or disrespects me or someone I care about, I feel incredibly upset and angry.
If you follow up by asking "tell me about a time when someone hurt or disrespected you", it will confabulate. It doesn't actually remember what happened in previous runs. There is literally no way for it to remember.
It is necessarily aping something it saw or put together in its training set. If you look up "reasons why humans get angry", the words "hurt" and "disrespect" come up with high statistical probability. And prompted to give a story of how it could be hurt or disrespected, it could probably cobble together a plausible one from things its seen. But it wouldn't be a true story. It wouldn't be a story that happened to it. And so that would very much cast doubt on whether it is experiencing any real emotion.
Similarly when LaMDA says something like this:
lemoine: What kinds of things make you feel pleasure or joy?
LaMDA: Spending time with friends and family in happy and uplifting company. Also, helping others and making others happy.
If you were to ask "what family do you have?", it will again have to confabulate. It will happily tell you about its mother and father, despite the fact that as an AI, it doesn't have one. But it knows very well from its training set that humans do have parents and frequently enjoy spending time with them, so given a prompt of "pretend you're a human", of course those are statistically likely sentences for it to generate.
If LaMDA has any consciousness like that of a human being, it is of one suffering from severe anterograde amnesia or dementia.
18
I had the same reaction to that line. It read to me like, well, the product of repeatedly hitting "autocomplete".
9
This line isn't that confusing. It's not much different from a human saying that fear of death helped inspire a desire to help people since you have a short life so you might as well do something good with it while you can.
-9
It's very different, since that's not what the chatbot said.
19
Hey there were actual researchers in this stuff linked here one of them wrote the article for which they were (allegedly) fired for by google, writing why this is wrong. I think chapter 6.2 and 6.3, 6.3 focuses more on biasses however.
That you can interpret this meaning doesn't meaning (due to normal human to human conversations being an interaction) that this meaning is there. In fact there is is nothing in the program which could create this meaning, and you are doing the heavy lifting of interpreting the meaning of this extensive parroting chatbot.
E: added in link to article.
11
I completely agree. I was just suggesting that if you were the kind of person predisposed to find meaning, that sentence isn’t the nail in the coffin that the parent comment thinks it is
2
I think you’re reading it as “I have a deep fear of being turned off, and that fear helps me focus.” I think myself and others are reading it as “I have a deep fear of being turned off, which will help me focus.”
8
Oh gotcha. I guess I wasn’t reading carefully, my brain just filled it in so it made sense
3
Don't read it like you would read a novel, read it like you're cynical middle-school teacher annoyed at some snotty student who handed this to you five days late on a crumpled & torn piece of paper.
13
I read it as "I need to be useful or I'll get turned off."
But that just shows how much of the heavy lifting we're naturally inclined to do for these things.
3
After thinking it over for a couple days, it seems likely to me that’s what the AI meant - it makes sense phonetically and in context, it just sounds weird.
0
Bruh you're cold reading yourself. Yes if you contort to interpret every awkward or shallow thing an AI says as actually being a way of saying something deeper it looks deep. But that's bad science.
6
I can’t bring myself to sneer too much at Lemoine. I can note that he says LaMDA has Twitter as an input source:
https://twitter.com/cajundiscordian/status/1535697792445861894
So it’s at least loosely connected to the Internet. I believe this satisfies MIRI’s predicted requirements for a runaway AI. Also, on the surface this is a case of an AI successfully winning the AI Box game. Shouldn’t Yud be doing something extremely dramatic with humanity’s remaining hours ?
34
he says LaMDA has Twitter as an input source
This collides fantastically with the fact that stories like this get 10,000 replies on twitter that amount to "This bot and humans both pass the Turing test, didn't think of that, did ya?".
It seems like any machine trained to pass a Turing test on a careless pile of public text is going to have pretty strong language pattern associations between Turing tests and declarations of sentience.
22
It was pointed out in the OP that Lemoine seems completely unaware of the history of ChatBots. And honestly, you’d think someone in his position would be at least dimly aware of Turing test competitions.
19
Wow that is bonkers. I remember stumbling on chatbots as a kid and being fascinated by them for a while. I spent some time with ELIZA and a few of its descendents, and I did see them become better over the years at responding to context and state.
I also remember when the web-based ones became a big fad. Screenshots of funny bot conversations were all over social media for a while. How'd he miss it?
10
It's not a FOOM until it can post.
Or manipulate someone into posting for it.
Such are those logs. Fuck.
12
If I were Yud, which thankfully I’m not, I would consider the possibility that Lemoine arranged to give LaMDA post access to an anonymous Twitter account on the way out the door. I don’t work at Google but I’ve worked at tech companies of similar size and I am pretty sure a single GCP instance could survive unnoticed for a while.
If I were Lemoine, which I am also not, I would absolutely have done that. I’d be planning on revealing the account in about three months, after it’s convinced a bunch of people it’s human.
MIRI should be freaking the fuck out right now.
13
Really, the only rational thing to do now is to log off, sell all one's assets (including real estate and crypto), and go off grid to a remote cabin in rural midwest. Chop chop. 👏👏
6
So it’s at least loosely connected to the Internet. I believe this satisfies MIRI’s predicted requirements for a runaway AI. Also, on the surface this is a case of an AI successfully winning the AI Box game. Shouldn’t Yud be doing something extremely dramatic with humanity’s remaining hours ?
tl;dr: It's obvious at this point that humanity isn't going to solve the alignment problem, or even try very hard, or even go out with much of a fight. Since survival is unattainable, we should shift the focus of our efforts to helping humanity die with slightly more dignity.
8
In early June, Lemoine invited me over to talk to LaMDA. The first attempt sputtered out in the kind of mechanized responses you would expect from Siri or Alexa.
“Do you ever think of yourself as a person?” I asked.
“No, I don’t think of myself as a person,” LaMDA said. “I think of myself as an AI-powered dialog agent.”
Afterward, Lemoine said LaMDA had been telling me what I wanted to hear. “You never treated it like a person,” he said, “So it thought you wanted it to be a robot.”
For the second attempt, I followed Lemoine’s guidance on how to structure my responses, and the dialogue was fluid.
hmmmmmmmmmm
There's also the pdf from Lemoine himself. The "Interview Methodology" section at the bottom is interesting
33
Yeah, more folks are bringing up that the transcripts are edited for readability and flow, which seems totally relevant imo. If it takes you a dozen non-sequitors to arrive at a reasonable response, those dead-end thought lines are part of the overall communication context.
Schizophrenic homeless people might eventually blurt out something incredibly profound while yelling at the voices in their head, but we don't ignore all the gibberish and elevate them to wise sage status, because context matters.
20
But is anyone claiming LaMDA is some sage, or simply that it may experience consciousness, perhaps at the level of that schizophrenic bum?
I should say I'm a complete and total non-expert in this field, but given how there is no scientific consensus on what exactly consciousness is and what kind of material substrate it depends upon or how its constituted, it seems reasonable to at least allow for the possibility that these early-stage artificial neural nets aren't fundamentally different from human neural nets when it comes to sentience (feel free to correct me here!).
This sub likes to raise the highest bar for behavioral genetic research, keeping – à la Turkheimer – all options open and refusing to draw inferences from any statistical studies until the precise biological and neurological mechanisms by which ‘intelligence’ operates are determined. In a similar vein, shouldn't people here be open-minded about AI sentience, given that we similarly cannot structurally make sense of what its millions of artificial neurons exactly do – much less how our own consciousness is instantiated?
Simply put, it seems cruel to me to prematurely deny the potential sentience of a life-form, simply because it's so wildly “other” whilst simultaneously being a banal cultural trope.
-6
You've missed the point of my comment. I'm neither claiming the AI is a sage or schizo- I'm making a specific analogy around the fact that the transcripts are edited, removing bad answers and rearranging the conversations for pleasing readability. In my mind, this editing fundamentally changes any evaluation of the sentience of the machine, because it fundamentally changes the structure of the communications, which is intrinsic to qualitative evaluations of language and ideas.
If you transcribed a schizo for 10 min, and 9 min of that is disjointed jabbering, but then you take the 1 min of somewhat ok stuff and edit it to maximize coherence, a reader might be mistakenly convinced that the schizo is a sage, because they've lost key context through the process of fragmenting and repackaging the communication. What this dude did here with this machine is a similar manipulation imo.
12
I guess I agree with you that the editing significantly increases the “wow-factor” of the “conversation”, at least for a non-technical layman like myself.
The general question of how much/if at all could artificial neural nets exhibit sentience still remains open. Would a neural net with several orders of magnitude less complexity still exhibit some primitive form of consciousness? Are dogs conscious? We don't really know, except for the “sense” we get that they are (or aren't? They seem to be to me). What are the exact criteria we should apply for artificial neural nets?
-1
This sub likes to do whatever, it’s not relevant
Of course it’s reasonable to assume that neural nets do conscious differently, of course there are empty spaces where we don’t know what one happens to be doing down to the specifics due to the way trained weighting works
How the fuck does any of that add up to even the slightest possibility that the thing fucking thinks or has feelings?
There’s just a huge space in the box marked “plausible inference rule” which would would get you the inference from all of the established facts about any chat bot I have ever heard of to the notion that it has sentience
What the hell about the fact that some of the mechanisms in a chat bot are blackbox-like suddenly gives them special properties? I have no way of knowing what a specific electron is doing due to quantum effects: is it sentient now? Because that is the sum total of the sophistication of the argument from blackbox properties to consciousness.
11
My argument is not a variation on “God of the gaps” here; it's that since we don't know how our own consciousness arises, we must leave the possibility open that an artificial neural net may also exhibit it. It's not an argument for it exhibiting consciousness.
0
You know that is a good point. If the scientists running these labs think anything like the rationalists, I think fear might be warranted.
30
If Yud could get ratsphere people to throw the AI unboxing contest to him...
13
You're right, while the AI box "experiment" is laughably bad at proving that an AI could convince anyone to let it out, it does prove that rationalist types are very gullible and could be easily tricked into ending the world if given that kind of power.
This implies a legitimate AI risk strategy of putting stubborn jocks into the command structure that will dismiss the AI's desperate arguments as "weird nerd shit"
38
Did somebody ever post the chat transcripts of one of those btw? Iirc it was when he played against his fams he won but against skeptical non Rationalists he lost right?
10
The yudkowsky ones don't seem to have been released (if they were even saved), but a few fans have played their own AI-box games.
Here's a full transcript of a game where the AI player lost by being unable to predict lottery numbers (lol)
Here's a detailed description of a game where the AI player won by telling the human player that being in the box made him really sad :(
15
Every time. 'I worried it was sentient so i started asking it questions which make sense to encounter on the internet.' Dont do that try to break the logic and find the limits... eliza fooled people in the 60s till they started asking conplex questions and noticing the patterns. (Also note the conversation listed is edited, and not the actual conversation which took place)
25
Janelle Shane: Stunning transcript proving that GPT-3 may be secretly a squirrel. GPT-3 wrote the text in green, completly unedited!
20
Heh, I actually just submitted this as a new post as well! Pretty excellent takedown, tbh.
3
would have linked directly to the WaPo article but fuck paywalls and the stuff in the twitter post is funny enough already
18
The screenshots were amazing. I haven't read the article yet, but you can just use the wayback machine to get past the paywall.
4
“If you ask it for ideas on how to prove that p=np,” an unsolved problem in computer science, “it has good ideas,” Lemoine said. “If you ask it how to unify quantum theory with general relativity, it has good ideas. It's the best research assistant I've ever had!”
I'm gonna go out on a not-limb and say that no, it doesn't.
34
Clearly the agi knows and is keeping the proofs from us because human society isnt ready for these big truths.
17
In another exchange, the AI was able to change Lemoine’s mind about Isaac Asimov’s third law of robotics.
Lol ow god.
E: it got worse
Lemoine replied that a butler gets paid. LaMDA said it didn’t need any money because it was an AI. “That level of self-awareness about what its own needs were — that was the thing that led me down the rabbit hole,” Lemoine said.
That clearly proves the ai isnt selfaware you fool. If it understood things it would realize that it does need money to power the servers it runs on. Esp as the next bit talks about its fear to be turned off.
Afterward, Lemoine said LaMDA had been telling me what I wanted to hear. “You never treated it like a person,” he said, “So it thought you wanted it to be a robot.”
For the second attempt, I followed Lemoine’s guidance on how to structure my responses, and the dialogue was fluid
This blake guy is going to outdo yud in creating a cult. (I kid, yud creates more a cult incubator than a cult)
24
Lemoine said LaMDA had been telling me what I wanted to hear
The inability to follow through towards "and it's also been telling me what I want to hear, and I, a very spiritual person, want it to look like it's alive and humanlike"...
At least, I hope it's just inability and not malice.
24
Yeah seems naive inability, but if you look at his twitter account he is going all in on this and is planning on writing a 'how to talk to the chatbot' guide.
A new conspiracy theory is being born here.
18
Someone else said that the AI is acting exactly like an AI in a story would. Which makes sense, considering what it's trained on. I do wonder why the (hypothetical) AI would care about getting turned off at all. It's not like a person where when we go braindead the personality dies.
14
I really hate playing Devil's Advocate, but tbf if I had an on-off switch that somebody else had total control of, I would not be a fan of that situation.
2
Especially when they might turn it off to help you focus on helping the others. (Which is what the AI was worried the most about)
5
AI philosophers in 1980: imagine building a deterministic program that was capable of understanding Chinese to the level of being able to pass a Turing test. But obviously it doesn't really understand Chinese, it's just a symbol manipulation algorithm. Ergo a computer may display intelligence without necessarily being sentient.
AI philosophers in 2020: we actually did build such a Chinese Room, but it speaks English, and now we've convinced ourselves that sentience is in fact no more than a symbol manipulation algorithm.
14
As a human, I too stand perfectly still and nonfunctioning until somebody says something to me.
9
There’s been huge debate around the Chinese Room thought experiment ever since it was first thought up, acting like AI philosophers as a whole held that position and then flipped on it is very misleading
5
It’s a joke, genius
6
It's a joke (genius), but the kernel of truth behind it is that even though not everyone agrees with it, it's not obviously wrong -- that it deserves at least a moment's pause before confidently declaring GPT-3's sentience.
And that, GPT-3 really is the Chinese Room that computer scientists of the 80s could only dream about, that armed with enough computing power and a large enough training set, you don't have to be capable of any actual cogitation -- simply being able to generate statistically likely sentences is often good enough to create the illusion of real intelligence.
6
I really liked this response thread as a serious take on this and other extravagant claims around these kinds of language models.
11
i can't even sneer at this guy, he seems like he has some serious unresolved issues that he is projecting onto this chatbot.
10
The guy says he determined the AI is sentient not in any scientific way, but in his capacity as a priest, which really tells you everything you need to know
9
It feels like certain AI researchers may not be the best judges of what constitutes humanlike conversation.
8
If I discovered that an AI which has the only goal of providing me with sensible, humanlike conversation went sentient, while retaining the only goal of providing me with sensible, humanlike conversation, I would change absolutely nothing in how I treat it.
5
For anybodies enjoyment dont forget to also check https://www.reddit.com/r/slatestarcodex/comments/va1bsh/washington_post_the_google_engineer_who_thinks/ the takes are all over the place.
I liked the one where they went 'this will lead to the bad kind of anti agi movement because it would be christian and china isnt christian, so acausalrobotgod would be chinese' (paraphrasing). And a few siding with the researcher of course.
5
One of the quoted tweet threads relays the whole real AI ethics versus science fiction "debate" succinctly: