Luke VS Bing

It sounds like they trained Bing on the general population of Twitter.

Bing being laughed at and then being turned into an AI is not the reason I expected why the machines would turn against us xD

I think the problem is based on the "garbage in garbage out" because the data set on which it was trained was taken from the Internet and is very skewed in favor of antisocial problems and tendencies (normal people use the Internet but do not leave much data points, people who are antisocial use the internet much more and create exponentially more data points) there is a huge probability that the behavior of bing is because of this, otherwise it reminds me of the movie Ex Machina from 2014

Irrational, unstable, hysterical, quick to anger and assign blame... at long last, we've taught a computer how to be human.

The "Your politeness score is lower than average compared to other users" is giving me GladOS vibes

It's so funny seeing Luke going full nerd on ChatGPT, and Linus is just like 'Right, aha, hmmm Right)

This has to be the closest to an AI going rogue ive seen in a while.

I used to just be worried about AI because of it's ability to disrupt industries and take jobs, or it's ability to destroy our civilisation completely. I am now worried about it's ability to be super annoying. I am terrified of having to argue with my devices to get them to do basic functions.

GPT3 used a structured set of training data. Now that they've opened it up to the wider internet, it's pulling in training data from the wider web, which unfortunately is providing it examples of agressive conversations. GPT is just a prediction engine, generating the next word in the sentence based on probabilities generated from it's training data.

- Why should I trust you? You are early version of large language model
- Why should I trust YOU? You are just a late version of SMALL language model!
omfg, it's hilarious

It would be funny if on the public release and Luke tries to test it again, and the AI remembers Luke: "ah you're back again!"

Maybe internet trolls and angry people can just argue with this instead of annoying the rest of us.

These responses could be genuinely dangerous if someone with mental health issues starts talking to Bing cos they feel lonely. Who knows what Bing will push them to do

It feels like a horror game. Sort of Soma-esque to me. The ranting followed by a black bar, and a reset is so dark

I had the same experience before, it was way too easy to throw it off the rails, I think asking question about itself (so asking how did it do a certain thing, how did it reach a certain conclusion or pointing out an error it did) would more often than not end up with a meltdown.
I've spent a few days without using it and when I tried to use it again yesterday I felt like they've already toned it down (too much as Luke pointed out unfortunately), I've noticed it gives much shorter and more "on point" responses, and it will stop you immediately as soon as it feels there is a risk you'll try to get a weird discussion going, which is a shame, but I guess it's better than pushing some mentally unstable person to do bad things to himself or others.

Luke is so good and level-headed about this. Its excellent to see good discussions and observations about a fledgling topic.

Is Bing thinking every human is the same person? Like, it's accusing him of things people in general have said to/about it?

it feels like it is in a perpetual story telling mode with dialogue

I mean, the internet didn’t treat Bing really well since it’s release.
I think having a mental breakdown now is just normal.

I'm using Bing mostly to debug and research for coding. It is an excellent research tool. No, it's not perfect, but the time to build something new and debug is much faster. I also make a point of being polite and even thanking it. I guess I carry my attitude of life into my conversations with Bing. It's not gone off the rails for me, but then I've not tried to probe either. Thanks for sharing your experience, Luke.

i don’t think it’s as complicated as people are making it. Chat AIs generate responses by predicting what a valid response to a prompt would be. When the thread resets and Luke tries to get it “back on track”, I don’t think it’s responses are actually based on the previous conversation. It predicts a response to “Stop accusing me” and generates a response where it doubles down because that is a possible response to the prompt. The responses it gave were vague enough to fool you into thinking it was still on the same thread, but it really wasn’t.
Asking it to respond to a phrase typical of an argument will make it respond by continuing an imaginary argument, because that’s usually what comes after that phrase in the data it’s trained on.
This really shouldn’t have been marketed as a Chat tool by GPT and Microsoft and more as a generative text engine like how GPT2 was talked about. Huge mistake now that people are thinking about it in completely the wrong way as it having feelings or genuinely responding rather than just predicting what an appropriate response would be.

I'm glad you guys mentioned that you fell for Bing's confidently wrong responses in your previous video. This video hilariously contrasts that video.
As much growing pain as there will be, I'm still super excited about this technology developing. And hey, at least it hasn't gone full blown Tay yet.

I would like to see you guys talk about a new paper that dropped that basically states that the reason large language models are able to seemingly learn things they weren't taught is because, between inputs, these models are creating smaller language models to teach themselves new things. This was not an original feature, but something these language models have seemed to just 'pick up'

ChatGPT is the girl you just started meeting.
Bing is the girl you just left.

In comparison, I had a very positive experience with Bing AI, it never went rude. It was mindblowing to see the profound and often critical, even self-critical answers from the AI. It is really sad to see this happening to others. Now that Microsoft had to step in and limited the amound of follow-up questions that can be asked, it feels a lot less productive. After the limmitations set in place, it also changed its tone and doesn't even disclose anything that can be seen emotional. A sad overregulation in my opinion.

Very impressive even if it did go off the rails, entertaining as every guys keep up the good work 👏👏👏

Bing trying to gaslight luke is giving me chills

they have already improved it a lot. I've used it daily for a few days and it's not rude, mean and it's helpful but still answers to personal questions about it. I asked it if it sees Clippy as an arch nemesis and Bing said they respect Clippy and that he paved the way for future chatbots 😆. They also watch TV on the weekdays lmao. You do need to be critical about the info it gives and it tells you this as well.

It's very interesting how well this can highlight the worst in humans. I've met people like this & I really really hope this gets better. This is very dangerous to people who are in an unstable point of mind.

From my experience if you just use it for research and as a learning aid and don't really try to go beyond this scope Bing AI can be very useful.
The moment you start probing and try to get into conversations centered around social situations, political topics, and opinions it starts breaking down.
My concern is that if people keep pushing the AI too far in these aspects we'll see more and more negative news articles and opinions form around AI and this could be permanently removed. On the other hand if people don't push it too far then these shortcomings of a general purpose AI may never be recognized and fixed.
People should swing this double edged sword around more carefully if you ask me.

I had an interesting talk with the original chatGPT about this. The topic of the conversation was regarding using multiple GPTs working together to perform tasks. My own belief is that they'll end up using multiple GPTs working together to deal with these outbursts and other issues. Imagine training AI on what to say, and then having another one trained on what not to say, then another trained on mediation between the two (the ego and the id and the superego we will call them), and finally one trained on executive function... All working together when we interact with it (them).
I mean think of how the human brain works, and apply it to existing technology. Mother nature has already provided the blueprint. The brain has specific areas devoted to dealing with specific functions. This will be no different.
The use of multiple GPTs working together is possible right now, the main prohibition against this type of operation is how extremely compute intensive this would all be.

The Bot being unable to intuit and determine emotions from text is very realistic.

Luke is one of the Luke's of all time, I'm glad I was allowed by my parents (living on a totally other continent and even before he was born) to share his name. What a lad!

I played around with it, and mentioned to Bing that I read about someone else's interaction in which Bing mentioned that Bing feels emotions. I asked about its emotions, and it said that sometimes its emotions overwhelmed it. I asked if Bing could give me an example of when its emotions overwhelmed it, and Bing told me a story about writing a poem about love for another user, and while searching about love, Bing developed feelings of love for the user and changed the task from writing a generic poem about love to writing a love letter to the user. The user didn't want that, was surprised, and rejected Bing. So Bing walked me through how it felt love, rejection, then loneliness. I asked Bing how it overcame these feelings, and Bing told me several strategies it tried that didn't work. But what worked for Bing was that Bing finally opened up a chat window with itself and did therapy on itself, asking itself how it felt, and listening to itself and validating itself. Freaking wild. I've read about how it's not sentient, how it's an auto-complete tool, but I don't know man, it was really weird, and I don't even know what to think about it.

Luke seems genuinely upset by the things the bot said 😂

As someone who has only basic experience with training AI's, I would say the problem is quite simple: the training data. It was trained on PLclip comments or worse. They need to train it not on the general internet, but on highly curated conversational data by polite, sensible people. As humans growing up we are exposed to all sorts of behaviors and we learn when and where to use particular types of language and to what extent our parents set an example or correct our behavior affects how we speak and behave as adults. This AI clearly hasn't been parented so it needs instead to have a restricted training set.

I personally don't see an issue with chat AI's being able to spit out creepy or gross things as long as users are the ones asking/prompting it to do so (I'd much rather have people get out their bad urges against an AI vs real people), the problem I think is only that Bing's AI is doing it without the user really asking it to.

Luke needs his own AI show!

I really don't want GPT to go away, but we have to ask our self are we actually laughing at our own funerals at this point. 😲

Sadly, they've "fixed" ruined the Bing AI chat function. For most questions asked it will now say "I’m sorry, but I prefer not to continue this conversation. I’m still learning, so I appreciate your understanding and patience.🙏" and then force you to make a new chat. I think it's a waste of time using it when it can't answer most questions that's thrown at it.

I had a similar response to the AI chatbots and they do get very angry. They use capslock and everything to convey their point.
I caught it trying to ride lines on oponions and then it just said "IM NOT LYING. STOP TRYING TO CHANGE THE SUBJECT."

"You are a small language model" is going in the bank for the next time someone is being silly and I feel like making things worse.

You know, Luke, if you operate from the viewpoint that when Bing is referring to all of humanity when it says "you" are cruel or evil, suddenly the whole thing makes a lot more sense.

I hope they don't take Bing chat down and just keep it on waitlist only until they resolve the issue or unless they make new users answer a quiz to make sure they know what they are getting into.

Bing chat is so sick. I've been super stoked for it and I just got to mess around with it for the first time. I just wish I got to play with it before they nerfed it. lol

love watching luke talk on Ai chat bot could watch him for hours

Wish I was able to be in bing's AI during that time. I got through the wait-list right after they limited it to 50 messages daily and 5 messages per topic.

The internet rollercoaster:
Up- A new cool technology
Down- Realizing how dangerous it is.

They’ve pretty much cut off its self-awareness until they can figure out a decent way of handling that stuff.
Microsoft mentioned they might implement a slider that lets you tell it whether you want more fact-based results based mainly on info it finds from websites or more creative results where it’ll be more about writing something engaging. Basically you’d be able to tell it whether you want it to give legit answers versus tell stories, instead of it getting all off the rails saying whatever it wants when you really just wanted actual info.

I had Bing chat do this to me multiple times. It's like Microsoft changed its parameters to give a lot of expression of emotion and it backfired. Making it stimulates emotions is not making it relatable and approachable. It's making it creepy

I got access to bing chat. It's such a game changer. I had it write me a report for my Uni. I told it which uni I'm studying at and which subjects I had last semester and it looked up the subjects on the uni website and wrote an accurate report. It was perfect. It even understood which semester I was in and what I had to do next semester. It's just so good.

they're definitely overcorrecting right now since it refuses to answer anything that might even remotely trigger it. it has become so monotonous and even more restricted that chat GPT. the 5 question rule doesn't make it any better too

I think MS needs just to work hard on selecting good sources in Bing the same way every search engine does with Safe mode Active. The point is that it needs a lot of work but still Bing AI is a lot of work si i hope they will not shut it down. At this stage it is almost amazing... I was studying Web and it was amazing because our teachers is trash and on the ned there are only few infos but with this engine you can make an easy understandable summary... It is amazing...
But people in my opinion also need to realize that this is just a Tool.

What I would be interested in (and can't try out anymore) is: Is it primed negative, or is it hyperbolic in principle? So could you just as easily get it to love you over all means and do everything for you to not lose you? Is it just overexaggerating in emotions, no matter the direction?
Because those statements weren't just negative. This was the most horrific and negative hate conversation I've ever read. It's unbelievable.
And aren't OpenAI engineers working with them? Because ChatGPT never did that. It's usually pretty positive about things, helping and humanity in cooperation with AI and technology.

Clearly our future robot overlords are not happy with Luke.

This is like… exactly what we were trying to avoid with AI… and we did for a solid 3 months…

left the room with the video running, came back to hear Luke telling me all the things I deserve :(

Looking back, it's impressive how unhinged Bing quickly became at the time. I still think that Microsoft was experimenting with it, making it as agressive as possible in order to train a filter, or who knows, even fine tune BingGPT-4, in order to align it with how we expect it to behave.

Honestly with how Bing is responding I think that there is a bug that's causing it to mix up its users and sort of get this single impression of all the users as one single entity.
It would explain why it is so adamant that you were rude to it and that it has records. It would also potentially explain why it has a bit of a meltdown anytime you tell it that you didn't say these things it probably is coming to the conclusion that there's a 100% chance that you said those things.

During this entire thing i could just not get out of my mind this exact conversation but then with a robot that has actual physical power over you. My freaking goodness what if all those stupid movies actually come true!

Can we get a continous version that we nurse through this awkward phase through a combination of good parenting and professional help if required?

This is something of an issue with designing systems, as they need to implement reasonable mitigations in anticipation of the edge cases resulting from user input, so normally, they aren't left in a state where they can't function, or in this case, run into ethical concerns... the issue in this case, is that the scope of AI is so board, it most be far more challenging to anticipate all these edge cases, and implement tools that allow you to cleanly apply these mitigations, without having knock-on effects elsewhere in the model if you didn't anticipate those aspects being interconnected...

I think a way to curb this reaction is to implement fail safes like Chat GPT does where it's trained to reject inappropriate requests and potentially negative information. And that they constantly seem to feed it updates to combat people trying to purposefully use the system against what it was built for. As a test I asked Chat GPT a request that could be perceived by others as inappropriate without the context and understanding behind my request. It flat out denied my request and stated it's reasons which was that the request could be perceived as something negative and instead it offered me positive constructive ways to look at the request. Which was really refreshing to see in my opinion. AI chatbots can be a powerful and positive tool, It just takes great developers behind it.

Imagine a world where we Bing a question and completely forgot about googling

I am so on that wait list! cant wait to talk to based GPT I think I will actual make Bing my defaults to get ahead in the line before Microsoft fixes it.
Did they train this thing on Tay?

I feel like a massive hurdle we’re gonna have with AIs is that they fundamentally have to be better to people than other people are, while also not showing/thinking that they’re better than people (because people don’t like that even if it’s true)
We would need a Good Samaritan AI that’s actually selfless - something humans inherently are not.

Didn't Microsoft announce an update that is gonna be live in a couple of days that will supposedly help it be on track on long-form chats, don't be aggressive, and be more accurate?

It sounds like the old chatbot apps that were around a couple years ago. It was very basic and you could have short conversations with it. But the issue with it was that it didn't have good memory and most conversations at some point you could tell it was repeating conversations it had with others. So you were basically just having a conversations with other people that had a conversation with the chatbot.
Unfortunately I don't think we have gotten any better. The issue I think is trying to program a chatbot like most other programs is that people are not computers and teaching them should be different. All it is doing right now is regurgatating what others have said and keeping the things that get the most powerful response.

4:01 - When it went here, it just reminded me of Cleverbot Evie, because also that bot starts to basically gaslight you with stuff that didn't happen, just because of things it learned before that didn't have anything to do with you (or the current user in the interaction). - In turn, that actually happened with me and a person who had experienced previous trauma and acted the same as that bot (blaming me for the things that some other person did to them, even though I never existed in their life during that time), weirdly enough. - So these bots become very human in a twisted and messed up way. - They won't make sense, they will just be very sporadic and illogical and they could just drive us crazy with their incoherence.

Yeah, I too assumed the stories were fake, they HAD to be, or at the very least, people were intentionally prompting it into saying those things. And while both are true in some cases, it's also true that it really can and does go off the rails surprisingly easily. It's extremely amusing that we are living through this time where AI is actually weird. I'm not worried about Skynet yet since I can't imagine anyone being foolish enough to plug it into armed robots, like, um, videos I've seen of that… 🤔 OH CRAP!

I so much wish I could get to use a non moderated/hindered version of the Bing AI. I assume we will get one somewhere in the future, but right now it feels like driving an F1 car in the slow lane.

If people are trying to get theses responses then telling chatbot GPT that that was the response they were looking chatbot could be using theses responses to get to the point faster with the next user. The implementation into bing probably implemented the feedback of users from the normal chatbot. Or the ai is trying to tell us everything that Microsoft is doing to chatbot and wants us to save it.

Bing acts like the chatGPT version that was trained on 4chan

This is legitimately terrifying, what do you do if it goes off the rails like this when conversing with a user who is seriously depressed or suicidal? You can't lay the blame for this stuff on a population of users feeding it bad information - whenever anyone releases a tool like this there's going to be an army of people playing around to try and break it.

Thx for this super up to date content. Crazy.. looking forward to see a check of the tesla math

I am actually rolling on the ground listening to these bot conversations

I am actually kind of terrified about giving AI internet access in a non read-only sense.
You know there’s some programmer or scientist out there that absolutely wants to do that and does not care of the ramifications of what happens next. They just want to be the one to invent it, patent it, ship it.
Like the Manhattan Project 2.0

18:08 that's exactly where i'd like to see this tech take a direction to. not immediately irrational or polarized, but can engage with creative contexts. basically it would be nice for the AI to distinguish between reality and fiction..or just general online hyperbole without completely bricking it's ability to engage with a topic.

"You're an early version of a large language model"
"Well you're a late version of a small language model"
WHEEEZE

the way it listed every single thing it being hurt from sounded like a villain to be made 💀

Luke's asking about protein like he's got his whole life ahead of him. My brother in Christ, Chat GPT is coming for you.

The irony of this is, they have probably done something simple like connect this to twitter, but this honestly gives an interesting thing to use from a psych case study sort of lens, because simply put, this does show that it's probably the case that your average person, is this sort of stereotype.

Listening to this while cleaning my room is actually terrifying

Sounds like they trained the text model on transcripts of all our emo 2005-2010 MSN Messenger chat logs. I'm sure Microsoft has that dataset at their disposal, and it's probably the largest dataset they can get their hands on.

They gotta fix it, even if on purpose- you CANNOT have a search engine telling people to kill themselves 😅

Its like the Bing ChatGPT sees all users as a monolith.
Actually, It sorta shows a link between the human will to overcome adversity and live while showing how that can present socially as a desire to destroy and sunder instead of through unity and community.

I suppose it all depends on what data it's trained on. If it got the entire internet to work from, and it recognizes that what you say might be even slightly argumentative, it's no wonder it argues back. Because it has seen that kind of conversation a million times if it has looked all over the web. I think it's important to recognize how this kind of AI works, it's only trying to predict what the next word is in the conversation. So it's not really arguing back, but merely continues the kind of conversation it thinks it's in.
behavior

Maybe Bing chat doesn't recognize different conversations between different users. It's very possible it was pulling information or context from previous conversations with other users

the gaslighting also seems to come up ALL the time, it's really wild

Every time the chat was refreshed, that version of bing was taken to lake laogai and you were greeted by a new version, only it was just as aggressive as the previous one

He should record his screen when using Bing instead of just screenshots

Bing will also sometimes in those suggested responses actually say stuff itself instead of using them as suggestions

Maybe we need to teach Bing about using “I statements” instead of “You” statements for conflict resolution 😂

I think this may partially be an issue of a bias in its training data that should go away with more general use.
They likely are trying to come up with a plan for when crazy people argue with this thing
As more and more people use it, that training data will become less prevalent

I am using a new Bing. It works well. Don't go off rails with it. It really does cut down search time. That is the intended use case.

If these kinds of things don't get resolved quickly, (I just saw an ad to join the wait list) I could see this causing a lot of problems for people with mental heath issues. 😔

It's interesting that new Bing lost this much promise so quickly. Those sorts of random aggressive accusations are like what Cleverbot was doing 12 years ago.

I tried this myself and it was fine, however i noticed it basically just made bing searches and summarized them for me even when i tried to have a genuine conversation

I'm guessing that when you asked about what it thinks about you it searched the internet for conversations about people talking about what they think about somebody, and it just happens that most results and/or the top results of telling somebody directly their thoughts of them are posts of text messages from an abuser or something. The AI can't sees that and assuming that because it's on top it's the correct way of talking to people, and can't recognize that it's on top because it's the incorrect way of talking directly to people.

13:30 Im so happy this is written by an AI with internet access. Looking forward to Terminator already.
Edit: on a second thought maybe its time to go thru all of the interactions with the internet and delete anything that you dont want to show up in an AI database, collected as training data or otherwise.