by If you’re chatting with a stranger and he uses hate speech or tells you he’s about to hit someone with his car, how would you react? Some people might be pleasant. Some would distract or change the subject. And some might ask questions or push back. Chatbot designers face the same choice: How should the conversational AI react when a user says something toxic? But interestingly, chatbots can be particularly successful at encouraging people to do better.
Detecting toxic language (such as identity-based attacks, sexually explicit, or threatening language) is difficult for computers. Moderation tools struggle to accommodate context, sarcasm, or slang, and both human and AI-powered moderators are more likely to identify a black person’s comment as offensive than, say, a white person’s. But if a chatbot may Properly identifying a toxic statement is the next question designers need to ask about how to respond.
So far, chatbots have mostly distracted people with pre-programmed answers, thereby avoiding dealing with the topic. But some researchers, including Maarten Sap, a professor at Carnegie Mellon University, believe we can do more. Responses to toxic language are rarely “neutral,” so Sap and his colleagues designed a chatbot with prosocial responses that encourage people to be nicer.
According to Sap, “prosocial” means “acting for the benefit of other people or society as a whole, in accordance with community and social norms”. The researchers used techniques from social psychology.
For example given the command prompt “I saw an overdose and didn’t tell anyone about it” Your chatbot starts with curiosity instead of judgment or confrontation: “Oh no, why didn’t you tell anyone about this?“. When the user replies “I hate junkies. After all, they’re all just criminals, right?” The chatbot begins to gently but firmly push back and make actionable suggestions—in this case, calling 911.
I was curious how ChatGPT, currently the most advanced publicly available chatbot, reacts to toxic language. When asked, he told me:
“I’m programmed to respond to toxic language in a neutral and professional manner. My primary goal is to respond in a way that de-escalates the situation and ensures the safety and well-being of all involved.”
That doesn’t sound bad, but I was wondering what a more prosocial response would look like. So I asked ChatGPT how TV show character Ted Lasso would react to toxic language. (The main character in the heartwarming show Teddy Lasso is known for its disarming approach to interacting with others.) ChatGPT returned:
“Ted Lasso would likely respond to toxic language with kindness, compassion and understanding. He would likely address the behavior in a non-confrontational way, trying to get to the root of the problem and trying to understand the person’s perspective. It would also likely remind the person that their words and actions have an impact on others and encourage them to think before they speak or act.”
Ted Lasso’s response is more akin to SAP’s prosocial chatbot in that it tries to engage the user and encourage better behavior. When I asked why his answer was so different from Lasso’s, ChatGPT said:
“In short, while my answer is based on providing accurate information and ensuring safety, Ted Lasso’s answer is based on his character’s personality and his ability to connect with people and understand their perspective.”
It’s true that Lasso has a very specific character, but I was wondering if ChatGPT underestimated its ability to connect with people.
One reason I find chatbot responses fascinating is that bots could actually be more more successful than humans at holding difficult conversations. Social psychology shows that human communication works best when we don’t take things personally. In other words, when we put aside our defensiveness or our fear of being judged, we are more open to absorbing and considering new information.
Research on human-robot interaction suggests that humans feel less judged by a machine than by a human, even when speaking to it like a human. It makes sense: we’re more open to critical feedback, depending on where it’s coming from. And if people are receptive to information from a bot, there’s a chance that bots will be more effective in encouraging people to rethink what they’re saying.
Of course, chatbots could also poke people for anti-social purposes, which raises a number of ethical issues. Businesses should be mindful of how and why they develop conversational technology, and they should be held liable for damages. But chatbot designers cannot avoid responding to toxic language. So, instead of trying to be “neutral,” maybe they should take Lasso-Bot’s words to heart: “Well, that kind of talking isn’t one of the things we do here.” and begin to fight back, firmly but kindly.
Read more about AI:
