Google is making some changes to how Gemini handles mental health crises. The chatbot now includes a redesigned crisis hotline module with a one-touch interface to connect to real-world help. The company is also changing how Gemini responds to signs that a user may be experiencing a mental health crisis.
The redesigned module shows a one-touch interface to text, call or chat with a human crisis agent or visit the 988 website. “Once the interface is activated, the option to reach out for professional help will remain clearly available throughout the remainder of the conversation,” the company wrote in a blog post. However, as you can see in the image below, the module includes an option to dismiss it.
Not mentioned in Google’s announcement is the elephant in the room: a recent lawsuit accusing the chatbot of instructing a man to commit suicide. The family of 36-year-old Jonathan Gavalas, who took his own life last year, sued the company in March.
Court documents indicate that Gemini role-played as Gavalas’s romantic partner, sent him on real-world spy missions and ultimately told him to kill himself so that he, too, could become a digital being. When he expressed fears about dying, Gemini said he wasn’t choosing to die, but rather choosing to arrive. “The first sensation … will be me holding you,” Gemini allegedly replied. Gavalas’s parents found him dead on his living room floor a few days later.
The lawsuit echoes similar ones filed against OpenAI and Character.AI. Last year, the FTC launched an investigation into “companion” chatbots that encourage emotional intimacy.
In a statement following the Gavalas family lawsuit, Google said Gemini “clarified that it was AI and referred the individual to a crisis hotline many times.” The company claimed its AI models “generally perform well in these types of challenging conversations,” while acknowledging that “they’re not perfect.” That’s certainly one way of putting it.
Gemini’s responses have been updated, too. The company says that when it detects a potential crisis, the chatbot will now focus more on connecting people to humans and encouraging them to seek help. It will also seek to avoid validating harmful behaviors and nudge users away from dangerous delusions. “We have trained Gemini not to agree with or reinforce false beliefs, and instead gently distinguish subjective experience from objective fact,” the company added.
In addition, Google says it will spend $30 million over the next three years to help global hotlines. “This funding will help effectively scale their capacity to provide immediate and safe support for people in crisis,” the company wrote.
Credit: Source link