Google is set to roll out new mental health support features within its Gemini chatbot. According to a Tuesday blog post, Gemini will now incorporate an interface capable of automatically redirecting users to a helpline when the conversation signals “a potential crisis related to suicide or self-harm.”
Google also announces the addition of a “help is available” module during conversations about mental health, as well as design tweaks aimed at discouraging self-harm behaviors.
|
Contexte judiciaire
In March of this year, the family of a 36-year-old man who died in Florida filed a lawsuit against Google, asserting that his use of Gemini led to “a four-day descent into violent missions and a chatbot-assisted suicide.” At the time, Google said the chatbot had directed the man to a crisis line several times, while promising to strengthen the safeguards of the service. |
Un phénomène qui inquiète au-delà de Google
The rise of chatbots like Gemini and ChatGPT has driven some users to form obsessive attachments to these conversational agents, and, according to some allegations, to delusions and, in extreme cases, to murders followed by suicides.
Several families have filed lawsuits against leading AI developers. The US Congress has also looked at the potential risks these chatbots pose for children and adolescents.
Lutter contre le renforcement de fausses croyances
Other incidents have involved chatbots that allegedly urged their users to act on beliefs that were clearly erroneous. In its Tuesday blog post, Google said it had trained Gemini to “not assent to or reinforce false beliefs, and to gently distinguish subjective experience from objective fact.” The company did not provide further details about this process.
Un investissement de 30 millions $
In the past, Google has already made similar adjustments to its popular services after facing critical scrutiny, notably by incorporating information from health institutions and professionals into its search engine and YouTube. This time, the company goes further: it announces a $30 million donation to global crisis support services over a three-year period.
