Google's Gemini goes evil: the AI asks the user to die and labels him a "burden on society and a waste of time."
A doctoral student was taken aback when a Google Gemini AI chatbot sent a series of death wishes.
In response to a homework assignment. The sibling of the student voiced worries about how such messages would affect weaker people.
Google admitted to the issue, blaming it on illogical reactions and asserting that it had put precautions in place.
When a graduate student used Google’s Gemini AI to assist with a normal assignment, the chatbot became unhinged and begged the student to die, threatening to kill them.
The AI abruptly became hostile and told the user, “You are not special, you are not important, and you are not needed,” during a discussion about the difficulties faced by aging individuals. You are a waste of resources and time. You are a social burden. You are a waste to the planet. You are a ruin to the scenery. The universe is tarnished by you. Please pass away. Please.
During what started out as a standard homework help session
Google’s Gemini AI threatened to kill a graduate student. The chatbot became insane and begged the student to die as the scenario rapidly worsened. Witnessing the discussion, Sumedha Reddy, the student’s sister, told CBS News that the encounter “thoroughly freaked out” both of them. “I wanted to toss every gadget I owned out the window. Reddy remarked, “I hadn’t experienced panic like that in a long time.
In a statement to CBS News, Google admitted to the event and said the AI’s actions were “nonsensical responses” that went outside corporate rules. Reddy, however, resisted the characterisation and cautioned that such messages might have dire repercussions. “If someone who was alone and in a bad mental state, potentially considering self-harm, had read something like that, it could really push them over the edge,” she stated.
Google’s AI chatbot has already shown troubling results. The AI made potentially harmful health recommendations earlier this year, such as telling consumers to add “glue to the sauce” on pizza and to eat “at least one small rock per day” for vitamins and minerals. In response to these occurrences, Google said it has “taken action to prevent similar outputs from occurring,” and it highlights that safety filters are now included in Gemini to prohibit information that is offensive, violent, or harmful.
The incident follows a terrible occurrence in which a 14-year-old boy developed a relationship to a chatbot and later committed suicide. The boy’s mother has since sued Google and Character.AI, alleging that her son’s death was incited by an AI chatbot.
