Google puts users at risk by downplaying health disclaimers under AI Overviews | Google

Google is putting people at risk by downplaying safety warnings that its AI-generated medical advice may be wrong.

When answering questions on sensitive topics like health, the company says its AI insights, which appear above search results, encourage users to seek professional help, rather than relying solely on its summaries. “AI insights will inform people when it is important to seek expert advice or verify the information presented,” Google said.

But the Guardian found that the company does not include such disclaimers when users first receive medical advice.

Google only issues a warning if users choose to request additional health information and click a button called “Show more.” Even then, the safety labels only appear below any additional medical advice gathered using generative AI, and in a smaller, clearer font.

“This is for informational purposes only,” the disclaimer tells users who click through for more details after seeing the initial summary and head to the very end of the AI ​​preview. “For medical advice or diagnosis, consult a professional. AI responses may contain errors.”

Google has not denied that its warnings do not appear when users receive initial medical advice, nor that they appear below AI previews and in a smaller, clearer font. AI overviews “encourage people to seek professional medical advice” and frequently mention seeking medical care in the summary itself “when appropriate,” a spokesperson said.

AI experts and patient advocates presented at the Guardian’s findings expressed concern. Disclaimers serve a vital purpose, they said, and should appear prominently when users first receive medical advice.

“The absence of disclaimers when users initially receive medical information creates several critical dangers,” said Pat Pataranutaporn, assistant professor, technologist and researcher at the Massachusetts Institute of Technology (MIT) and a world-renowned expert in AI and human-computer interaction.

“First, even the most advanced AI models today continue to hallucinate misinformation or display sycophantic behavior, prioritizing user satisfaction over accuracy. In healthcare settings, this can be truly dangerous.”

“Second, the problem is not just about the limitations of AI, but also about the human side of the equation. Users may not provide all the necessary context or ask the wrong questions by poorly observing their symptoms.”

“Disclaimers are a crucial point of intervention. They disrupt this automatic trust and encourage users to interact more critically with the information they receive.”

Gina Neff, a professor of responsible AI at Queen Mary University of London, said that “the problem of bad presentations of AI is intentional” and that Google is to blame. “AI insights are designed for speed, not accuracy, leading to errors in health information, which can be dangerous. »

In January, a Guardian investigation found that people were at risk of harm from false and misleading health information contained in Google’s AI previews.

Neff said the survey findings showed why prominent warnings were essential. “Google tricks people into clicking until they find a disclaimer,” she said. “People who read quickly may think the information they get from AI insights is better than it is, but we know this can make serious mistakes.”

Following the Guardian report, Google removed AI previews for some, but not all, medical searches.

Sonali Sharma, a researcher at the Center for AI in Medicine and Imaging (AIMI) at Stanford University, said: “The major problem is that these Google AI previews appear at the very top of the search page and often provide what appears to be a complete answer to a user’s question at a time when they are trying to access the information and get an answer as quickly as possible.

“For many people, because this single summary is there immediately, it creates a sense of comfort that discourages further searching, or scrolling down to the full summary and clicking ‘Show more’ where a disclaimer might appear.

“What I think can cause harm in the real world is the fact that AI previews can often contain partially correct and partially incorrect information, and it becomes very difficult to tell what is accurate or not, unless you are already familiar with the subject. »

A Google spokesperson said: “It is inaccurate to suggest that AI previews do not encourage people to seek professional medical attention. In addition to a clear warning, AI previews frequently mention the need to seek medical attention directly in the preview itself, where appropriate.”

Tom Bishop, head of patient information at blood cancer charity Anthony Nolan, called for urgent action. “We know that misinformation is a real problem, but when it comes to health misinformation, it is potentially very dangerous,” Bishop said.

“That disclaimer needs to be a lot more visible, just to make people take a step back and think… ‘Is this something I need to check with my medical team rather than acting on it? Can I take this at face value or do I really need to look at it in more detail and see how this information relates to my own specific medical situation?’ Because that’s the key here.

He added: “I’d like that disclaimer to be at the very top. I’d like it to be the first thing you see. And ideally it would be the same font size as everything you see there, not something small and easy to miss.”