Google jeopardizes user safety by minimizing health disclaimers in AI summaries.

Google’s approach to AI-generated medical information is putting individuals at risk by minimizing safety warnings regarding the potential inaccuracies of its advice. Despite the fact that the company claims its AI Overviews, which appear prominently above search results, encourage users to seek professional guidance rather than relying solely on these summaries, there are notable gaps in its messaging.

When addressing health-related queries, Google states that AI Overviews prompt users to verify information with experts. The company explained that “AI Overviews will inform people when it’s important to seek out expert advice or to verify the information presented,” according to Google’s official documentation.

However, an investigation by the Guardian revealed that Google fails to present these critical disclaimers right when users first encounter medical advice. Warnings only emerge if users delve deeper and click a button labeled “Show more” for additional health information. Even then, the safety labels are situated below the supplementary AI-generated medical details and are rendered in a smaller, less noticeable font.

For those who venture to explore more details, the disclaimer states, “This is for informational purposes only,” followed by a recommendation to consult a professional for any medical advice or diagnosis, while also noting that “AI responses may include mistakes.”

Google did not dispute that disclaimers do not appear when users initially receive medical advice or that these warnings are relegated to a less prominent position. According to a spokesperson, the AI Overviews “encourage people to seek professional medical advice,” and often mention the need for medical attention directly within the summary itself, “when appropriate.”

The findings presented by the Guardian prompted alarms from AI experts and advocates for patient welfare. They emphasize that disclaimers should not only be present but prominently displayed when users receive medical advice for the first time.

“The lack of visible disclaimers when users first receive medical information poses significant risks,” remarked Pat Pataranutaporn, an assistant professor and technologist at the Massachusetts Institute of Technology (MIT), renowned for expertise in AI and human-computer interaction.

“Firstly, even the most advanced AI systems still suffer from glitches, misinforming or prioritizing user satisfaction over factual accuracy—particularly dangerous in healthcare settings,” Pataranutaporn cautioned.

“Secondly, the challenges extend beyond AI’s limitations. Often, users may provide incomplete context or misinterpret their symptoms, leading to erroneous inquiries.”

“Disclaimers act as essential intervention points, disrupting blind trust and enabling users to engage critically with the information provided,” he added.

Gina Neff, a responsible AI professor at Queen Mary University of London, asserted that issues arising from ineffective AI Overviews stem from intentional design flaws within Google itself. “These Overviews prioritize speed over accuracy, leading to potentially dangerous inaccuracies in health information,” she stated.

In a preceding investigation conducted by the Guardian, it was revealed that individuals are exposed to misleading health information through Google’s AI Overviews, creating situations ripe for harm.

Neff emphasized that the earlier investigation underscored the necessity for clear disclaimers. “Google makes users click through layers to find any disclaimer, causing the rushed reader to misjudge the reliability of AI Overviews,” she pointed out. “Many might mistakenly assume the information is more reliable than it truly is, unaware that the AI can indeed make significant errors.”

Following the Guardian’s exposé, Google took action by removing AI Overviews for certain medical searches, but not universally across all health inquiries.

Sonali Sharma, a researcher at Stanford University’s centre for AI in medicine and imaging (AIMI), highlighted the pressing concerns. “The predominant problem is that these AI Overviews are placed prominently at the top of search results, often delivering what seems like a comprehensive answer when users are seeking swift information,” she explained.

“This compression of information basically creates a false sense of confidence that can deter users from further investigation or scrolling to the end for disclaimers,” she added.

“Moreover, the AI Overviews can present a mix of accurate and inaccurate information, making it difficult for users to discern what is reliable if they lack prior subject knowledge,” Sharma cautioned.

In response to the situation, a Google spokesperson conveyed the company’s stance: “It is misleading to claim that AI Overviews do not motivate individuals to seek professional medical advice. Disclaimers are clearly displayed, alongside frequent mentions of consulting medical professionals within the Overviews themselves, when deemed appropriate.”

Tom Bishop, the head of patient information at the blood cancer charity Anthony Nolan, called for immediate reforms. “Misinformation is a critical issue, especially regarding health-related content, where the risks can be substantial,” he asserted.

“We advocate for a more noticeable disclaimer, prompting users to reflect: ‘Should I consult my medical practitioner rather than accepting this information at face value? Is it essential for me to explore this data further, considering my own unique medical context?’ This reflection is crucial,” Bishop emphasized.

He went on to propose: “My wish is for the disclaimer to be right at the forefront. I envision it prominently displayed as the first thing users see, ideally in the same font size as the rest of the information, rather than in a smaller, easily overlooked font.”