On July 22, 2026, the official Google Research blog published the findings of a first-of-its-kind study conducted by Joseph Breda, a student researcher, and Jake Sunshine, a senior staff research scientist at Google Research. The study, titled "SymptomAI: Towards a conversational AI agent for everyday symptom assessment," introduces SymptomAI—an experimental suite of conversational AI agents powered by the Gemini Flash 2.0 model, designed to conduct end-to-end symptom interviews and differential diagnosis (DDx) assessments for research benchmarking purposes. Conducted at a national scale in the United States, the study enrolled 13,917 consented participants. The researchers emphasize at the outset that all diagnoses, labels, and disease associations generated during the study were strictly for research analysis and do not constitute confirmed clinical diagnoses or official medical assessments.
The Data Gap and the Need for Conversational Agents in the Real World
According to the Google Research paper, a large proportion of clinical diagnoses are derived from language-based interviews alone. These diagnostic interviews are typically conducted by clinicians through doctor-patient interactions during in-person or remote visits. While these interactions are considered the gold standard for symptom assessment, they often suffer from financial, geographic, and systemic barriers that limit their accessibility. Existing language models (LMs) have demonstrated strong differential diagnosis assessment capabilities when evaluated on curated, highly structured medical case studies, highlighting their potential to support the diagnostic process.
However, these existing evaluations have largely relied on highly detailed, curated, and sometimes synthetic patient vignettes, which do not necessarily reflect the complexity and variability of everyday clinical presentation. These evaluations fail to capture how everyday patients naturally report their symptoms, which involves varying levels of medical literacy, incomplete information, and other complexities inherent in natural conversation. This significant gap leads to uncertainty regarding how LMs might perform in real-world contexts. To address this, the researchers conducted an in-situ comparative study using the experimental SymptomAI agents.
Research Methodology: Five Agents and 13,917 Participants
The study enrolled 13,917 consenting participants who described their symptoms to one of five randomized SymptomAI agents, which varied in their level of dialogue flexibility. During these conversations, participants described their symptoms, and the AI agents asked follow-up questions. The interactions culminated in a final differential diagnosis (DDx—a list of plausible diagnoses) and recommendations for next steps.
Following the session, participants could choose to consult a real healthcare provider. Two weeks after the interaction, participants were asked to report any diagnoses received from their healthcare provider via a survey. To evaluate and establish a baseline for SymptomAI's performance, the researchers conducted a clinical expert annotation study in which a panel of three board-certified clinicians reviewed the conversation transcripts and provided their own independent differential diagnoses. Subsequently, in a blinded fashion, each clinician ranked the DDx lists provided by SymptomAI and those provided by the other clinicians in the panel.
Key Findings: Clinical Expert Preference and Diagnostic Accuracy
The key results of the study indicated a clear preference among clinical experts for the diagnoses generated by SymptomAI. The clinicians preferred the DDx lists generated by the AI agent over those provided by their peer clinicians in over 50% of the cases. This suggests that SymptomAI's differential diagnoses aligned with the clinicians' own medical assessments just as often or more often than those of other clinicians. Additionally, the DDx generated by SymptomAI was most likely to be ranked as the best overall quality by the clinical raters.
Furthermore, the researchers compared diagnostic accuracy using a "top-5 Accuracy" metric—determining whether the true diagnosis reported by the patient after visiting their personal healthcare provider appeared as one of the five possible options in the DDx. The clinical evaluators reviewed the lists in a blinded manner to verify if the actual diagnosis was included (including both SymptomAI's lists and those generated by other clinicians). The findings revealed that clinicians ranked the DDx lists generated by SymptomAI as accurate more often than those of other clinicians, and they were more likely to contain the self-reported real-world diagnosis.
The Impact of Prompting Strategies and Information Elicitation
The study evaluated five distinct study arms, employing different prompting strategies to elicit medical history from participants:
- Dynamic Live: An agent with full autonomy to ask unrestricted follow-up questions.
- Dynamic Final: Another agent with full autonomy to ask unrestricted follow-up questions.
- Fixed Canonical: An agent that asked questions from a standard, fixed set of history-taking questions taught in medical schools.
- Flexible Canonical: An agent utilizing the same standard questions but with greater flexibility.
- Base: An unprompted baseline language model, representing the current user-driven status quo of standard LM chatbots.
The researchers found that all agent-driven prompting strategies (where SymptomAI actively asked follow-up questions) significantly outperformed the Base condition. This finding highlights the critical importance of active information elicitation from participants in improving differential diagnostic accuracy. Additionally, the performance advantage of SymptomAI over clinical baselines was greatest in cases where the clinicians themselves felt the least confident in their own differential diagnoses.
Integrating Physiological Signals from Fitbit Wearables
Beyond evaluating accuracy against clinical baselines, the researchers explored SymptomAI's potential at scale. Currently, the high cost of obtaining validated clinical labels prohibits population-scale analysis of physiological data. Accurate symptom-checking systems like SymptomAI could enable automated, clinical-quality reference labeling, thereby unlocking large-scale physiological data analysis—a task that is currently impossible at this scale.
One example is correlating wearable biosignals with different categories of illness. The most notable shifts in wearable biosignals were observed for acute respiratory infections. To study this at a population scale, the researchers collected daily biometric data from consenting participants for up to 30 days prior to their interaction with SymptomAI. The researchers identified clear biosignal shifts indicating symptom onset in the days leading up to the user's report.
The cohorts were separated based on SymptomAI's top-1 candidate diagnosis, grouping diagnoses classified as respiratory infections. This group excluded non-infectious respiratory conditions such as allergic rhinitis or chronic obstructive pulmonary disease (COPD). The analysis revealed distinct shifts in physiological metrics—including cardiovascular function, respiration, skin temperature, and sleep quality—in the days preceding the user's conversation with SymptomAI. These biometric changes peaked around the time of symptom reporting, providing observational physiological evidence that aligns with self-reported symptoms and offering a potential way to validate patient reports or provide passive data to support a differential diagnosis.
Additionally, this real-time accessibility highlights a core benefit of AI-based symptom checkers. Unlike traditional clinical appointments, which are often subject to scheduling delays, participants could participate in the SymptomAI study contemporaneously while their symptoms were fresh. This contemporaneous interaction can improve the accuracy of patient-reported onset timelines, a crucial factor for population-scale health analysis.
Study Limitations and Future Outlook
Despite these achievements, the researchers outlined several limitations of the study:
First, differential diagnosis is an inherently ambiguous task, and reported diagnoses can change and evolve over time. A symptom assessment is a snapshot in time, capturing symptoms as they present in that moment. Due to the scale of the deployment, it was impossible to control the frequency and timing of symptom reporting. Some participants reported their symptoms very early before representative indicators fully developed, while others reported obvious indicators from an informed context after years of living with a chronic illness. Future work may focus on specific illnesses at specific points during symptom development, such as early-onset metabolic syndrome or symptoms discussed at the onset of respiratory infections. Additionally, all diagnoses, labels, and disease associations generated during the study are AI-derived for research analysis only and do not constitute confirmed clinical diagnoses or official medical assessments.
Second, during the evaluation, the reviewing clinicians read static chat transcripts and were not given the agency to ask their own follow-up questions. Clinicians might have intuitively elicited different information had they directed the interview themselves. Furthermore, conversational AI systems may miss alternative signals such as body language, visual assessment, medical records, or, in primary care, the existing rapport and trust built with a patient.
In conclusion, the study introduces SymptomAI as an experimental conversational AI agent for conducting real-world patient interviews and symptom assessments. The study demonstrates the system's end-to-end performance on a broad population sample and shows how AI-based diagnoses can enable the analysis of population-scale signals, such as wearable biosignals, to identify physiological associations with reported illnesses.