Testing an AI Tool to Help Primary Care Clinicians With Specialty Consultation Questions
SAGE
Physician Usability and Output Quality Evaluation of a Retrieval-Augmented Language Model System for Specialty Medical Consultation: A Human-Computer Interaction Study
1 other identifier
interventional
15
1 country
1
Brief Summary
The goal of this study is to test an artificial intelligence (AI) tool called SAGE. SAGE helps primary care doctors with questions that often need a specialist. Primary care doctors are the doctors people usually see first. SAGE reviews a case and suggests what a specialist might advise. The main questions it aims to answer are:
- Do doctors make sound, timely care decisions when they use SAGE?
- Do those decisions match what a specialist would advise? Researchers will compare the decisions doctors make with and without SAGE. Doctors in the study will:
- Review made-up patient cases (these are not real patients)
- Make a decision for each case, their usual way and with SAGE
Trial Health
Trial Health Score
Automated assessment based on enrollment pace, timeline, and geographic reach
participants targeted
Target at below P25 for not_applicable
Started Jul 2026
Shorter than P25 for not_applicable
1 active site
Health score is calculated from publicly available data and should be used for screening purposes only.
Trial Relationships
Click on a node to explore related trials.
Study Timeline
Key milestones and dates
Study Start
First participant enrolled
July 1, 2026
CompletedFirst Submitted
Initial submission to the registry
July 10, 2026
CompletedFirst Posted
Study publicly available on registry
July 16, 2026
CompletedPrimary Completion
Last participant's last visit for primary outcome
September 1, 2026
ExpectedStudy Completion
Last participant's last visit for all outcomes
September 1, 2026
July 16, 2026
July 1, 2026
2 months
July 10, 2026
July 10, 2026
Conditions
Keywords
Outcome Measures
Primary Outcomes (2)
Clinical actionability of the clinician's decision
For each case, independent reviewers judge whether the physician's decision is clinically actionable, meaning the physician can act on it now rather than wait for a specialist. The outcome is the proportion of cases judged actionable, measured separately for decisions made with SAGE and for decisions made the standard way.
Post-session independent review, through study completion (up to approximately 2 months)
Concordance of the clinician's decision with the specialist reference answer
Agreement between the physician's management decision and a specialist reference answer, rated by independent reviewers, with inter-rater agreement summarized. Assessed for decisions made with SAGE and for decisions made the standard way.
Post-session independent review, through study completion (up to approximately 2 months)
Secondary Outcomes (3)
System usability (System Usability Scale, SUS)
End of study session (Day 1, approximately 60 minutes)
Order acceptance in the SAGE condition
During study session (Day 1)
Time to complete case review
During study session (Day 1)
Study Arms (1)
Primary care physicians
EXPERIMENTALPrimary care physicians make management decisions on synthetic patient cases. Each physician reviews four cases in random order: two with SAGE (AI clinical decision support) and two using a standard eConsult approach.
Interventions
SAGE is an artificial intelligence (AI) based clinical decision support tool. It generates specialty-informed recommendations for a primary care consultation question. Physicians review the SAGE output while making a management decision for the synthetic case.
A standard electronic consultation (eConsult) format, without SAGE. Physicians review this information while making a management decision for the synthetic case.
Eligibility Criteria
You may qualify if:
- Physician practicing in primary care internal medicine and/or family medicine
- Attending physician, or resident physician at postgraduate year 2 (PGY-2) or higher
- Stanford-affiliated (faculty, staff, or trainee)
- Able to complete a single, approximately 60-minute remote (Zoom) session conducted in English
You may not qualify if:
- Not currently practicing in primary care internal medicine or family medicine
- Member of the study team or otherwise involved in the development of SAGE
- Unable to complete the remote study session
Contact the study team to confirm eligibility.
Sponsors & Collaborators
Study Sites (1)
Stanford University
Stanford, California, 94305, United States
Study Officials
- PRINCIPAL INVESTIGATOR
Jonathan Chen, MD, PhD
Stanford University
Study Design
- Study Type
- interventional
- Phase
- not applicable
- Allocation
- NA
- Masking
- NONE
- Purpose
- HEALTH SERVICES RESEARCH
- Intervention Model
- SINGLE GROUP
- Sponsor Type
- OTHER
- Responsible Party
- PRINCIPAL INVESTIGATOR
- PI Title
- Associate Professor of Medicine
Study Record Dates
First Submitted
July 10, 2026
First Posted
July 16, 2026
Study Start
July 1, 2026
Primary Completion (Estimated)
September 1, 2026
Study Completion (Estimated)
September 1, 2026
Last Updated
July 16, 2026
Record last verified: 2026-07
Data Sharing
- IPD Sharing
- Will not share