Teaching Doctors in Training to Reason With Artificial Intelligence

NCT07813754 | Enrolling By Invitation

• 2 other identifiers: 2026P00036963901784-350738
• Study Type: interventional
• Target: 200
• Locations: 1 country
• Sites: 4

Brief Summary

The purpose of the TEACH-AI study is to assess whether a brief, structured workshop on artificial intelligence can improve the performance of medicine doctors in training (i.e. residents) in their diagnostic and management reasoning. In this multi-site randomized controlled trial, internal medicine and family medicine residents are assigned either to receive an in-person workshop on safe, effective LLM use before a standardized AI-assisted assessment, or to complete the same assessment before receiving the workshop. Residents will review clinical cases that are fully synthetic, no protected health information is used, using a password protected LLM interface.

Conditions

Artificial Intelligence (AI); Clinical Reasoning; Clinical Decision-making; Medical Education

Keywords

artificial intelligence; medical education; clinical reasoning; clinical decision-making; human-computer interaction

Outcome Measures

Primary Outcomes (1)

• Total Score on Expert-Development Rubrics

  The primary outcome will be the number of correct responses for all cases using select questions from expert-developed scoring rubrics. The rubrics were developed using a Delphi consensus process by expert physicians as established by Goh et al (Nature Medicine, 2025; DOI: 10.1038/s41591-024-03456-y). The primary outcome will be analyzed at the case level, comparing performance between the randomized study groups, with a higher number of correct responses indicating a better outcome.

  Within 24 hours of assessment completion.

Secondary Outcomes (4)

• Time Spent on Management

  Within 24 hours of assessment completion.

• Prompt Count

  Within 24 hours of assessment completion.

• Management Reasoning Using Expert-Derived Rubrics

  Within 24 hours of assessment completion.

• Diagnostic Reasoning

  Within 24 hours of assessment completion.

Study Arms (2)

Assessment-First

NO INTERVENTION

Participants assigned to this group will be tasked with completing the assessment using a password protected LLM interface first in advance of the workshop. The workshop will be delivered during a scheduled mandatory didactic time slot.

Workshop-first

EXPERIMENTAL

Participants assigned to this group will be tasked with completing the assessment using a password protected LLM interface after they participate in a scheduled mandatory didactic time slot.

Other: Workshop-First

Interventions

Workshop-First OTHER

Participants will attend a workshop during their didactic time that will review various aspects of large language models including fundamentals and best practices of interacting and interpreting their outputs.

Workshop-first

Eligibility Criteria

• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)

You may qualify if:

• Participants must be licensed physicians and have started at least post-graduate year 1 (PGY1) of medical training.
• Training in Internal medicine or family medicine or emergency medicine.
• Able to provide informed consent and complete assessments in English.

You may not qualify if:

• Not currently practicing clinically.
• Resident directly involved in the design of the study, development of the workshop, or creation/piloting of the assessment vignettes.
• Resident who declines or withdraws consent.

Sponsors & Collaborators

• Beth Israel Deaconess Medical Center: lead
• Stanford University: collaborator
• Cambridge Health Alliance: collaborator
• AdventHealth: collaborator

Study Sites (4)

Stanford University

Palo Alto, California, 94305, United States

Location

AdventHealth

Orlando, Florida, 32804, United States

Location

Beth Israel Deaconess Medical Center

Boston, Massachusetts, 02215, United States

Location

Cambridge Health Alliance

Cambridge, Massachusetts, 02139, United States

Location

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: SINGLE
• Who Masked: OUTCOMES ASSESSOR
• Masking Details: The grading of responses will be performed by assessors blinded to participant identity and whether workshop-first or assessment-first assignment.
• Purpose: DIAGNOSTIC
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Instructor in Medicine

Model Details: The trial will be designed as a randomized, two-arm, single-blind parallel group study.

Study Record Dates

• First Submitted: August 10, 2026
• First Posted: September 10, 2026
• Study Start: September 24, 2026
• Primary Completion (Estimated): December 31, 2026
• Study Completion (Estimated): June 1, 2027
• Last Updated: September 25, 2026

Record last verified: 2026-09

Data Sharing

• IPD Sharing: Will not share

Locations

US (4)