The Big Unknown: A Journey Into Generative AI's Transformative Effect on Medical Professions

NCT07374926 | Completed

• 1 other identifier: ERCIC_572_25_04_2024
• Study Type: interventional
• Target: 249
• Locations: 3 countries
• Sites: 3

Brief Summary

A parallel group randomized controlled trial using a superiority framework. Clinical vignettes will be used to assess the impact of a large language model on the clinical reasoning of physicians. Quantitative analyses will be performed on graded vignette responses.

Conditions

Diagnosis; Vignette of Fictional Patients

Keywords

LLM; Quality of Care; Clinical Reasoning; Primary Care

Outcome Measures

Primary Outcomes (1)

• Percentage Correct Score

  Following Peabody et al (2000), the primary outcome is a percentage correct score across all steps in a vignette. This is generated by dividing the weighted total sum of rubric items assessed as present by the total number of rubric items possible in a vignette. Rubric items will be weighted with regards to their relevance by our expert panel.

  During Evaluation

Secondary Outcomes (3)

• Quality Per Answer

  During Evaluation

• Number of Answers

  During evaluation

• Less obvious answers

  During evaluation

Study Arms (2)

Own Knowlege

NO INTERVENTION

Group will not be given access to GPT-4 or other online resources

GPT-4o

ACTIVE COMPARATOR

Group given GPT-4o access

Other: GPT-4o

Interventions

GPT-4o OTHER

GPT-4o provided via an iFrame in the online Qualtrics environment

GPT-4o

Eligibility Criteria

• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)

You may qualify if:

• Registered medical physicians
• Training in internal or family medicine

You may not qualify if:

• Not currently practicing clinically

Sponsors & Collaborators

• Maastricht University: lead
• Aga Khan University: collaborator
• University of Indonesia, Jakarta, Indonesia: collaborator

Study Sites (3)

Universitas Indonesia

Jakarta, Indonesia

Location

Aga Khan University Hospital

Nairobi, Kenya

Location

Maastricht University

Maastricht, Netherlands

Location

MeSH Terms

Conditions

Disease

Condition Hierarchy (Ancestors)

Pathologic Processes; Pathological Conditions, Signs and Symptoms

Study Officials

• Mark Levels, PhD

  Maastricht University

  PRINCIPAL INVESTIGATOR

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: SINGLE
• Who Masked: OUTCOMES ASSESSOR
• Purpose: DIAGNOSTIC
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: SPONSOR

Model Details: We designed clinical vignettes to simulate patient-physician consultations. We follow the structure of clinical performance and value vignettes, a vignette design validated to perform well against standard patient designs. Our vignette design follows 9 stages: (1) Presenting Problem \& Initial Differential Diagnosis, (2) Asking about patient history, (3) Additional Differential Diagnosis, (4) Listing Physical Exams, (5) Differential Diagnosis, (6) Additional diagnostic (Lab) Tests, (7) Final Differential Diagnosis, (8) Medication, (9) Follow-Up/Advice. Participants were randomly assigned either to the control or intervention group using simple randomization and asked to complete the vignettes in an online environment. Intervention group participants were given access to an LLM (GPT-4o)

Study Record Dates

• First Submitted: January 21, 2026
• First Posted: January 29, 2026
• Study Start: August 1, 2024
• Primary Completion: January 17, 2025
• Study Completion: January 17, 2025
• Last Updated: January 29, 2026

Record last verified: 2025-09

Locations

KE (1); ID (1); NL (1)