Use of Artificial Intelligence to Assess Trainee Communication Compared to Human Assessment

NCT07107880 | Recruiting

• 1 other identifier: 81684
• Study Type: interventional
• Target: 45
• Locations: 1 country
• Sites: 1

Brief Summary

This study will evaluate whether a web-based artificial intelligence platform (AI) (Clinical Mind AI \[CMAI\] Stanford, CA), can assess communication skills in anesthesiology trainees, including residents and fellows, in the setting of disclosing medical errors. All participants will participate in an AI-generated simulation by using the platform remotely, and CMAI will assess trainee performance immediately after the simulation.

Conditions

Communication Skills

Keywords

Voice Artificial Intelligence (AI); Simulation; Clinical Mind Artificial Intelligence (CMAI); Medical Error Disclosure

Outcome Measures

Primary Outcomes (1)

• Reliability of AI Conversational Performance Assessment

  The primary outcome is to evaluate the reliability of the Clinical Mind AI (CMAI) platform in accurately assessing the communication skills of medical education trainees during a simulated interaction. To measure this outcome, the CMAI will use the Breaking Bad News Assessment Schedule (BBAS), a validated tool assessing participant's communication skills in five domains: 1) Setting the scene, 2) Breaking the news, 3) Eliciting concerns, 4) Informative giving, and 5) Empathy and Support. It includes 17 items, each with sub questions, rated on a 5-point Likert scale from (1-5), where score meanings may vary by each question. The scoring will be guided by Breaking Bad News Rubric, which outlines performance criteria for each item on the BBAS. In addition, two trained human evaluators will independently asses the participant's communication skills using the same BBAS tool and rubric. Lastly, both CMAI and human scores will be compared to determine the reliability of the CMAI platform.

  Immediately after the simulation

Secondary Outcomes (2)

• Usability of CMAI Simulation Platform

  immediatley after the simulation

• Satisfaction of CMAI Simulation

  immediately after the simulation

Study Arms (1)

Voice Artificial Intelligence (AI) Group

EXPERIMENTAL

Participants will remotely engage with an AI-simulated parent of a pediatric patient through a voice call, discussing a clinical scenario concerning their child. The participant's communication skills will be assessed using an artificial intelligence (AI) system and two human evaluators using the same rating scales.

Other: Communication Assessment through AI and Human

Interventions

Communication Assessment through AI and Human OTHER

Participants will remotely engage in an AI-generated simulation that presents an voice-based conversation with a simulated parent of a pediatric patient. The scenario involves discussing a clinical scenario concerning their child. Following the simulation, the participant's communication skills will be assessed using an artificial intelligence (AI) system trained to evaluate the key aspects of communication using standardized rating scales. In addition, two human evaluators will assess the participant's communication skills using the same rating scales.

Voice Artificial Intelligence (AI) Group

Eligibility Criteria

• Age: 18 Years - 99 Years
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• Participant's must be 18 years of age or older
• Graduate medical trainess such as residents and fellows at all levels of training excluding PGY1 trainees at Department of Anesthesiology, Perioperative, and Pain Medicine at Stanford University
• Able to speak and understand English
• Willing and able to provide consent to participate in research
• Able to participate in Artificial Intelligence communication simulation

You may not qualify if:

• Non-English speaker
• unable to provide consent to participate in research
• individual with no access to the necessary technology (internet, computer/smart device) required for participating in the digital, voice-based interaction with the AI platform

Sponsors & Collaborators

• Stanford University: lead

Study Sites (1)

Lucile Packard Children's Hospital Stanford

Palo Alto, California, 94304, United States

RECRUITING

Study Officials

• Thomas Caruso, MD, PhD

  Stanford University

  PRINCIPAL INVESTIGATOR

Central Study Contacts

Asheen Rama, MD

CONTACT

650-723-5728; asheen21@stanford.edu

Thomas Caruso, MD, PhD

CONTACT

650-497-0927; tjcaruso@stanford.edu

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: NA
• Masking: NONE
• Purpose: SUPPORTIVE CARE
• Intervention Model: SINGLE GROUP
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Clinical Professor

Study Record Dates

• First Submitted: July 23, 2025
• First Posted: August 6, 2025
• Study Start: November 10, 2025
• Primary Completion (Estimated): September 14, 2026
• Study Completion (Estimated): November 14, 2026
• Last Updated: April 24, 2026

Record last verified: 2026-04

Locations

US (1)