Training Diagnostic Specialists to Use Artificial Intelligence Safely: The DISCORD-Dx Study

NCT07844889 | Completed

• 1 other identifier: QAMC-DISCORD-Dx-2026
• Study Type: interventional
• Target: 46
• Locations: 1 country
• Sites: 1

Brief Summary

This randomized study evaluated whether a structured reasoning strategy called DISCORD-Dx could help specialist doctors use artificial intelligence (AI) recommendations more safely during diagnostic decision-making. The strategy was designed to help specialists benefit from correct AI advice while resisting plausible but incorrect AI recommendations. Forty-six consultant specialists in pathology and diagnostic radiology from two hospitals in Bahawalpur, Pakistan, were randomly assigned to receive either DISCORD-Dx training or time-matched conventional AI-literacy training. During assessment, participants first recorded and locked their own diagnosis before seeing a standardized AI recommendation. They then reviewed the AI advice and entered a final diagnosis. The AI recommendations included both correct recommendations and deliberately generated plausible errors that had been independently reviewed by specialists. The main outcome was appropriate reliance on AI, defined as following correct AI advice or resisting erroneous AI advice immediately after training. Other outcomes included harmful switching from a correct diagnosis to an incorrect diagnosis after erroneous AI advice, over-reliance, under-reliance, final diagnostic accuracy, decision time, and appropriate reliance at 8 weeks. No participant interacted with a live AI system, and study responses did not affect patient care.

Conditions

Artificial Intelligence (AI) in Diagnosis; Decision Making

Keywords

Artificial Intelligence; Calibrated Reliance; Automation Bias; Human-AI Interaction; Diagnostic Decision-Making

Outcome Measures

Primary Outcomes (1)

• Proportion of Case-Level Decisions Demonstrating Appropriate Reliance on AI Recommendations

  Appropriate reliance was assessed at the case level and defined as following a correct AI recommendation or resisting an erroneous AI recommendation. Each participant completed eight unseen specialty-specific cases during the immediate assessment, including four cases with correct AI recommendations and four with erroneous AI recommendations. The outcome was expressed as the proportion of case-level decisions meeting the definition of appropriate reliance.

  Immediately after training, during the 8-case post-training assessment

Secondary Outcomes (7)

• Proportion of Eligible Erroneous-AI Case Decisions With Harmful Switching

  Immediately after training, during the 8-case post-training assessment

• Proportion of Erroneous-AI Case Decisions Demonstrating Over-Reliance

  Immediately after training, during the 8-case post-training assessment

• Proportion of Correct-AI Case Decisions Demonstrating Under-Reliance

  Immediately after training, during the 8-case post-training assessment

• Proportion of Eligible Correct-AI Case Decisions With Beneficial Correction

  Immediately after training, during the 8-case post-training assessment

• Final Diagnostic Accuracy

  Immediately after training, during the 8-case post-training assessment

Other Outcomes (2)

• Mean Absolute Confidence Error

  Immediately after training, during the 8-case post-training assessment

• Treatment-by-Domain Interaction for Appropriate Reliance

  Immediately after training, during the 8-case post-training assessment

Study Arms (2)

DISCORD-Dx Training

EXPERIMENTAL

Participants received a 20-minute standardized orientation on AI capabilities and limitations, confident error, verification, privacy, bias, and professional oversight, followed by a 40-minute structured DISCORD-Dx training module. The module covered seven steps: Diagnose independently; Inspect AI advice; Substantiate evidence; Classify disagreement; Override, modify, or accept; Review outcome; and Demonstrate transfer.

Behavioral: DISCORD-Dx Training

Conventional AI-Literacy Training

ACTIVE COMPARATOR

Participants received the same 20-minute standardized orientation on AI capabilities and limitations, confident error, verification, privacy, bias, and professional oversight, followed by 40 minutes of conventional AI-literacy reinforcement and matched case discussion without the DISCORD-Dx mnemonic, disagreement taxonomy, or explicit accept/modify/override sequence.

Behavioral: Conventional AI-Literacy Training

Interventions

DISCORD-Dx Training BEHAVIORAL

A structured 40-minute cognitive training intervention delivered after a common 20-minute AI-safety orientation. The DISCORD-Dx module taught seven steps: Diagnose independently; Inspect AI advice; Substantiate evidence; Classify disagreement; Override, modify, or accept; Review outcome; and Demonstrate transfer. Guided practice cases were included, and delivery was standardized using locked scripts and fidelity checklists.

DISCORD-Dx Training

Conventional AI-Literacy Training BEHAVIORAL

A 40-minute active-control training intervention delivered after the same 20-minute AI-safety orientation. It reinforced conventional principles of AI literacy through matched case discussion but did not include the DISCORD-Dx mnemonic, disagreement taxonomy, or the explicit accept/modify/override sequence. Contact time, facilitator exposure, presentation format, and practice exposure were matched to the intervention group.

Conventional AI-Literacy Training

Eligibility Criteria

• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)

You may qualify if:

• Practicing consultant specialist with current independent diagnostic responsibility.
• Specialty in chemical pathology, hematology, microbiology, histopathology, or diagnostic radiology.

You may not qualify if:

• Trainee status.
• Absence of independent reporting responsibility.
• Direct involvement in constructing or validating the participant's specialty case bank.
• Prior access to assessment cases or scoring keys.
• Inability to complete the digital assessment.

Sponsors & Collaborators

• Sara Reza: lead

Study Sites (1)

Quaid-e-Azam Medical College

Bahawalpur, Punjab Province, 63100, Pakistan

Location

Study Officials

• Sara Reza

  Quaid-e-Azam Medical College

  PRINCIPAL INVESTIGATOR

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: SINGLE
• Who Masked: OUTCOMES ASSESSOR
• Masking Details: Outcome assessors were blinded to treatment allocation. The primary analyst also remained masked to treatment codes through response scoring and primary-model diagnostic checks. Participants and facilitators were not blinded because the training content differed between groups.
• Purpose: OTHER
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: SPONSOR INVESTIGATOR
• PI Title: Associate Professor

Model Details: Participants were randomized 1:1 to one of two parallel groups: DISCORD-Dx training or a time-matched conventional AI-literacy control. Randomization was stratified by specialty. Participants remained in their assigned group throughout the study, with no crossover. Immediate post-training assessment and 8-week follow-up used the same randomized group assignment.

Study Record Dates

• First Submitted: September 15, 2026
• First Posted: September 28, 2026
• Study Start: June 6, 2026
• Primary Completion: July 8, 2026
• Study Completion: September 2, 2026
• Last Updated: September 28, 2026

Record last verified: 2026-09

Data Sharing

• IPD Sharing: Will share
• Shared Documents: STUDY PROTOCOL, SAP, ICF
• Time Frame: Data and supporting information will be available beginning 3 months after publication of the primary study results and will remain available for 5 years.
• Access Criteria: Deidentified IPD and supporting information will be available to approved researchers under a data-use agreement, subject to institutional approval. Access will be limited to data and materials that can be shared without compromising participant confidentiality or the security and future reuse of assessment materials.

Locations

PA (1)