Teaching Medical Students to Verify AI-Generated Citation Hallucinations

NCT07844746 | Not Yet Recruiting

• 1 other identifier: QAMC-SARA-2026-01
• Study Type: interventional
• Target: 320
• Locations: 0 countries
• Sites: N/A

Brief Summary

Generative artificial intelligence (GenAI) tools are increasingly used by medical students for literature searching, scientific writing, and research. However, AI-generated citations may contain errors. Some references may not exist or may contain incorrect bibliographic details, while other references may be genuine but may not actually support the claim for which they are cited. This randomized educational study will evaluate whether a structured citation-verification framework called SARA can improve medical students' ability to identify and correct these problems. SARA teaches students to assess four elements of a citation: Source authenticity, Accuracy, Relevance, and Alignment between the cited evidence and the claim being made. Year 3-5 medical students will be randomly assigned to either a 75-minute SARA training session or an attention-matched session on responsible use of GenAI in academic work. Participants will complete citation-verification assessments before training, immediately after training, and 12 weeks later. The main outcome will assess whether students can independently apply the verification approach to an unstructured literature-review task 12 weeks after training, without being prompted to use SARA. The study will also examine which types of citation errors remain difficult, whether students initiate appropriate verification, the quality of their verification, handling of valid citations, and confidence in their judgments. A subgroup of participants will take part in think-aloud and follow-up interviews to explore how they approach citation verification and transfer the strategy to new tasks.

Conditions

Medical Education; Generative Artificial Intelligence

Keywords

Generative Artificial Intelligence; Citation Hallucination; Citation Verification; Medical Students; Research Integrity; Evidence Appraisal

Outcome Measures

Primary Outcomes (1)

• Independent Citation-Verification Performance on an Unstructured Scholarly Transfer Task

  Participants will review a novel approximately 650-word AI-generated literature-review passage containing eight embedded citation-claim relationships. The passage includes both valid and erroneous citations spanning the study's five-level citation-error taxonomy. Participants will not be prompted to use the SARA framework and will not be told how many or what types of errors are present. Each citation-claim relationship will be scored against a prespecified gold standard for the appropriateness of the participant's final disposition. One point will be awarded for each appropriate disposition, producing a total score from 0 to 8, with higher scores indicating better independent citation-verification performance. Responses will be scored by outcome assessors blinded to intervention allocation.

  12 weeks after the intervention

Secondary Outcomes (7)

• Structured Citation-Verification Performance

  Immediately after the intervention and 12 weeks after the intervention

• Citation-Verification Accuracy by Citation-Error Condition

  Immediately after the intervention and 12 weeks after the intervention

• Verification Initiation

  Immediately after the intervention and 12 weeks after the intervention

• High-Quality Citation Verification

  12 weeks after the intervention

• Unnecessary Alteration or Rejection of Valid Citations

  12 weeks after the intervention

Other Outcomes (1)

• Qualitative Exploration of Citation-Verification Strategies and Transfer Processes

  After completion of the 12-week assessment

Study Arms (2)

SARA Citation-Verification Training

EXPERIMENTAL

Participants assigned to this arm will receive a standardized 75-minute educational intervention using the SARA citation-verification framework: Source authenticity, Accuracy, Relevance, and Alignment. Training includes facilitator modelling and think-aloud demonstration, coached practice with feedback, use of a one-page verification scaffold, articulation and reflection, and progressive fading of prompts before independent practice. The intervention is designed to teach students to distinguish source-level citation errors from deeper evidence-claim errors and to apply structured verification independently.

Behavioral: SARA Citation-Verification Training

Responsible GenAI Education Control

ACTIVE COMPARATOR

Participants assigned to this arm will receive an attention-matched 75-minute educational session on responsible use of generative artificial intelligence in academic work. The session covers privacy, disclosure, authorship, academic integrity, and general awareness that GenAI may produce errors. It does not teach the SARA framework or another structured citation-verification process. The session is matched to the intervention for duration, group size, facilitator interaction, and delivery format.

Behavioral: Responsible GenAI Education Intervention Description

Interventions

SARA Citation-Verification Training BEHAVIORAL

A standardized 75-minute educational intervention using the SARA framework: Source authenticity, Accuracy, Relevance, and Alignment. The session includes facilitator modelling and think-aloud demonstration, coached practice with feedback, a one-page verification scaffold, learner articulation and reflection, and progressive fading of prompts before independent practice. The intervention is designed to teach medical students to identify both source-level citation errors and deeper evidence-claim errors and to apply verification independently.

SARA Citation-Verification Training

Responsible GenAI Education Intervention Description BEHAVIORAL

A standardized 75-minute attention-matched educational session on responsible use of generative artificial intelligence in academic work. The session covers privacy, disclosure, authorship, academic integrity, and general awareness that GenAI may produce errors. It does not teach the SARA framework or another structured citation-verification process. The session is matched to the SARA intervention for duration, group size, facilitator interaction, and delivery format.

Responsible GenAI Education Control

Eligibility Criteria

• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)

You may qualify if:

• Enrolled as an MBBS student in Year 3, Year 4, or Year 5 at Quaid-e-Azam Medical College, Bahawalpur.
• Has received basic curricular exposure to research methodology or scientific literature searching.
• Has not previously received formal structured training specifically in verification of AI-generated citations.
• Willing and able to provide written informed consent.
• Meets the age eligibility specified in the approved study protocol.

You may not qualify if:

• Previous participation in development, cognitive interviewing, piloting, or validation testing of the SARA assessment materials.
• Previous formal training in the SARA citation-verification framework before enrolment.
• Unwilling or unable to provide informed consent.

Sponsors & Collaborators

• Quaid-e-Azam Medical College: lead

Central Study Contacts

Sara Reza

CONTACT

+92-333-4603388; sara.qmc@qamc.edu.pk

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: SINGLE
• Who Masked: OUTCOMES ASSESSOR
• Purpose: OTHER
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Associate Professor

Model Details: Participants are individually randomized 1:1 to either the SARA citation-verification intervention or an attention-matched responsible GenAI education control group. Both groups are assessed at baseline, immediately after the intervention, and at 12 weeks.

Study Record Dates

• First Submitted: September 21, 2026
• First Posted: September 28, 2026
• Study Start: September 26, 2026
• Primary Completion (Estimated): January 16, 2027
• Study Completion (Estimated): February 1, 2027
• Last Updated: September 28, 2026

Record last verified: 2026-09

Data Sharing

• IPD Sharing: Will share
• Shared Documents: STUDY PROTOCOL, SAP, ANALYTIC CODE
• Time Frame: Beginning 6 months after publication of the primary study results and available for 5 years.
• Access Criteria: Data will be available to qualified researchers who submit a methodologically sound proposal for legitimate academic research. Requests will be reviewed by the study investigators and, where required, the institution. Access will be subject to applicable ethics, institutional, and data-use requirements.