Artificial Intelligence-Generated vs Academician-Developed Multiple True/False Questions in Anesthesiology Education

NCT07522658 | Completed

• 1 other identifier: E-41997688-050.99-212840
• Study Type: observational
• Target: 26
• Locations: 1 country
• Sites: 1

Brief Summary

This prospective observational study aims to evaluate the effectiveness and educational value of artificial intelligence (AI)-generated multiple true/false questions compared to those developed by experienced academicians in anesthesiology training. A total of 27 anesthesiology residents will be included in the study. Question sets consisting of 200 multiple true/false items will be created, with half generated by academicians and the other half generated using an artificial intelligence model (ChatGPT-based system). The questions will be based on standardized educational materials from the anesthesiology training curriculum. Participants will complete the test in a single session. Each correct answer will be scored as one point, and total scores will be calculated. In addition to test performance, item difficulty, discrimination indices, and test reliability will be analyzed. Furthermore, participants' perceptions regarding question quality will be evaluated. The study aims to determine whether AI-generated questions can provide a reliable and effective alternative to traditional question development methods in medical education and contribute to more objective and standardized assessment processes.

Conditions

Medical Education; Artificial Intelligence; Assessment Methods in Medical Training

Keywords

Artificial Intelligence; Anesthesiology Education; Multiple True False Questions; Medical Education Assessment; Item Analysis; Question Quality

Outcome Measures

Primary Outcomes (1)

• Item Difficulty Index of AI-generated and expert-authored questions

  For each question, the item difficulty index will be calculated as the proportion of participants who answer the item correctly. Item difficulty indices will be compared between AI-generated and expert-authored questions.

  Assessed once after completion of each participant's single 60-minute examination session; final item analysis performed after all participants complete the examination, up to 1 month.

Study Arms (2)

AI-Generated Questions

Multiple true/false questions generated using an artificial intelligence model

Academician-Developed Question

Multiple true/false questions prepared by experienced academicians in anesthesiology.

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)
• Sampling Method: Non-Probability Sample

Study Population

Anesthesiology residents undergoing training at Kütahya Health Sciences University.

You may qualify if:

• Being an anesthesiology resident
• Voluntary participation in the study
• Providing informed consent

You may not qualify if:

• Refusal to participate
• Incomplete test responses

Sponsors & Collaborators

• Kutahya Health Sciences University: lead

Study Sites (1)

Kütahya Health Sciences University

Kütahya, Turkey (Türkiye)

Location

Related Publications (3)

• Kaya M, Sonmez E, Halici A, Yildirim H, Coskun A. Comparison of AI-generated and clinician-designed multiple-choice questions in emergency medicine exam: a psychometric analysis. BMC Med Educ. 2025 Jul 1;25(1):949. doi: 10.1186/s12909-025-07528-6.

  PMID: 40597998 BACKGROUND

• Reid M, French M, Andreopoulos S, Wong C, Kee N. AI-generated multiple-choice questions in health science education: Stakeholder perspectives and implementation considerations. Curr Res Physiol. 2025 Aug 1;8:100160. doi: 10.1016/j.crphys.2025.100160. eCollection 2025.

  PMID: 40799313 BACKGROUND

• Kocer Tulgar Y, Tulgar S, Guven Kose S, Kose HC, Cevik Nasirlier G, Dogan M, Thomas DT. Anesthesiologists' Perspective on the Use of Artificial Intelligence in Ultrasound-Guided Regional Anaesthesia in Terms of Medical Ethics and Medical Education: A Survey Study. Eurasian J Med. 2023 Jun;55(2):146-151. doi: 10.5152/eurasianjmed.2023.22254.

  PMID: 37161553 BACKGROUND

Study Design

• Study Type: observational
• Observational Model: COHORT
• Time Perspective: PROSPECTIVE
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Assistant Professor of Anesthesiology

Study Record Dates

• First Submitted: March 28, 2026
• First Posted: April 13, 2026
• Study Start: April 14, 2026
• Primary Completion: April 20, 2026
• Study Completion: April 30, 2026
• Last Updated: May 6, 2026

Record last verified: 2026-04

Data Sharing

• IPD Sharing: Will not share

Locations

TU (1)