Scientific Validity Assessment and Optimization of AI-Generated A3/A4 Type Questions for the Chinese Medical Licensing Examination

NCT07505862 | Recruiting

• 1 other identifier: KY2025-1298-02
• Study Type: observational
• Target: 20
• Locations: 1 country
• Sites: 1

Brief Summary

This is a cross-sectional study that primarily employs quantitative analysis, supplemented by qualitative assessment. The research is conducted in two stages: Phase I consists of a model performance comparison experiment, and Phase II involves an item quality evaluation experiment. The entire study adheres to the principles of single-blinding, randomization, and standardization to ensure scientific rigor and reproducibility. The single-blind design is implemented during the "standardized testing" phase, where the system intersperses AI-generated items with those authored by human experts. Participants remain blinded to the source of each item (AI-generated vs. human-authored) throughout the testing and scoring processes, thereby ensuring the objectivity of the evaluation results.

Conditions

AI (Artificial Intelligence)

Outcome Measures

Primary Outcomes (1)

• Comparative Analysis of Item Accuracy

  The investigators will evaluate the average accuracy rates coefficients of AI-generated items compared to human-generated items

  Baseline

Eligibility Criteria

• Age: 18 Years - 60 Years
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64)
• Sampling Method: Non-Probability Sample

Study Population

Medical students currently enrolled in a Standardized Residency Training (SRT) program.

You may qualify if:

• Professional Status: Medical students currently enrolled in a Standardized Residency Training (SRT) program.
• Educational Background: Holders of a Bachelor of Medicine degree or higher, with foundational clinical knowledge.
• Informed Consent: Voluntarily participate in the study and provide written informed consent.
• Technical Competency: Proficient in using digital platforms to complete assessments and scoring.

You may not qualify if:

• Conflict of Interest: Individuals involved in the AI model training, prompt engineering, or the creation of the human-authored question bank for this study.
• Inability to Complete: Presence of visual/auditory impairments or severe illness that precludes completion of the assessment within the specified time.
• Investigator's Discretion: Any other condition that, in the opinion of the investigator, renders the participant unsuitable for the study.

Sponsors & Collaborators

• Guangdong Provincial People's Hospital: lead

Study Sites (1)

Guangdong Provincial People's Hospital

Guangzhou, Guangdong, China

RECRUITING

Central Study Contacts

Zhuoyi Chen MD

CONTACT

+86 18737552662; 18737552662@163.com

Study Design

• Study Type: observational
• Observational Model: OTHER
• Time Perspective: CROSS SECTIONAL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: MD, PhD

Study Record Dates

• First Submitted: March 22, 2026
• First Posted: April 1, 2026
• Study Start: October 1, 2025
• Primary Completion: March 30, 2026
• Study Completion: March 30, 2026
• Last Updated: April 1, 2026

Record last verified: 2026-03

Locations

CN (1)