The Effects of a Large Language Model on Clinical Questioning Skills

NCT06229379 | Completed DMC

• 1 other identifier: 2023KYPJ283
• Study Type: interventional
• Target: 84
• Locations: 1 country
• Sites: 1

Brief Summary

The researchers have used the ophthalmology textbook, clinical guideline consensus, the Internet conversation data and knowledge base of Zhongshan Ophthalmology Center in the early stage, combined with artificial feedback reinforcement learning and other techniques to fine-tune and train the LLM, and developed "Digital Twin Patient", a localized large language model that has the ability to answer ophthalmology-related medical questions, and also constructed a combination of automated model evaluation and manual evaluation by medical experts. The evaluation system combining automated model evaluation and manual evaluation by medical experts was constructed at the same time. This project intends to integrate "Digital Twin Patient" into undergraduate ophthalmology apprenticeship, simulate the consultation process of real patients through the online interaction between students and "Digital Twin Patient", explore the effect of "Digital Twin Patient" consultation teaching, provide emerging technology tools for guiding medical students to actively learn a variety of ophthalmology cases, cultivate clinical thinking, and provide the possibility of creating a new mode of intelligent teaching.

Conditions

Cataract; Glaucoma; Diabetic Retinopathy; Keratitis; Conjunctivitis

Keywords

Large language model; Clinical education; Ophthalmology

Outcome Measures

Primary Outcomes (1)

• Students' scores in the medical history acquisition exam

  Weekly during this study (up to 10 months)

Study Arms (2)

"Digital twin patient"

EXPERIMENTAL

The students in the "Digital twin patient" group were trained in history taking using a "digital twin patient" on the first day of a training program, and then took a 15-minute clinical questioning exam using the "digital twin patient" on the second day.

Device: "Digital twin patient"

Real patient

OTHER

The students in the Real patient group were trained in history taking using a real patient on the first day of a training program, and then took a 15-minute clinical questioning exam using the "digital twin patient" on the second day.

Device: "Digital twin patient"; Behavioral: Interaction with real patients

Interventions

"Digital twin patient" DEVICE

"Digital twin patient" can serve as patients with specific diseases for medical students to acquire disease history and thus practice clinical questioning skills.

"Digital twin patient"; Real patient

Interaction with real patients BEHAVIORAL

As in traditional medical education, medical students need to interact with real patients to practice history collection skills.

Real patient

Eligibility Criteria

• Age: 18 Years - 25 Years
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64)

You may qualify if:

• All undergraduate students from Sun Yat-sen University who participate in the ophthalmological internship.

You may not qualify if:

• Students who refuse to sign informed consent.

Sponsors & Collaborators

• Sun Yat-sen University: lead

Study Sites (1)

Zhongshan Ophthalmic Center, Sun Yat-sen Univerisity

Guangzhou, Guangdong, 510060, China

Location

Related Publications (1)

• Luo MJ, Bi S, Pang J, Liu L, Tsui CK, Lai Y, Chen W, Yang Y, Xu K, Zhao L, Jin L, Lin D, Wu X, Chen J, Chen R, Liu Z, Zou Y, Yang Y, Li Y, Lin H. A large language model digital patient system enhances ophthalmology history taking skills. NPJ Digit Med. 2025 Aug 4;8(1):502. doi: 10.1038/s41746-025-01841-6.

  PMID: 40760042 DERIVED

MeSH Terms

Conditions

Cataract; Glaucoma; Diabetic Retinopathy; Keratitis; Conjunctivitis

Interventions

Drug Interactions

Condition Hierarchy (Ancestors)

Lens Diseases; Eye Diseases; Ocular Hypertension; Retinal Diseases; Diabetic Angiopathies; Vascular Diseases; Cardiovascular Diseases; Diabetes Complications; Diabetes Mellitus; Endocrine System Diseases; Corneal Diseases; Conjunctival Diseases

Intervention Hierarchy (Ancestors)

Pharmacological Phenomena; Pharmacological and Toxicological Phenomena; Physiological Phenomena

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: NONE
• Purpose: OTHER
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Professor

Study Record Dates

• First Submitted: January 3, 2024
• First Posted: January 29, 2024
• Study Start: November 13, 2023
• Primary Completion: May 10, 2024
• Study Completion: August 7, 2024
• Last Updated: November 22, 2024

Record last verified: 2024-02

Data Sharing

• IPD Sharing: Will not share

Locations

CN (1)