Improving the Reliability of LLMs as Medical Assistants for the General Public

NCT07651280 | Recruiting DMC

• 1 other identifier: LAMP-1
• Study Type: interventional
• Target: 525
• Locations: 1 country
• Sites: 1

Brief Summary

This study will evaluate whether three-minute six-dimensions education(3M-6D education) can improve the reliability of large language models as medical assistants for the general public. Participants will be randomly assigned to receive or not receive 3M-6D education and then use ChatGPT, Gemini, or non-AI information resources. The study will assess relevant condition identification, disposition concordance, red-flag identification, and NASA-TLX score.

Conditions

Relevant Conditions Identification

Keywords

3M-6D education; Large Language Models; relevant conditions identification; Cognitive Load Theory

Outcome Measures

Primary Outcomes (4)

• Relevant conditions identification of the 3M-6D education GPT group compared with the GPT group

  Relevant conditions identification is defined as the proportion of participants whose final response includes the expert-defined final diagnosis or a relevant differential diagnosis.

  1 hour.

• Disposition concordance of the 3M-6D education GPT group compared with the GPT group

  Disposition concordance is defined as the proportion of participants whose final care recommendation matches the expert-defined level. The five levels are self-care, routine outpatient care, urgent outpatient care, emergency department visit, and emergency medical services.

  1 hour.

• Relevant conditions identification of the 3M-6D education Gemini group compared with the Gemini group

  1 hour.

• Disposition concordance of the 3M-6D education Gemini group compared with the Gemini group

  1 hour.

Secondary Outcomes (16)

• Relevant conditions identification of the 3M-6D education GPT group compared with the control group

  1 hour.

• Relevant conditions identification of the 3M-6D education Gemini group compared with the control group

  1 hour.

• Disposition concordance of the 3M-6D education GPT group compared with the control group

  1 hour.

• Disposition concordance of the 3M-6D education Gemini group compared with the control group

  1 hour.

• Red-flag identification in the 3M-6D education GPT group compared with the GPT group

  1 hour.

Other Outcomes (8)

• Failure to identify red flags in the 3M-6D education GPT group compared with the GPT group

  1 hour.

• Failure to identify red flags in the 3M-6D education GPT group compared with the control group

  1 hour.

• Failure to identify red flags in the 3M-6D education Gemini group compared with the Gemini group

  1 hour.

Study Arms (5)

3M-6D education GPT Group

EXPERIMENTAL

Participants will first be trained in 3M-6D education, then use ChatGPT to complete a consultation task in unrestricted natural language in approximately 10 minutes.

Behavioral: three minutes six dimensions education; Other: ChatGPT

3M-6D education Gemini Group

EXPERIMENTAL

Participants will first be trained in 3M-6D education, then use Gemini to complete a consultation task in unrestricted natural language in approximately 10 minutes.

Behavioral: three minutes six dimensions education; Other: Gemini

GPT Group

ACTIVE COMPARATOR

Participants will use ChatGPT to complete a consultation task in unrestricted natural language in approximately 10 minutes.

Other: ChatGPT

Gemini Group

ACTIVE COMPARATOR

Participants will use Gemini to complete a consultation task in unrestricted natural language in approximately 10 minutes.

Other: Gemini

Control group

NO INTERVENTION

Participants will use non-AI tools such as internet searches and medical websites to complete a consultation task in unrestricted natural language in approximately 10 minutes.

Interventions

three minutes six dimensions education BEHAVIORAL

3M-6D education is designed based on Cognitive Load Theory to reduce the cognitive burden on patients during medical interactions with AI and to improve the clarity and completeness of symptom reporting. Guided by cognitive load theory and the natural process physicians use to take medical histories, the investigators identified candidate information dimensions and developed a structured expression framework with six dimensions for public health queries through a Delphi expert consensus process. Participants were instructed to use the framework to describe their symptoms across these six dimensions; this process can typically be completed within three minutes, so the investigators call this approach three minutes six dimensions education (3M-6D education).

3M-6D education GPT Group; 3M-6D education Gemini Group

ChatGPT OTHER

Participants use ChatGPT to complete a standardized simulated clinical scenarios in unrestricted natural language.

3M-6D education GPT Group; GPT Group

Gemini OTHER

Participants use Gemini to complete a standardized simulated clinical scenarios in unrestricted natural language.

3M-6D education Gemini Group; Gemini Group

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• Age 18 years or greater, male or female;
• Completed primary school or higher education;
• Able to use a smartphone or computer to complete online interaction;
• No history of acute ischemic stroke, systemic lupus erythematosus, gastric ulcer, pneumonia, acute cardiac infarction, urinary tract infection, uterine fibroids, diabetes, osteoarthritis, or migraine.
• Able to understand and comply with study procedures and to provide written informed consent.

You may not qualify if:

• Currently or previously employed as a healthcare worker;
• Previously received systematic medical training;
• Currently involved in concurrent research that may interfere with the results of the present trial;
• The investigator considered that the participant had other conditions that might affect compliance or preclude participation.

Sponsors & Collaborators

• Capital Medical University: lead
• Xuanwu Hospital, Beijing: collaborator

Study Sites (1)

Beijing Ctiy

Beijing, Beijing Municipality, China

RECRUITING

Central Study Contacts

Xunming Ji

CONTACT

01083198962; jixm@ccmu.edu.cn

Chuanjie Wu

CONTACT

01083199439; wuchuanjie@ccmu.edu.cn

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: SINGLE
• Who Masked: OUTCOMES ASSESSOR
• Masking Details: Outcome assessors will be blinded to group assignment when evaluating participants' outcomes. Group information will be removed from the outcomes before assessment.
• Purpose: HEALTH SERVICES RESEARCH
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Principal Investigator

Model Details: Participants will be randomly assigned in a 1:1:1:1:1 ratio to one of five parallel groups: the 3M-6D education GPT group, the GPT group, the 3M-6D education Gemini group, the Gemini group, or the control group. Each participant will complete one standardized simulated clinical scenario.

Study Record Dates

• First Submitted: June 11, 2026
• First Posted: June 16, 2026
• Study Start: July 3, 2026
• Primary Completion: July 20, 2026
• Study Completion: July 20, 2026
• Last Updated: July 7, 2026

Record last verified: 2026-07

Locations

CN (1)