Large Language Model Assistance for Clinical Decision-Making Among Rural Physicians

NCT07711600 | Not Yet Recruiting

• 1 other identifier: IRB00006761-M20260614
• Study Type: interventional
• Target: 240
• Locations: 1 country
• Sites: 1

Brief Summary

This study will evaluate whether, relative to conventional information retrieval approaches, direct large language models (LLM) access and LLM use training can improve the overall clinical decision-making ability of rural physicians in low-resource grassroots healthcare settings.

Conditions

Clinical Decision-making

Keywords

Large Language Models; Clinical Decision Support; Rural Physicians

Outcome Measures

Primary Outcomes (1)

• Overall Clinical Decision-Making Score

  Participants' responses to primary care clinical cases will be evaluated using a prespecified scoring rubric. The overall score will reflect performance across key components of clinical decision-making. Higher scores indicate better overall clinical decision-making performance.

  At the end of the initial 60-minute assessment

Secondary Outcomes (4)

• Diagnostic Judgment Domain Score

  At the end of the initial 60-minute assessment

• Clinical Warning Sign Recognition Domain Score

  At the end of the initial 60-minute assessment

• Treatment Plan Domain Score

  At the end of the initial 60-minute assessment

• Change in Overall Clinical Decision-Making Score After LLM Review

  Change from 60 to 90 minutes after the start of the assessment

Study Arms (4)

Conventional Resources Followed by LLM Second Opinion

ACTIVE COMPARATOR

Participants receive no LLM-use training. During the initial 60-minute assessment, they complete the clinical cases using conventional non-LLM resources only. After submitting their initial responses, they receive an additional 30 minutes to use the study-provided LLM as a second-opinion tool to review and revise their responses.

Other: Conventional Non-LLM Resources; Other: LLM Second-Opinion Review

Direct LLM Assistance

EXPERIMENTAL

Participants receive no LLM-use training and may use the study-provided LLM throughout the initial 60-minute assessment to complete the clinical cases. They submit their final responses at 60 minutes and do not participate in the additional second-opinion phase.

Other: Direct LLM Assistance

LLM Training and Conventional Resources Followed by LLM Second Opinion

EXPERIMENTAL

Participants receive brief structured training on the safe and effective use of LLMs. During the initial 60-minute assessment, they complete the clinical cases using conventional non-LLM resources only. After submitting their initial responses, they receive an additional 30 minutes to use the study-provided LLM as a second-opinion tool to review and revise their responses.

Behavioral: LLM-Use Training; Other: Conventional Non-LLM Resources; Other: LLM Second-Opinion Review

LLM Training and Direct LLM Assistance

EXPERIMENTAL

Participants receive brief structured training on the safe and effective use of LLMs and may use the study-provided LLM throughout the initial 60-minute assessment to complete the clinical cases. They submit their final responses at 60 minutes and do not participate in the additional second-opinion phase.

Behavioral: LLM-Use Training; Other: Direct LLM Assistance

Interventions

LLM Second-Opinion Review OTHER

After completing and submitting their initial responses using conventional non-LLM resources, participants receive an additional 30 minutes to use the study-provided DeepSeek-V4 as a second-opinion tool. They may review, verify, and revise their initial clinical decisions before submitting their final responses.

Conventional Resources Followed by LLM Second Opinion; LLM Training and Conventional Resources Followed by LLM Second Opinion

LLM-Use Training BEHAVIORAL

Before completing the clinical cases, participants receive brief structured training on the safe and effective use of LLMs. The training covers the role and limitations of LLMs, structured prompting and follow-up questioning, identification of warning signs and referral indications, medication safety, verification of LLM-generated information, high-risk situations in which LLMs should not be relied upon, and protection of patient privacy.

LLM Training and Conventional Resources Followed by LLM Second Opinion; LLM Training and Direct LLM Assistance

Conventional Non-LLM Resources OTHER

During the initial 60-minute assessment, participants complete primary care clinical cases using conventional non-LLM resources only, including clinical guidelines, textbooks, drug labels, training materials, medical websites, and standard search engines. Participants are not permitted to use LLMs during this phase.

Conventional Resources Followed by LLM Second Opinion; LLM Training and Conventional Resources Followed by LLM Second Opinion

Direct LLM Assistance OTHER

During the initial 60-minute assessment, participants may use the study-provided DeepSeek-V4 to assist with medical information retrieval, diagnostic and management reasoning, identification of warning signs, referral decisions, rational prescribing, patient education, and follow-up planning. Participants remain responsible for their final clinical decisions and responses.

Direct LLM Assistance; LLM Training and Direct LLM Assistance

Eligibility Criteria

• Age: 18 Years - 65 Years
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• Currently engaged in clinical practice at a rural primary healthcare institution in northwestern China.
• Has received formal medical education and holds a relevant diploma or degree.
• Able to read and understand clinical case materials in Chinese.
• Able to use a computer to complete the study tasks.
• Willing to participate and able to provide written informed consent.

You may not qualify if:

• Previously involved in the development of the clinical case tasks, reference answers, or scoring rubric for this study.
• Previously participated in pilot testing involving the same clinical case tasks or study procedures.

Sponsors & Collaborators

• Peking University Third Hospital: lead
• Xinjiang Second Medical College: collaborator

Study Sites (1)

Xinjiang Second Medical College

Karamay, Xinjiang Uygur Autonomous Region, China

Location

Central Study Contacts

Liyuan Tao, MD

CONTACT

+86 82265732; tendytly@163.com

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: SINGLE
• Who Masked: OUTCOMES ASSESSOR
• Purpose: HEALTH SERVICES RESEARCH
• Intervention Model: FACTORIAL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Research Professor

Model Details: Participants will be randomly assigned to four groups according to two intervention factors: receipt of brief training in LLM use and direct access to the LLM during the initial assessment. Participants assigned to selected groups will subsequently use the LLM as a second-opinion tool to review and revise their initial responses.

Study Record Dates

• First Submitted: July 14, 2026
• First Posted: July 17, 2026
• Study Start: July 1, 2026
• Primary Completion: August 1, 2026
• Study Completion (Estimated): September 1, 2026
• Last Updated: July 17, 2026

Record last verified: 2026-07

Data Sharing

• IPD Sharing: Will not share

Locations

CN (1)