Evaluating AI and Human Expert Decisions in Colorectal Cancer

NCT07045207 | Not Yet Recruiting

• 1 other identifier: PekingUCHI2025YJZ57
• Study Type: observational
• Target: 1,500
• Locations: 1 country
• Sites: 1

Brief Summary

The goal of this observational study is to evaluate the decision-making consistency between large language models (LLMs) and expert multidisciplinary teams (MDTs) in adult patients diagnosed with colorectal cancer who underwent MDT consultation between January 2023 and December 2024. The main questions it aims to answer are: How consistent are the treatment decisions generated by LLMs compared to actual MDT decisions? Do different LLMs (e.g., ChatGPT, DeepSeek) show varying levels of agreement with expert recommendations? What clinical factors contribute to differences between AI-generated and human expert decisions? Researchers will compare the AI-generated treatment recommendations with real-world MDT decisions using anonymized patient records to see if LLMs can reliably support clinical decision-making in oncology. Participants will: Have their de-identified clinical data (e.g., imaging, pathology, MDT notes) processed through several LLMs Not be contacted or receive any interventions, as this is a retrospective study using existing clinical records only.

Conditions

Colorectal Cancer

Keywords

Colorectal cancer; Multidisciplinary Team; Artificial Intelligence; Large Language Model

Outcome Measures

Primary Outcomes (1)

• Agreement Between AI-Generated and MDT Treatment Decisions

  Description: The primary outcome is the consistency between treatment recommendations generated by large language models (LLMs) and those made by expert multidisciplinary teams (MDTs) for colorectal cancer cases. Consistency will be quantified using Cohen's Kappa coefficient. Higher Kappa values indicate stronger agreement

  January 1, 2023 to December 31, 2024 (based on MDT consultation date)

Secondary Outcomes (3)

• Comparison of Agreement Across Different AI Models

  January 1, 2023 to December 31, 2024

• Output Stability of AI Models on Repeated InputsDescription

  January 1, 2023 to December 31, 2024

• Identification of Clinical Factors Associated With Decision Discordance

  January 1, 2023 to December 31, 2024

Interventions

LLM-MDT OTHER

Leveraging large language models (LLMs) to Generate Multidisciplinary Team (MDT) Treatment Recommendations

Eligibility Criteria

• Sex: all
• Healthy Volunteers: No
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)
• Sampling Method: Non-Probability Sample

Study Population

This study includes adult patients with histologically confirmed colorectal cancer who received multidisciplinary team (MDT) consultation at Peking University Cancer Hospital between January 1, 2023 and December 31, 2024. All clinical data were retrospectively collected from electronic medical records, including imaging, pathology, and MDT treatment recommendations. Patients represent a real-world tertiary cancer center population and were not selected based on treatment response or prognosis.

You may qualify if:

• Patients with a histologically confirmed diagnosis of colorectal cancer
• Patients who received multidisciplinary team (MDT) consultation at Peking University Cancer Hospital between January 1, 2023 and December 31, 2024
• Availability of complete clinical records, including(MDT consultation notes, CT or MRI imaging reports, Pathology reports, Outpatient or inpatient medical summaries)

You may not qualify if:

• Incomplete or missing medical records related to MDT decision-making
• MDT consultations conducted for non-oncologic purposes (e.g., hernia evaluation, stoma planning)
• Missing critical clinical data such as imaging or pathology reports
• Duplicate or conflicting records that prevent reliable data analysis

Sponsors & Collaborators

• Peking University Cancer Hospital & Institute: lead

Study Sites (1)

Peking University Cancer Hospital

Beijing, China

Location

MeSH Terms

Conditions

Colorectal Neoplasms

Condition Hierarchy (Ancestors)

Intestinal Neoplasms; Gastrointestinal Neoplasms; Digestive System Neoplasms; Neoplasms by Site; Neoplasms; Digestive System Diseases; Gastrointestinal Diseases; Colonic Diseases; Intestinal Diseases; Rectal Diseases

Central Study Contacts

Yongjiu Chen, PhD

CONTACT

+86 18813041827; yjchen@bjmu.edu.cn

Study Design

• Study Type: observational
• Observational Model: CASE ONLY
• Time Perspective: RETROSPECTIVE
• Sponsor Type: OTHER
• Responsible Party: SPONSOR

Study Record Dates

• First Submitted: June 22, 2025
• First Posted: July 1, 2025
• Study Start: July 1, 2025
• Primary Completion: June 1, 2026
• Study Completion: June 1, 2026
• Last Updated: July 1, 2025

Record last verified: 2025-06

Data Sharing

• IPD Sharing: Will not share

Locations

CN (1)