Multi-Disciplinary Treatment on the Anthropomorphism of Large Language Models
MDTALLM
1 other identifier
observational
300
1 country
1
Brief Summary
This retrospective clinical trial aims to better explore the potential of large language models in medicine by comparing the effectiveness of MDT consultations conducted by human doctors with those conducted by large language models. The main questions to be addressed are: Does using large language models to conduct anthropomorphic MDT consultations yield better results than using non-anthropomorphic processes? Is there a significant performance gap between MDT consultations conducted by large language models and those conducted by humans? How much greater is the economic benefit of MDT consultations from large language models compared to those conducted by humans? Retrospectively collect MDT consultation records from the past 20 years in northern Sichuan in China, as well as anonymized patient medical records. Group 1: Different large language models are assigned to act as doctors from different departments and as MDT secretaries to summarize consultations. Group 2: The large language model directly outputs diagnostic and treatment recommendations for patients. Compare the outputs of groups 1 and 2 with human performance retrospectively, score them, and select the best model from each department for a re-evaluation through anthropomorphic MDT consultations, once again comparing them to human results.
Trial Health
Trial Health Score
Automated assessment based on enrollment pace, timeline, and geographic reach
participants targeted
Target at P75+ for all trials
Started Oct 2024
Shorter than P25 for all trials
1 active site
Health score is calculated from publicly available data and should be used for screening purposes only.
Trial Relationships
Click on a node to explore related trials.
Study Timeline
Key milestones and dates
First Submitted
Initial submission to the registry
October 1, 2024
CompletedStudy Start
First participant enrolled
October 1, 2024
CompletedFirst Posted
Study publicly available on registry
October 4, 2024
CompletedPrimary Completion
Last participant's last visit for primary outcome
November 1, 2024
CompletedStudy Completion
Last participant's last visit for all outcomes
November 1, 2024
CompletedOctober 4, 2024
October 1, 2024
1 month
October 1, 2024
October 3, 2024
Conditions
Outcome Measures
Primary Outcomes (8)
Consultation Cost ($)
From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Consultation Time (min)
From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Comprehensiveness of the Multi-Disciplinary Treatment Results (Percentage Scale)
From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Clarity of Multi-Disciplinary Treatment Results (Percentage Scale)
From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Correctness of Multi-Disciplinary Treatment Results (Percentage Scale)
From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Cross-Professional Team Collaboration Practice Assessment (CPAT)
From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Rating Scale for Summarization
From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Flesch-Kincaid Readability Test
From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Secondary Outcomes (1)
Ethical Compliance (Boolean)
From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Study Arms (4)
Anthropomorphized Process Large Language Model Multidisciplinary Treatment Group
Using a locally deployed MedicalGPT, the commercially available online GPT-4o, Claude-3.5 Sonnet, GPT-4o mini, and Claude 3 Haiku, will each sequentially play the role of physicians from different departments involved in the Multi-Disciplinary Treatment Process. They will then sequentially take on the role of a summarizer to compile their recommendations into a final suggestion or treatment plan.
Non-anthropomorphized Process Large Language Model Multidisciplinary Treatment Group
Using a locally deployed MedicalGPT, the commercial online GPT-4o, Claude-3.5 Sonnet, GPT-4o mini, and Claude 3 Haiku to output multidisciplinary consultation results in a single instance, without separately assuming roles for each department and then compiling the results.
Real Doctors Multi-Disciplinary Treatment Group
In traditional multidisciplinary treatments, the results are documented in the consultation records of the patients involved, including the recommendations from doctors of various departments who participated in the consultation and the final summary by the secretary.
Best Large Language Model Multidisciplinary Treatment Group
After scoring the results of the Anthropomorphized Process Large Language Model Multidisciplinary Treatment Group against the outcomes of the Real Doctors' Multi-Disciplinary Treatment Group on a department-by-department basis, the best substitute models and the best summary models for each department were selected. These top models are set to assume roles in a Multi-Disciplinary Treatment consultation.
Interventions
Input all patient medical records, including text, examination reports, and imaging data, into GPT-4o. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Input all patient medical records, including text, examination reports, and imaging data, into GPT-4o mini. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Input all patient medical records, including text, examination reports, and imaging data, into MedicalGPT. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Input all patient medical records, including text, examination reports, and imaging data, into Claude-3.5 Sonnet. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Input all patient medical records, including text, examination reports, and imaging data, into Claude 3 Haiku. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Retrospectively collect the diagnostic and treatment recommendations from the corresponding departments involved in the multidisciplinary treatment of past patients, as well as the overall recommendations.
Eligibility Criteria
From hospital
You may qualify if:
- \. The medical records include interdisciplinary consultation notes, with recommendations from specialists of various departments and a well-documented final summary.
- \. The medical records contain data from at least one year prior to and one year following the consultation (including intact reports and imaging records).
- \. The patient\'s discharge conditions improved due to the multidisciplinary treatment plan after the consultation.
You may not qualify if:
- \. The medical records do not include multidisciplinary consultation notes, or the recommendations from various departmental physicians and the final summary notes are incomplete or inadequate.
- \. The medical records lack data from 1 year before and after the consultation, or miss necessary reports and imaging data, resulting in incomplete documentation.
- \. The patient\'s condition at discharge has not improved following the multidisciplinary treatment plan, or the condition has worsened.
Contact the study team to confirm eligibility.
Sponsors & Collaborators
- North Sichuan Medical Collegelead
- Affiliated Hospital of North Sichuan Medical Collegecollaborator
- University of Glasgowcollaborator
- Peking Universitycollaborator
- Peking University First Hospitalcollaborator
- Beijing Institute of Petrochemical Technologycollaborator
- Case Western Reserve Universitycollaborator
- Monash Universitycollaborator
Study Sites (1)
The Affiliated Hospital of North Sichuan Medical College
Nanchong, Sichuan, 637000, China
Related Publications (1)
Schroder C, Medves J, Paterson M, Byrnes V, Chapman C, O'Riordan A, Pichora D, Kelly C. Development and pilot testing of the collaborative practice assessment tool. J Interprof Care. 2011 May;25(3):189-95. doi: 10.3109/13561820.2010.532620. Epub 2010 Dec 23.
PMID: 21182434RESULT
MeSH Terms
Conditions
Condition Hierarchy (Ancestors)
Central Study Contacts
Study Design
- Study Type
- observational
- Observational Model
- OTHER
- Time Perspective
- RETROSPECTIVE
- Sponsor Type
- OTHER
- Responsible Party
- PRINCIPAL INVESTIGATOR
- PI Title
- Principal Investigator
Study Record Dates
First Submitted
October 1, 2024
First Posted
October 4, 2024
Study Start
October 1, 2024
Primary Completion
November 1, 2024
Study Completion
November 1, 2024
Last Updated
October 4, 2024
Record last verified: 2024-10
Data Sharing
- IPD Sharing
- Will share
- Shared Documents
- STUDY PROTOCOL, SAP, ICF, CSR, ANALYTIC CODE