Change in the Proportion of Correct Interpretations of Pulmonary Opacities

NCT07744555 | Active Not Recruiting

• 1 other identifier: 20260301
• Study Type: interventional
• Target: 50
• Locations: 1 country
• Sites: 1

Brief Summary

This randomized crossover clinical trial will evaluate whether an artificial intelligence-based expert system improves the interpretation of chest radiographs by final-year medical students, rural physicians, and general practitioners with less than two years of clinical experience. Participants will interpret chest radiographs containing normal findings or pulmonary opacities classified as alveolar, interstitial, or mixed. Each participant will review the same set of radiographs twice: once without assistance from the expert system and once with assistance from the artificial intelligence system. The order of these two reading conditions will be randomly assigned, with a six-week interval between sessions to reduce memory effects. The main outcome will be the proportion of correct interpretations compared with a previously established reference standard based on radiologist interpretation supported by chest computed tomography findings. The study will also assess diagnostic confidence and agreement with the reference standard. This study does not involve treatment decisions or direct patient care. It is intended to determine whether artificial intelligence support may improve the accuracy and confidence of less-experienced physicians when interpreting pulmonary opacities on chest radiographs.

Conditions

Pneumonia; Radiography

Keywords

Chest Radiography; Pulmonary Opacity; Artificial Intelligence; Diagnostic Accuracy; Crossover Trial

Outcome Measures

Primary Outcomes (1)

• Proportion of globally correct interpretations of pulmonary opacities

  Proportion of chest radiographs interpreted correctly by the participant, compared with a previously established reference standard. An interpretation is counted as globally correct only when both criteria are met: (1) the participant correctly identifies whether a pulmonary opacity is present or absent, and (2) when an opacity is present, the participant correctly classifies the pattern as alveolar, interstitial or mixed. The outcome is expressed as the percentage of correct interpretations out of the 16 radiographs read in each condition, and is compared between the reading condition without expert-system assistance and the reading condition with expert-system assistance.

  Each of the two reading sessions, separated by a 6-week washout period

Secondary Outcomes (4)

• Sensitivity and specificity for the detection of pulmonary opacities

  Each of the two reading sessions, separated by a 6-week washout period

• Proportion of correctly classified pulmonary opacity patterns

  Each of the two reading sessions, separated by a 6-week washout period

• Diagnostic confidence score

  Each of the two reading sessions, separated by a 6-week washout period

• Agreement between participant interpretation and the reference standard (Cohen's kappa)

  Each of the two reading sessions, separated by a 6-week washout period

Study Arms (2)

Group A without artificial intelligence

NO INTERVENTION

Reading condition without expert-system assistance (Phase A). Participants interpret the assigned set of chest radiographs on a calibrated workstation using their own judgement only, with no output from the artificial intelligence expert system and without access to clinical information or chest computed tomography findings. For each image, participants record whether a pulmonary opacity is present, classify the pattern as alveolar, interstitial or mixed, and rate their diagnostic confidence on a 5-point Likert scale.

Group B with artificial intelligence

ACTIVE COMPARATOR

Reading condition with expert-system assistance (Phase B). Participants interpret the same set of chest radiographs on the same calibrated workstation, with the output of the artificial intelligence expert system displayed together with each image. For each image, participants record whether a pulmonary opacity is present, classify the pattern as alveolar, interstitial or mixed, and rate their diagnostic confidence on a 5-point Likert scale. Participants remain responsible for the final interpretation.

Device: Artificial intelligence expert system

Interventions

Artificial intelligence expert system DEVICE

Artificial intelligence-based expert system for computer-aided interpretation of chest radiographs. The system is an open-source deep-learning model that analyzes posteroanterior chest radiographs and reports the presence of pulmonary opacities. During the intervention phase, each participant interprets the same set of chest radiographs while the output of the expert system is displayed together with each image. Participants remain responsible for the final interpretation and record it on the study data form. The expert system is used only as a reading aid for study participants; it is not used for clinical care, and no diagnostic, treatment, or management decision for any patient is based on its output.

Group B with artificial intelligence

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• Final-year undergraduate medical students.
• Physicians completing mandatory rural or social service.
• Licensed general practitioners with less than two years of clinical experience.
• Willingness to participate as a chest radiograph reader.
• Ability to complete both study reading sessions.

You may not qualify if:

• Corrected visual acuity worse than 20/60 in the better-seeing eye.
• Visual-field restriction, impaired contrast perception, binocular vision abnormalities, or another visual condition that prevents adequate interpretation of chest radiographs.
• Inability to complete the visual acuity assessment before the reading session.

Sponsors & Collaborators

• Universidad de la Sabana: lead

Study Sites (1)

Clínica Universidad de La Sabana

Chía, Cundinamarca, 111321, Colombia

Location

Related Publications (37)

• van Timmeren JE, Cester D, Tanadini-Lang S, Alkadhi H, Baessler B. Radiomics in medical imaging-"how-to" guide and critical reflection. Insights Imaging. 2020 Aug 12;11(1):91. doi: 10.1186/s13244-020-00887-2.

  PMID: 32785796 BACKGROUND

• McCague C, Ramlee S, Reinius M, Selby I, Hulse D, Piyatissa P, Bura V, Crispin-Ortuzar M, Sala E, Woitek R. Introduction to radiomics for a clinical audience. Clin Radiol. 2023 Feb;78(2):83-98. doi: 10.1016/j.crad.2022.08.149.

  PMID: 36639175 BACKGROUND

• Gefter WB, Post BA, Hatabu H. Commonly Missed Findings on Chest Radiographs: Causes and Consequences. Chest. 2023 Mar;163(3):650-661. doi: 10.1016/j.chest.2022.10.039. Epub 2022 Dec 12.

  PMID: 36521560 BACKGROUND

• Pesapane F, Gnocchi G, Quarrella C, Sorce A, Nicosia L, Mariano L, Bozzini AC, Marinucci I, Priolo F, Abbate F, Carrafiello G, Cassano E. Errors in Radiology: A Standard Review. J Clin Med. 2024 Jul 23;13(15):4306. doi: 10.3390/jcm13154306.

  PMID: 39124573 BACKGROUND

• Lee CS, Nagy PG, Weaver SJ, Newman-Toker DE. Cognitive and system factors contributing to diagnostic errors in radiology. AJR Am J Roentgenol. 2013 Sep;201(3):611-7. doi: 10.2214/AJR.12.10375.

  PMID: 23971454 BACKGROUND

• Itri JN, Tappouni RR, McEachern RO, Pesch AJ, Patel SH. Fundamentals of Diagnostic Error in Imaging. Radiographics. 2018 Oct;38(6):1845-1865. doi: 10.1148/rg.2018180021.

  PMID: 30303801 BACKGROUND

• Zhang L, Wen X, Li JW, Jiang X, Yang XF, Li M. Diagnostic error and bias in the department of radiology: a pictorial essay. Insights Imaging. 2023 Oct 2;14(1):163. doi: 10.1186/s13244-023-01521-7.

  PMID: 37782396 BACKGROUND

• Govindarajan A, Govindarajan A, Tanamala S, Chattoraj S, Reddy B, Agrawal R, Iyer D, Srivastava A, Kumar P, Putha P. Role of an Automated Deep Learning Algorithm for Reliable Screening of Abnormality in Chest Radiographs: A Prospective Multicenter Quality Improvement Study. Diagnostics (Basel). 2022 Nov 7;12(11):2724. doi: 10.3390/diagnostics12112724.

  PMID: 36359565 BACKGROUND

• Test M, Shah SS, Monuteaux M, Ambroggio L, Lee EY, Markowitz RI, Bixby S, Diperna S, Servaes S, Hellinger JC, Neuman MI. Impact of clinical history on chest radiograph interpretation. J Hosp Med. 2013 Jul;8(7):359-64. doi: 10.1002/jhm.1991. Epub 2012 Nov 26.

  PMID: 23184766 BACKGROUND

• Muthukrishnan N, Maleki F, Ovens K, Reinhold C, Forghani B, Forghani R. Brief History of Artificial Intelligence. Neuroimaging Clin N Am. 2020 Nov;30(4):393-399. doi: 10.1016/j.nic.2020.07.004. Epub 2020 Sep 18.

  PMID: 33038991 BACKGROUND

• Liu R, Rong Y, Peng Z. A review of medical artificial intelligence. Glob Health J. 2020;4:42-5.

  BACKGROUND

• Daye D, Wiggins WF, Lungren MP, Alkasab T, Kottler N, Allen B, Roth CJ, Bizzo BC, Durniak K, Brink JA, Larson DB, Dreyer KJ, Langlotz CP. Implementation of Clinical Artificial Intelligence in Radiology: Who Decides and How? Radiology. 2022 Dec;305(3):555-563. doi: 10.1148/radiol.212151. Epub 2022 Aug 2.

  PMID: 35916673 BACKGROUND

• Chang JY, Makary MS. Evolving and Novel Applications of Artificial Intelligence in Thoracic Imaging. Diagnostics (Basel). 2024 Jul 8;14(13):1456. doi: 10.3390/diagnostics14131456.

  PMID: 39001346 BACKGROUND

• Hao J, Wong LM, Shan Z, Ai QYH, Shi X, Tsoi JKH, Hung KF. A Semi-Supervised Transformer-Based Deep Learning Framework for Automated Tooth Segmentation and Identification on Panoramic Radiographs. Diagnostics (Basel). 2024 Sep 3;14(17):1948. doi: 10.3390/diagnostics14171948.

  PMID: 39272733 BACKGROUND

• Mese I, Taslicay CA, Sivrioglu AK. Improving radiology workflow using ChatGPT and artificial intelligence. Clin Imaging. 2023 Nov;103:109993. doi: 10.1016/j.clinimag.2023.109993. Epub 2023 Oct 6.

  PMID: 37812965 BACKGROUND

• Chetlen AL, Chan TL, Ballard DH, Frigini LA, Hildebrand A, Kim S, Brian JM, Krupinski EA, Ganeshan D. Addressing Burnout in Radiologists. Acad Radiol. 2019 Apr;26(4):526-533. doi: 10.1016/j.acra.2018.07.001. Epub 2018 Jul 31.

  PMID: 30711406 BACKGROUND

• Hanna TN, Shekhani H, Maddu K, Zhang C, Chen Z, Johnson JO. Structured report compliance: effect on audio dictation time, report length, and total radiologist study time. Emerg Radiol. 2016 Oct;23(5):449-53. doi: 10.1007/s10140-016-1418-x. Epub 2016 Jun 25.

  PMID: 27344141 BACKGROUND

• Castillo C, Steffens T, Sim L, Caffery L. The effect of clinical information on radiology reporting: A systematic review. J Med Radiat Sci. 2021 Mar;68(1):60-74. doi: 10.1002/jmrs.424. Epub 2020 Sep 1.

  PMID: 32870580 BACKGROUND

• Sharpe RE Jr, Tarrant MJ, Brook OR, Chatfield M, Chaudhry H, City RB, Donnelly LF, Goldberg-Stein S, Hernandez D, Hwang GL, Kunst MM, Lee R, Moriarity AK, Pahade JK, Patel S, Broder JC. Current State of Peer Learning in Radiology: A Survey of ACR Members. J Am Coll Radiol. 2023 Jul;20(7):699-711. doi: 10.1016/j.jacr.2023.03.018. Epub 2023 May 23.

  PMID: 37230234 BACKGROUND

• Machowska A, Stalsby Lundborg C. Drivers of Irrational Use of Antibiotics in Europe. Int J Environ Res Public Health. 2018 Dec 23;16(1):27. doi: 10.3390/ijerph16010027.

  PMID: 30583571 BACKGROUND

• Yi R, Tang L, Tian Y, Liu J, Wu Z. Identification and classification of pneumonia disease using a deep learning-based intelligent computational framework. Neural Comput Appl. 2023;35(20):14473-14486. doi: 10.1007/s00521-021-06102-7. Epub 2021 May 20.

  PMID: 34035563 BACKGROUND

• Bontrager KL, Lampignano JP, editors. Proyecciones radiologicas con correlacion anatomica. Elsevier; 2010.

  BACKGROUND

• Pletz MW, Blasi F, Chalmers JD, Dela Cruz CS, Feldman C, Luna CM, Ramirez JA, Shindo Y, Stolz D, Torres A, Webb B, Welte T, Wunderink R, Aliberti S. International Perspective on the New 2019 American Thoracic Society/Infectious Diseases Society of America Community-Acquired Pneumonia Guideline: A Critical Appraisal by a Global Expert Panel. Chest. 2020 Nov;158(5):1912-1918. doi: 10.1016/j.chest.2020.07.089. Epub 2020 Aug 25.

  PMID: 32858009 BACKGROUND

• Albastaki U, Al Hashemi H, Seyhoglu S, Mahmoud K, Al Hashmi A, Jerome CP. Comparative Assessment of Chest X-ray Interpretations by AI Model and Radiologist Vs Pulmonologist in Predicting the Clinical Status of Covid-19 Pneumonia Patients. J Adv Radiol Med Imaging. 7(1).

  BACKGROUND

• Fanni SC, Marcucci A, Volpi F, Valentino S, Neri E, Romei C. Artificial Intelligence-Based Software with CE Mark for Chest X-ray Interpretation: Opportunities and Challenges. Diagnostics (Basel). 2023 Jun 10;13(12):2020. doi: 10.3390/diagnostics13122020.

  PMID: 37370915 BACKGROUND

• Nam JG, Kim M, Park J, Hwang EJ, Lee JH, Hong JH, Goo JM, Park CM. Development and validation of a deep learning algorithm detecting 10 common abnormalities on chest radiographs. Eur Respir J. 2021 May 20;57(5):2003061. doi: 10.1183/13993003.03061-2020. Print 2021 May.

  PMID: 33243843 BACKGROUND

• Mutasa S, Sun S, Ha R. Understanding artificial intelligence based radiology studies: What is overfitting? Clin Imaging. 2020 Sep;65:96-99. doi: 10.1016/j.clinimag.2020.04.025. Epub 2020 Apr 23.

  PMID: 32387803 BACKGROUND

• Kottner J, Audige L, Brorson S, Donner A, Gajewski BJ, Hrobjartsson A, Roberts C, Shoukri M, Streiner DL. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011 Jan;64(1):96-106. doi: 10.1016/j.jclinepi.2010.03.002. Epub 2010 Jun 17.

  PMID: 21130355 BACKGROUND

• Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, Lijmer JG, Moher D, Rennie D, de Vet HC, Kressel HY, Rifai N, Golub RM, Altman DG, Hooft L, Korevaar DA, Cohen JF; STARD Group. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015 Oct 28;351:h5527. doi: 10.1136/bmj.h5527.

  PMID: 26511519 BACKGROUND

• Hopewell S, Chan AW, Collins GS, Hrobjartsson A, Moher D, Schulz KF, Tunn R, Aggarwal R, Berkwits M, Berlin JA, Bhandari N, Butcher NJ, Campbell MK, Chidebe RCW, Elbourne D, Farmer A, Fergusson DA, Golub RM, Goodman SN, Hoffmann TC, Ioannidis JPA, Kahan BC, Knowles RL, Lamb SE, Lewis S, Loder E, Offringa M, Ravaud P, Richards DP, Rockhold FW, Schriger DL, Siegfried NL, Staniszewska S, Taylor RS, Thabane L, Torgerson D, Vohra S, White IR, Boutron I. CONSORT 2025 Statement: Updated Guideline for Reporting Randomized Trials. JAMA. 2025 Jun 10;333(22):1998-2005. doi: 10.1001/jama.2025.4347.

  PMID: 40228499 BACKGROUND

• Gwet KL. Handbook of inter-rater reliability: the definitive guide to measuring the extent of agreement among raters. Advanced Analytics LLC; 2014.

  BACKGROUND

• Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. 2008 May;61(Pt 1):29-48. doi: 10.1348/000711006X126600.

  PMID: 18482474 BACKGROUND

• US Preventive Services Task Force; Mangione CM, Barry MJ, Nicholson WK, Cabana M, Chelmow D, Coker TR, Davis EM, Donahue KE, Epling JW Jr, Jaen CR, Krist AH, Kubik M, Li L, Ogedegbe G, Pbert L, Ruiz JM, Simon MA, Stevermer J, Wong JB. Screening for Impaired Visual Acuity in Older Adults: US Preventive Services Task Force Recommendation Statement. JAMA. 2022 Jun 7;327(21):2123-2128. doi: 10.1001/jama.2022.7015.

  PMID: 35608838 BACKGROUND

• Berlin L. Radiologic errors and malpractice: a blurry distinction. AJR Am J Roentgenol. 2007 Sep;189(3):517-22. doi: 10.2214/AJR.07.2209. No abstract available.

  PMID: 17715094 BACKGROUND

• Loeb MB, Carusone SB, Marrie TJ, Brazil K, Krueger P, Lohfeld L, Simor AE, Walter SD. Interobserver reliability of radiologists' interpretations of mobile chest radiographs for nursing home-acquired pneumonia. J Am Med Dir Assoc. 2006 Sep;7(7):416-9. doi: 10.1016/j.jamda.2006.02.004. Epub 2006 May 30.

  PMID: 16979084 BACKGROUND

• Albaum MN, Hill LC, Murphy M, Li YH, Fuhrman CR, Britton CA, Kapoor WN, Fine MJ. Interobserver reliability of the chest radiograph in community-acquired pneumonia. PORT Investigators. Chest. 1996 Aug;110(2):343-50. doi: 10.1378/chest.110.2.343.

  PMID: 8697831 BACKGROUND

• Collins J, Stern EJ. Chest radiology: the essentials: Lippincott Williams & Wilkins. 2008.

  BACKGROUND

MeSH Terms

Conditions

Pneumonia

Condition Hierarchy (Ancestors)

Respiratory Tract Infections; Infections; Lung Diseases; Respiratory Tract Diseases

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: NONE
• Purpose: DIAGNOSTIC
• Intervention Model: CROSSOVER
• Sponsor Type: OTHER
• Responsible Party: SPONSOR

Model Details: Randomized, controlled, two-period crossover trial with a six-week washout period. Each participant serves as their own control and reads the same set of chest radiographs under two conditions: Phase A, interpretation without expert-system assistance, and Phase B, interpretation with expert-system assistance. The order of the phases is randomized: Sequence 1 begins with Phase A and, after the washout period, proceeds to Phase B; Sequence 2 begins with Phase B and, after the washout period, proceeds to Phase A. The washout period and the independently randomized image order in each phase minimize memory and learning effects.

Study Record Dates

• First Submitted: July 1, 2026
• First Posted: August 4, 2026
• Study Start: July 1, 2026
• Primary Completion (Estimated): October 1, 2026
• Study Completion (Estimated): December 1, 2026
• Last Updated: August 4, 2026

Record last verified: 2026-08

Data Sharing

• IPD Sharing: Will not share

Locations

CO (1)