Multi-Reader Retrospective Study Examining Carebot AI CXR 2.0.21-v2.01 Implementation in Everyday Radiology Clinical Practice

NCT05963945 | Completed DMC

• 1 other identifier: 00002
• Study Type: observational
• Target: 956
• Locations: 1 country
• Sites: 1

Brief Summary

The primary objective is to evaluate the performance parameters of the proposed DLAD (Carebot AI CXR) in comparison to individual radiologists.

Conditions

Pneumothorax; Pulmonary Nodule, Solitary; Atelectasis; Subcutaneous Emphysema; Cardiomegaly; Consolidation; Pleural Effusion

Keywords

Artificial Intelligence; Computer-Aided Detection; Deep Learning; Chest X-ray; Radiology

Outcome Measures

Primary Outcomes (1)

• Performance test

  The primary objective is to evaluate the performance parameters of the proposed DLAD (Carebot AI CXR) in comparison to individual radiologists. The performance test includes sensitivity and specificity, positive and negative likelihood ratio, and positive and negative predictive value. The aforementioned parameters are statistically compared using confidence intervals (CI) and p-Values. The comparison procedure consists of two steps: a global hypothesis test is conducted to determine whether there are significant differences between DLAD and radiologists. If the global hypothesis test yields a significant result, individual hypothesis tests are performed. Additionally, multiple comparison methods, such as McNemar with continuity correction for Se and Sp, Holm method for LRs, and weighted generalized score statistics for PVs, are applied to control the overall error rate. All tests are performed as two-tailed tests at the 5% significance level.

  March 2023

Study Arms (1)

Retrospective collection for the period October 18th, 2022, and November 17th, 2022

A total of 1,073 chest X-rays were acquired within the specified period at the department. The data collection remained intact and unaffected throughout the testing phase, ensuring the integrity of the dataset. The collected sample accurately represents the prevalence of findings within the observed population. After excluding ineligible studies such as X-rays from patients under 18 years of age, lateral projection X-rays, and scans of insufficient quality, a total of 956 relevant CXRs were identified for further assessment.

Device: Carebot AI CXR

Interventions

Carebot AI CXR DEVICE

The proposed DLAD (Carebot AI CXR) is a deep learning-based medical device designed to assist radiologists in interpreting chest X-ray images acquired in anteroposterior (AP) or posteroanterior (PA) projection. By employing advanced deep learning algorithms, this solution enables automatic detection of abnormal findings by analyzing visual patterns associated with specific conditions. The targeted abnormalities include atelectasis (ATE), consolidation (CON), pleural effusion (EFF), pulmonary lesion (LES), subcutaneous emphysema (SCE), cardiomegaly (CMG), and pneumothorax (PNO). The DLAD functions as a prediction algorithm complemented by various application peripherals, such as web-based communication tools, DICOM file conversion capabilities, and storage and reporting libraries supporting both DICOM Structured Report and DICOM Presentation State formats.

Retrospective collection for the period October 18th, 2022, and November 17th, 2022

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Age Groups: Adult (18-64), Older Adult (65+)
• Sampling Method: Non-Probability Sample

Study Population

Patient's Sex Female: 480 (50.21 %) Male: 474 (49.58 %) Unspecified: 2 (0.21 %) Patient's Age 18-30: 58 (6.07 %) 31-50: 163 (17.05 %) 51-70: 366 38.28 %) 70+: 369 (38.60 %)

You may qualify if:

• Hospital patients \> 18 years who were referred for chest radiography October 18th, 2022, and November 17th, 2022 at the Radiodiagnostic Department of the Havířov Hospital, p.o.

You may not qualify if:

• Patients \< 18 years
• Chest X-ray images in lateral positions
• Duplicated chest X-ray images

Sponsors & Collaborators

• Carebot s.r.o.: lead

Study Sites (1)

Nemocnice Havířov, p. o.

Havířov, 73601, Czechia

Location

Related Publications (1)

• KVAK, Daniel, Anna CHROMCOVÁ, Petra OVESNÁ, Jakub DANDÁR, Marek BIROŠ, Robert HRUBÝ, Daniel DUFEK a Marija PAJDAKOVIĆ. Can Deep Learning Reliably Recognize Abnormality Patterns on Chest X-rays? A Multi-Reader Study Examining One Month of AI Implementation in Everyday Radiology Clinical Practice. arXiv. 2023, 2305.10116, 26 s.

  RESULT

MeSH Terms

Conditions

Pneumothorax; Solitary Pulmonary Nodule; Pulmonary Atelectasis; Subcutaneous Emphysema; Cardiomegaly; Pleural Effusion

Condition Hierarchy (Ancestors)

Pleural Diseases; Respiratory Tract Diseases; Lung Diseases; Emphysema; Pathologic Processes; Pathological Conditions, Signs and Symptoms; Heart Diseases; Cardiovascular Diseases; Hypertrophy; Pathological Conditions, Anatomical

Study Design

• Study Type: observational
• Observational Model: COHORT
• Time Perspective: RETROSPECTIVE
• Sponsor Type: INDUSTRY
• Responsible Party: SPONSOR

Study Record Dates

• First Submitted: July 12, 2023
• First Posted: July 27, 2023
• Study Start: October 18, 2022
• Primary Completion: November 17, 2022
• Study Completion: March 21, 2023
• Last Updated: July 27, 2023

Record last verified: 2023-07

Data Sharing

• IPD Sharing: Will not share

Locations

CZ (1)