Artificial Intelligence and Radiologists at Prostate Cancer Detection in MRI: The PI-CAI Challenge
1 other identifier
observational
10,207
1 country
1
Brief Summary
The PI-CAI challenge aims to validate the diagnostic performance of artificial intelligence (AI) and radiologists at clinically significant prostate cancer (csPCa) detection/diagnosis in MRI, with respect to histopathology and follow-up (≥ 3 years) as reference. The study hypothesizes that state-of-the-art AI algorithms, trained using thousands of patient exams, are non-inferior to radiologists reading bpMRI. As secondary end-points, it investigates the optimal AI model for csPCa detection/diagnosis, and the effects of dynamic contrast-enhanced imaging and reader experience on diagnostic accuracy and inter-reader variability.
Trial Health
Trial Health Score
Automated assessment based on enrollment pace, timeline, and geographic reach
participants targeted
Target at P75+ for all trials
Started Feb 2022
1 active site
Health score is calculated from publicly available data and should be used for screening purposes only.
Trial Relationships
Click on a node to explore related trials.
Study Timeline
Key milestones and dates
Study Start
First participant enrolled
February 1, 2022
CompletedFirst Submitted
Initial submission to the registry
August 3, 2022
CompletedFirst Posted
Study publicly available on registry
August 5, 2022
CompletedPrimary Completion
Last participant's last visit for primary outcome
June 1, 2023
CompletedStudy Completion
Last participant's last visit for all outcomes
November 1, 2023
CompletedNovember 18, 2023
July 1, 2022
1.3 years
August 3, 2022
November 16, 2023
Conditions
Keywords
Outcome Measures
Primary Outcomes (2)
AI vs Radiologists from Reader Study
Diagnostic performance of the top 5 AI models from the grand challenge and 50+ radiologists from the reader study, at csPCa detection/diagnosis in prostate bpMRI, with respect to histopathology and MRI with follow-up (≥ 3 years) as reference, to assess the clinical viability of present-day AI solutions.
6 months
AI vs Radiologists from Clinical Routine
Diagnostic performance of the top 5 AI models from the grand challenge and the historical reads of radiologists from clinical routine, at csPCa detection/diagnosis in prostate bpMRI, with respect to histopathology and MRI with follow-up (≥ 3 years) as reference, to assess the clinical viability of present-day AI solutions.
6 months
Secondary Outcomes (2)
AI vs AI
6 months
Radiologists vs Radiologists from Reader Study
6 months
Study Arms (4)
Public Training and Development Set (1500 cases)
Available for all participants and researchers, to train and develop AI models. Includes multi-vendor (Siemens Healthineers, Philips Medical Systems) prostate bpMRI cases from three Dutch centers (Radboud University Medical Center, Ziekenhuisgroep Twente, University Medical Center Groningen), acquired between 2012-2021. All data is fully anonymized and made available under a non-commercial CC BY-NC 4.0 license. Includes 328 cases from the PROSTATEx challenge (prostatex.grand-challenge.org). Imaging data has been released via: zenodo.org/record/6624726 (DOI: 10.5281/zenodo.6624726). Lesion annotations of csPCa have been released and are maintained via: github.com/DIAGNijmegen/picai\_labels.
Private Training Set (7500-9500 cases)
Used exclusively by the organizers to retrain the top-ranking 5 AI algorithms, with large-scale data. Includes multi-vendor (Siemens Healthineers, Philips Medical Systems) prostate bpMRI cases from three Dutch centers (Radboud University Medical Center, Ziekenhuisgroep Twente, University Medical Center Groningen), acquired between 2012-2021.
Hidden Validation and Tuning Cohort (100 cases)
Used for a live, public leaderboard that enables AI model selection and tuning throughout the open development phase of the challenge. Includes multi-vendor (Siemens Healthineers, Philips Medical Systems) prostate bpMRI cases from three Dutch centers (Radboud University Medical Center, Ziekenhuisgroep Twente, University Medical Center Groningen), acquired between 2012-2021, that remain fully hidden throughout the course of the challenge.
Hidden Testing Cohort (1000 cases)
Used to benchmark AI, radiologists, and test all hypotheses at the end of the PI-CAI challenge. A subset of 400 cases from this cohort is used to facilitate the PI-CAI: Reader Study. Includes multi-vendor (Siemens Healthineers, Philips Medical Systems) internal testing data (unseen prostate bpMRI cases from three seen Dutch centers {Radboud University Medical Center, Ziekenhuisgroep Twente, University Medical Center Groningen}) and external testing data (unseen prostate bpMRI cases from one unseen Norwegian center {Norwegian University of Science and Technology}), acquired between 2012-2021.
Interventions
Reference standard establishes histologically-confirmed (ISUP ≥ 2) cases of csPCa as positives, and histopathology- (ISUP ≤ 1) or MRI- (PI-RADS ≤ 2) with follow-up (≥ 3 years) confirmed cases of indolent PCa or benign tissue as negatives.
Reference standard establishes histologically-confirmed (ISUP ≥ 2) cases of csPCa as positives, and histopathology- (ISUP ≤ 1) or MRI- (PI-RADS ≤ 2) confirmed cases of indolent PCa or benign tissue as negatives.
Eligibility Criteria
All patient exams are of men suspected of harboring csPCa, with elevated levels of prostate-specific antigen (≥ 3 ng/mL) and/or abnormal findings on digital rectal exam, and without a history of treatment or any prior positive histopathology (ISUP ≥ 2) findings. Patients underwent prostate MRI, and were primarily examined at one of three Dutch centers (Radboud University Medical Center, Ziekenhuisgroep Twente, University Medical Center Groningen) or one Norwegian center (Norwegian University of Science and Technology) during regular clinical routine, between 2012-2021.
You may qualify if:
- Men suspected of harboring csPCa, with elevated levels of prostate-specific antigen (≥ 3 ng/mL) and/or abnormal findings on digital rectal exam, who subsequently underwent prostate MRI.
You may not qualify if:
- Patients who opted-out or did not give permission to reuse clinical data.
- Patients with a history of prior prostate treatment.
- Patients with a history of prior positive csPCa findings in histopathology (ISUP ≥ 2).
- Patients whose prostate MRI exhibit severe artifacts (e.g. heavy warping due to rectal air, metal artifacts from hip prostheses, heavy motion blur), thereby impeding their usage.
- Patients, whose positive histopathology findings (ISUP ≥ 2) cannot be reliably localized on MRI (e.g. MRI-invisible lesions, systematic biopsy diagnostic reports with ambiguous, "random" or missing location information).
Contact the study team to confirm eligibility.
Sponsors & Collaborators
- Radboud University Medical Centerlead
- Ziekenhuisgroep Twentecollaborator
- University Medical Center Groningencollaborator
- Norwegian University of Science and Technologycollaborator
Study Sites (1)
RadboudUMC
Nijmegen, Gelderland, 6525 GA, Netherlands
Related Publications (1)
Saha A, Bosma JS, Twilt JJ, van Ginneken B, Bjartell A, Padhani AR, Bonekamp D, Villeirs G, Salomon G, Giannarini G, Kalpathy-Cramer J, Barentsz J, Maier-Hein KH, Rusu M, Rouviere O, van den Bergh R, Panebianco V, Kasivisvanathan V, Obuchowski NA, Yakar D, Elschot M, Veltman J, Futterer JJ, de Rooij M, Huisman H; PI-CAI consortium. Artificial intelligence and radiologists in prostate cancer detection on MRI (PI-CAI): an international, paired, non-inferiority, confirmatory study. Lancet Oncol. 2024 Jul;25(7):879-887. doi: 10.1016/S1470-2045(24)00220-1. Epub 2024 Jun 11.
PMID: 38876123DERIVED
Related Links
Biospecimen
Biospecimen is not stored beyond the timeframe of this study, for the purpose of this study. However, biospecimen is stored beyond the timeframe of this study, for the purpose of regular clinical care. In this case, biospecimen refers to histopathology tissue acquired from confirmatory prostate biopsies and prostatectomies. Within the scope of clinical routine, storing such specimen can facilitate reassessments through the future, e.g. for comparisons if the patient presents new findings or metastasis of their initial findings, for comparisons against histopathology findings of the original patient's offspring, or even for legal purposes in the case of misdiagnosis.
MeSH Terms
Conditions
Interventions
Condition Hierarchy (Ancestors)
Intervention Hierarchy (Ancestors)
Study Officials
- PRINCIPAL INVESTIGATOR
Henkjan Huisman, PhD
Radboud University Medical Center
Study Design
- Study Type
- observational
- Observational Model
- COHORT
- Time Perspective
- RETROSPECTIVE
- Sponsor Type
- OTHER
- Responsible Party
- SPONSOR
Study Record Dates
First Submitted
August 3, 2022
First Posted
August 5, 2022
Study Start
February 1, 2022
Primary Completion
June 1, 2023
Study Completion
November 1, 2023
Last Updated
November 18, 2023
Record last verified: 2022-07
Data Sharing
- IPD Sharing
- Will share
- Shared Documents
- STUDY PROTOCOL, SAP, CSR, ANALYTIC CODE
- Time Frame
- Individual Participant Data Set (PI-CAI: Public Training and Development Set), Study Protocol, Statistical Analysis Plan (SAP) and Analytic Code has been shared with all participants of the PI-CAI challenge and the research community at large, towards the start of the challenge (June 2022). Clinical Study Report (CSR) will be released in the form of multiple publications after the completion of the challenge (tentatively May 2023). All of the aforementioned IPD will remain publicly accessible perpetually.
- Access Criteria
- Please refer to the "References" section of this protocol.
To facilitate open and transparent science, our end-to-end study protocol and our source code for preprocessing prostate MRI data archives, training baseline diagnostic AI models, evaluating lesion detection/diagnosis performance, and implementing statistical tests for AI/radiologists vs AI/radiologists comparisons, have been publicly released. Furthermore, a fully-anonymized dataset of 1500 prostate bpMRI scans from the PI-CAI challenge, and their outcomes, have been released to promote further research.