Validation of an Artificial Intelligence-based Algorithm for Skeletal Age Assessment

NCT03530098 | Completed Results FDA Device

• 1 other identifier: IRB #44764
• Study Type: interventional
• Target: 1,903
• Locations: 1 country
• Sites: 6

Brief Summary

The purpose of this study is to understand the effects of using an Artificial Intelligence algorithm for skeletal age estimation as a computer-aided diagnosis (CADx) system. In this prospective real-time study, the investigators will send de-identified hand radiographs to the Artificial Intelligence algorithm and surface the output of this algorithm to the radiologist, who will incorporate this information with their normal workflows to make an estimation of the bone age. All radiologists involved in the study will be trained to recognize the surfaced prediction to be the output of the Artificial Intelligence algorithm. The radiologists' diagnosis will be final and considered independent to the output of the algorithm.

Conditions

Bone Age

Keywords

Age Determination by Skeleton; Machine Learning; Deep Learning; Artificial Intelligence; Prospective; Clinical Validation

Outcome Measures

Primary Outcomes (1)

• Paired Difference of Skeletal Age Estimate

  Mean absolute difference between dictated final impressions (baseline measure by Radiologist) and the consensus determination of a panel of radiologists following review.

  Up to 10 minutes to acquire the scan; up to 2 days to complete diagnosis review

Secondary Outcomes (1)

• Time for Diagnosis

  Up to approximately 4 minutes

Study Arms (2)

Control (Without-AI)

NO INTERVENTION

This is the control arm where no intervention is provided; represents current standard of care.

Experiment (With-AI)

EXPERIMENTAL

This is the experiment arm where the intervention, "BoneAgeModel", is provided. The participating radiologists in this arm will receive the output of the Artificial Intelligence algorithm. They will be asked to incorporate this new information with their normal workflows to make a diagnosis. The radiologists' diagnosis will be considered final.

Device: BoneAgeModel

Interventions

BoneAgeModel DEVICE

BoneAgeModel is an Artificial Intelligence tool that takes in a hand radiograph and gender, and outputs the skeletal (bone) age. The intervention involves using this tool as a factor in the clinical decision making process of the participating radiologists. The radiologist's decision will be considered final.

Experiment (With-AI)

Eligibility Criteria

• Sex: all
• Healthy Volunteers: No
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)

Sponsors & Collaborators

• Stanford University: lead

Study Sites (6)

Stanford University

Stanford, California, 94305, United States

Location

Yale New Haven Hospital

New Haven, Connecticut, 06519, United States

Location

Boston Children's Hospital

Boston, Massachusetts, 02115, United States

Location

New York University

New York, New York, 10016, United States

Location

Cincinnati Children's Hospital Medical Center

Cincinnati, Ohio, 45229, United States

Location

Children's Hospital of Philadelphia

Philadelphia, Pennsylvania, 19104, United States

Location

Results Point of Contact

• Title: Safwan S Halabi, MD
• Organization: Stanford University

safwan.halabi@stanford.edu

(650) 721-2850

Study Officials

• Curtis Langlotz, M.D. Ph.D.

  Stanford University

  STUDY CHAIR

• David Eng, B.S.

  Stanford University

  STUDY DIRECTOR

• Nishith Khandwala, B.S.

  Stanford University

  STUDY DIRECTOR

• Safwan Halabi, M.D.

  Stanford University

  PRINCIPAL INVESTIGATOR

Publication Agreements

• PI is Sponsor Employee: No
• Restrictive Agreement: No

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: NONE
• Purpose: DIAGNOSTIC
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Principal Investigator

Model Details: A hand radiograph will be randomly assigned to one of two groups - control and experiment. In the control group, participating radiologists will diagnose the exam using the current standard of care (no intervention). In the experiment group, the radiologists will factor in the output of the Artificial Intelligence algorithm in their skeletal age estimation. In all cases, the decision of the radiologist will be considered final.

Study Record Dates

• First Submitted: May 7, 2018
• First Posted: May 21, 2018
• Study Start: July 12, 2018
• Primary Completion: August 31, 2019
• Study Completion: August 31, 2019
• Last Updated: June 9, 2021
• Results First Posted: October 26, 2020

Record last verified: 2021-05

Data Sharing

• IPD Sharing: Will share
• Shared Documents: STUDY PROTOCOL, SAP, ANALYTIC CODE
• Time Frame: Beginning 3 months and ending 5 years following article publication.
• Access Criteria: Researchers who provide a methodologically sound proposal.

Locations

US (6)