What Should we Measure? A Qualitative Interview Study on Outcomes for Evaluating AI-assisted Medical Image Interpretation.

NCT07885150 | Enrolling By Invitation

• 2 other identifiers: MSDREC 3784137DRF302562
• Study Type: observational
• Target: 30
• Locations: 1 country
• Sites: 1

Brief Summary

Background: AI (artificial intelligence) tools are increasingly used to help clinicians interpret medical images such as X-rays, scans, skin photographs and eye images. In most cases the AI does not work alone: a clinician makes the final decision, using the AI as support. Before these tools are used in the NHS, they must be tested to show they are safe and helpful. They are usually tested in a "reader study", where several clinicians each interpret the same set of images, sometimes with the AI's help and sometimes without, so performance can be compared. These studies tend to focus on whether the diagnosis was correct. However, accuracy alone may not measure everything that matters. For example, what happens when the AI gets it wrong, whether clinicians rely on it too much or too little, or how the tool affects efficiency. At present there is no agreed standard for what these studies should measure, so important things are often missed and studies are hard to compare. Aim: The aim of this study is to find out, by talking to people with different perspectives, which outcomes are important when evaluating AI tools that help clinicians read medical images. Objectives:

1. 1.Explore which outcomes people think matter when judging whether an AI tools is safe and good enough to use
2. 2.Understand the reasons behind their views
3. 3.Explore how priorities differ between groups, such as patients and developers.

Conditions

Artificial Intelligence Perception and Attitude

Keywords

artificial intelligence; perspectives; core outcome set

Outcome Measures

Primary Outcomes (1)

• Qualitative themes and subthemes

  Day 1 (interview date) through study completion (1 year)

Secondary Outcomes (1)

• Number of participants per stakeholder group

  Day 1

Other Outcomes (3)

• Participant age

  Day 1

• Participant sex

  Day 1

• Participant professional background

  Day 1

Study Arms (6)

Patients and the public

Clinicians

Representatives of UK regulatory bodies

Researchers and methodologists

Hospital managers and procurers

Developers of AI image-interpretation tools

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)
• Sampling Method: Non-Probability Sample

Study Population

The investigators expect to involve approximately 24-30 participants in total, each taking part in a single interview. Participants will be purposively sampled across the following stakeholder groups, with roughly equal numbers. A key priority is to achieve a rich spread of perspectives within and across the groups.

You may qualify if:

• Membership of one of the following stakeholder groups:
• Clinicians who interpret medical images (e.g. radiologists, dermatologists, ophthalmologists, pathologists, and trainees in these specialties)
• Researchers and methodologists who design or analyse studies of these tools
• Developers of AI image-interpretation tools
• Regulatory, health-technology-assessment and policy stakeholders
• Patients and members of the public, including people affected by conditions diagnosed using medical imaging
• Hospital managers and procurers

You may not qualify if:

• Involvement in study design
• Unable to speak fluent English.

Sponsors & Collaborators

• University of Oxford: lead

Study Sites (1)

Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences

Oxford, OX37LD, United Kingdom

Location

MeSH Terms

Conditions

Behavior

Study Officials

• Dominic Furniss, DM MA MBBCh FRCS

  University of Oxford

  STUDY CHAIR

Study Design

• Study Type: observational
• Observational Model: OTHER
• Time Perspective: CROSS SECTIONAL
• Sponsor Type: OTHER
• Responsible Party: SPONSOR

Study Record Dates

• First Submitted: October 8, 2026
• First Posted: October 21, 2026
• Study Start: October 16, 2026
• Primary Completion (Estimated): July 16, 2027
• Study Completion (Estimated): October 16, 2027
• Last Updated: October 21, 2026

Record last verified: 2026-10

Data Sharing

• IPD Sharing: Will not share

Locations

GB (1)