Evaluation of One-Shot Vision Differential Diagnosis (OSVDE) and Multi-Step Conversational Non-Inferiority (MSCNE) in AI Medical Interviewing

NCT07470463 | Recruiting

• 1 other identifier: NH-OSVDE-MSCNE-1026
• Study Type: observational
• Target: 30
• Locations: 1 country
• Sites: 1

Brief Summary

This study evaluates the diagnostic performance of a multimodal artificial intelligence (AI) system (AIMD.1) using de-identified medical images and semi-synthetic patient simulations. The study combines retrospective analysis of existing publicly available image datasets with prospective data collection from licensed clinicians who complete diagnostic evaluation tasks. In the One-Shot Vision Differential Evaluation (OSVDE) stage, clinicians review individual de-identified medical images and generate a ranked list of potential diagnoses based solely on visual features. In the Multi-Step Conversational Non-Inferiority Evaluation (MSCNE) stage, clinicians complete diagnostic assessments using semi-synthetic patient simulations derived from de-identified medical images. Clinician performance will be compared with the AI system on the same diagnostic tasks. Human participants consist solely of licensed clinicians who provide diagnostic responses. Medical images and simulated cases are study materials and are not considered study participants. No identifiable patient data are used, and the AI system is evaluated in an offline research environment and is not used for clinical decision-making or patient care.

Conditions

Differential Diagnosis; Diagnostic Accuracy

Keywords

Artificial Intelligence; Diagnostic Accuracy; Differential Diagnosis; Clinical Decision Support; Computer Vision; Medical Imaging; Multimodal AI; Physician Performance; Non-Inferiority; Diagnostic Reasoning

Outcome Measures

Primary Outcomes (1)

• Top-1 Diagnostic Accuracy

  Proportion of evaluated cases in which the primary diagnosis generated by the AI Diagnostic System matches the reference (ground truth) diagnosis. Accuracy will be calculated across de-identified medical image cases and semi-synthetic patient simulation cases and compared with clinician performance.

  At completion of diagnostic evaluations (up to 6 months)

Secondary Outcomes (6)

• Top-5 Diagnostic Accuracy

  At completion of diagnostic evaluations (up to 6 months)

• Diagnostic Accuracy of Clinician Participants

  At completion of diagnostic evaluations (up to 6 months)

• Non-Inferiority of AI Diagnostic Accuracy Compared to Clinicians

  At completion of diagnostic evaluations (up to 6 months)

• Calibration of AI Diagnostic Confidence

  At completion of diagnostic evaluations (up to 6 months)

• Area Under the Receiver Operating Characteristic Curve (AUC) and Precision Recall Curve (PRC)

  At completion of diagnostic evaluations (up to 6 months)

Study Arms (1)

Clinician Participants

Licensed clinicians who participate in diagnostic evaluation tasks using de-identified medical images and semi-synthetic patient simulations to assess diagnostic accuracy. Clinicians provide differential diagnoses for benchmark comparison with an AI diagnostic system.

Diagnostic Test: AI Diagnostic System (AIMD.1)

Interventions

AI Diagnostic System (AIMD.1) DIAGNOSTIC_TEST

AIMD.1 (also known as NollaMD agent) is a multimodal artificial intelligence (AI) diagnostic system designed to generate differential diagnoses based on analysis of medical images and structured clinical information. In this study, the system is evaluated using de-identified medical images and semi-synthetic patient simulations under controlled research conditions. The AI system generates ranked diagnostic outputs and associated confidence scores, which are compared with reference diagnoses and clinician performance metrics. The system is evaluated in an offline research environment. AI outputs are not used for clinical decision-making, patient management, or real-world medical care.

Also known as: Artificial Intelligence Differential Diagnostic System, AI Clinical Decision Support System, AI Conversational Clinical System, NollaMD

Clinician Participants

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)
• Sampling Method: Non-Probability Sample

Study Population

The study population consists of licensed physicians across multiple medical specialties who participate in diagnostic evaluation tasks using de-identified medical images and semi-synthetic patient simulations. Participants are recruited through professional networks and medical associations. No patients are enrolled in this study.

You may qualify if:

• Active license in Dermatology, Internal Medicine, Otolaryngology, Gynecology, Orthopedics, Pediatrics, Geriatrics, Emergency Medicine, Ophthalmology, Psychiatry, Endocrinology, Family Medicine, or a closely related specialty
• Age 18 years or older
• Ability to complete diagnostic evaluation sessions remotely using a computer or tablet with reliable internet access

You may not qualify if:

• Loss of active license in an eligible specialty
• Inability to complete the evaluation session remotely

Sponsors & Collaborators

• Magic Health Inc. (d.b.a. Nolla Health): lead

Study Sites (1)

Nolla Health (Magic Health Inc.)

New York, New York, 10003, United States

RECRUITING

Study Officials

• Luis R Soenksen, MSE, PhD

  Nolla Health

  PRINCIPAL INVESTIGATOR

Central Study Contacts

Luis R Soenksen, MSE, PhD

CONTACT

(617) 936-9293; soenksen@nollahealth.com

Sean Geiger, B.S.

CONTACT

(412) 412-8786; sean@nollahealth.com

Study Design

• Study Type: observational
• Observational Model: CASE ONLY
• Time Perspective: PROSPECTIVE
• Target Duration: 6 Months
• Sponsor Type: INDUSTRY
• Responsible Party: SPONSOR

Study Record Dates

• First Submitted: March 10, 2026
• First Posted: March 13, 2026
• Study Start: March 19, 2026
• Primary Completion (Estimated): September 19, 2026
• Study Completion (Estimated): September 19, 2026
• Last Updated: March 25, 2026

Record last verified: 2026-03

Data Sharing

• IPD Sharing: Will not share

Locations

US (1)