Artificial Intelligence-powered Virtual Assistant for Emergency Triage in Neurology

NCT06334796 | Completed Early Phase 1

• 1 other identifier: 10476
• Study Type: interventional
• Target: 10
• Locations: 1 country
• Sites: 1

Brief Summary

This study examines the use of an AI-powered virtual assistant for quickly identifying and handling neurological emergencies, particularly in places with limited medical resources. The research aimed to check if this AI tool is safe and accurate enough to move on to more advanced testing stages. In a first-of-its-kind trial, the virtual assistant was tested with patients having urgent neurological issues. Neurologists first reviewed the AI's recommendations using clinical records and then assessed its performance directly with patients. The findings were as follows: neurologists agreed with the AI's decisions nearly all the time, and the AI outperformed earlier versions of Chat GPT in every tested aspect. Patients and doctors found the AI to be highly effective, rating it as excellent or very good in most cases. This suggests the AI could significantly enhance how quickly and accurately neurological emergencies are dealt with, although further trials are needed before it can be widely used.

Conditions

Stroke; Guillain-Barre Syndrome; Facial Palsy; Migraine; Status Epilepticus; Vertigo Benign Positional; Delirium; Trigeminal Neuralgia; Meningitis; Subarachnoid Hemorrhage

Outcome Measures

Primary Outcomes (1)

• Diagnostic performance

  Refers to the accuracy and effectiveness of medical tests or diagnostic tools in correctly identifying a disease or condition in patients. Syndromic diagnosis agreement: evaluating neurologists considered a syndromic diagnosis accurate when AI tools could identify a condition based on a set of commonly coexisting signs and symptoms, rather than identifying a specific disease. This method is applied when the precise disease causing the symptoms is not immediately identifiable, allowing healthcare providers to effectively monitor and treat the patient's presenting symptoms. Differential diagnosis agreement: a differential diagnosis was considered accurate when the differentials provided by each AI tool matched those presented by the participants. The gold standard for diagnosis was considered to be the one given in the emergency department, unchanged over a one-month period.

  The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.

Secondary Outcomes (2)

• Appropriate medical conduct or recommendation

  The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.

• Assessment of Usability and Satisfaction

  The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.

Study Arms (3)

Virtual Assistant

EXPERIMENTAL

Patients answer question with a virtual assistant about their recent visit to the ER.

Diagnostic Test: Virtual Assistant

ChatGPT 3.5

ACTIVE COMPARATOR

Patients answer question with ChatGPT about their recent visit to the ER.

Diagnostic Test: Virtual Assistant

ChatGPT 4

ACTIVE COMPARATOR

Patients answer question with ChatGPT about their recent visit to the ER.

Diagnostic Test: Virtual Assistant

Interventions

Virtual Assistant DIAGNOSTIC_TEST

Stage 1 focused on safety, using only medical information from clinical records for the virtual assistant. In Stage 2, which evaluated accuracy, participants interacted with the virtual assistant post-medical stabilization. Additionally, participants also provided initial symptom details for Chat-GPT input. Nine neurologists specializing in emergency participated in the study. In Stage 1, they assessed the virtual assistant's performance using clinical history information. In Stage 2, they analyzed the results from participant interactions with the assistant and performed a comparative evaluation of Chat-GPT. The virtual assistant functioned as a chatbot on WhatsApp and Telegram, using Spanish and incorporating advanced algorithms, decision trees, and large language models for interaction. For comparison, we utilized Chat-GPT versions 3.5 and 4, employing two prompt types in natural Spanish: one incorporating clinical record data and the other based on participant narratives.

ChatGPT 3.5; ChatGPT 4; Virtual Assistant

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: No
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• Patients over 18 years old consulting in the ER due to a neurological emergency

You may not qualify if:

• Pregnancy

Sponsors & Collaborators

• Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia: lead
• Entelai: collaborator

Study Sites (1)

Fleni

Buenos Aires, 1428, Argentina

Location

Related Publications (3)

• Haug CJ, Drazen JM. Artificial Intelligence and Machine Learning in Clinical Medicine, 2023. N Engl J Med. 2023 Mar 30;388(13):1201-1208. doi: 10.1056/NEJMra2302038. No abstract available.

  PMID: 36988595 BACKGROUND

• Au Yeung J, Wang YY, Kraljevic Z, Teo JTH. Artificial intelligence (AI) for neurologists: do digital neurones dream of electric sheep? Pract Neurol. 2023 Nov 23;23(6):476-488. doi: 10.1136/pn-2023-003757.

  PMID: 37977806 BACKGROUND

• Patel UK, Anwar A, Saleem S, Malik P, Rasul B, Patel K, Yao R, Seshadri A, Yousufuddin M, Arumaithurai K. Artificial intelligence as an emerging technology in the current care of neurological disorders. J Neurol. 2021 May;268(5):1623-1642. doi: 10.1007/s00415-019-09518-3. Epub 2019 Aug 26.

  PMID: 31451912 BACKGROUND

MeSH Terms

Conditions

Stroke; Guillain-Barre Syndrome; Facial Paralysis; Migraine Disorders; Status Epilepticus; Benign Paroxysmal Positional Vertigo; Delirium; Trigeminal Neuralgia; Meningitis; Subarachnoid Hemorrhage

Condition Hierarchy (Ancestors)

Cerebrovascular Disorders; Brain Diseases; Central Nervous System Diseases; Nervous System Diseases; Vascular Diseases; Cardiovascular Diseases; Polyradiculoneuropathy; Autoimmune Diseases of the Nervous System; Demyelinating Diseases; Polyneuropathies; Peripheral Nervous System Diseases; Neuromuscular Diseases; Autoimmune Diseases; Immune System Diseases; Post-Infectious Disorders; Chronic Disease; Disease Attributes; Pathologic Processes; Pathological Conditions, Signs and Symptoms; Mouth Diseases; Stomatognathic Diseases; Paralysis; Neurologic Manifestations; Signs and Symptoms; Headache Disorders, Primary; Headache Disorders; Seizures; Vertigo; Vestibular Diseases; Labyrinth Diseases; Ear Diseases; Otorhinolaryngologic Diseases; Confusion; Neurobehavioral Manifestations; Neurocognitive Disorders; Mental Disorders; Trigeminal Nerve Diseases; Facial Neuralgia; Facial Nerve Diseases; Cranial Nerve Diseases; Neuroinflammatory Diseases; Intracranial Hemorrhages; Hemorrhage

Study Officials

• Mauricio F Farez, MD MPH

  Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia

  PRINCIPAL INVESTIGATOR

Study Design

• Study Type: interventional
• Phase: early phase 1
• Allocation: NON RANDOMIZED
• Masking: NONE
• Purpose: DIAGNOSTIC
• Intervention Model: SINGLE GROUP
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: PI

Model Details: The model of this Phase 1 interventional study involves evaluating the diagnostic accuracy and utility of an AI-powered virtual assistant in the triage of emergency neurological conditions. Patients and healthcare professionals interacted with the assistant, providing real-world clinical scenarios to assess its effectiveness. The study's design aimed to demonstrate the assistant's potential to enhance emergency neurological care by comparing its performance with established diagnostic tools and patient outcomes.

Study Record Dates

• First Submitted: March 6, 2024
• First Posted: March 28, 2024
• Study Start: October 1, 2023
• Primary Completion: January 1, 2024
• Study Completion: January 1, 2024
• Last Updated: March 28, 2024

Record last verified: 2024-03

Data Sharing

• IPD Sharing: Will not share

Locations

AR (1)