SeeMe: Using Automated Facial Tracking to Detect Voluntary Behavior in Brain Injury

NCT07560631 | Recruiting DMC FDA Device

• 2 other identifiers: IRB2019--001991R61MH138612-01
• Study Type: interventional
• Target: 80
• Locations: 1 country
• Sites: 1

Brief Summary

Objective: This prospective interventional study introduces "SeeMe," an automated, high-resolution computer vision platform designed to objectively quantify microscopic, auditory command-evoked movements in patients with Traumatic Brain Injury (TBI). Current clinical assessments, such as the Glasgow Coma Scale (GCS) and Coma Recovery Scale-Revised (CRS-R), rely on subjective human observation and often fail to detect low-amplitude motor responses, potentially misclassifying up to 25% of patients as unresponsive. Methodology: SeeMe utilizes vector analysis, cross-correlation, and deep neural networks (DNNs) to track individual facial pores and hand movements with sub-millimeter precision (0.5 mm) and high temporal resolution (0.03s). The study will enroll a cohort of 60-80 TBI patients, alongside healthy controls and pharmacologically paralyzed subjects, to validate SeeMe's sensitivity and specificity. Primary Goals:

1. 1.Validation: Compare SeeMe's detection of voluntary motor recovery against gold-standard clinical examinations (CRS-R).
2. 2.Synchronization: Simultaneously record and time-lock electroencephalography (EEG) and electrocorticography (ECoG) with SeeMe-detected movements.
3. 3.Biomarker Identification: Characterize neural signatures (specifically Beta-band oscillations) associated with the return of voluntary behavior.

Conditions

Traumatic Brain Injury (TBI) Patients; Severe Traumatic Brain Injury

Keywords

Traumatic Brain Injury; Disorders of Consciousness; Coma; Cognitive-Motor Dissociation; Covert Awareness; Computer Vision; Deep Learning; Artificial Intelligence

Outcome Measures

Primary Outcomes (1)

• Lead Time to Detection of Command-Following by SeeMe Compared With Blinded CRS-R Assessment

  Number of days between the first SeeMe-detected significant stimulus-evoked motor response and the first blinded Coma Recovery Scale-Revised (CRS-R) assessment demonstrating command-following. A SeeMe-positive detection is defined as a stimulus-evoked movement that meets the prespecified criteria of a Kolmogorov-Smirnov statistic \>0.1 and a pixel displacement \>400. This stimulus-evoked movement must be detected reliably (at least 3 out of 10 trials). CRS-R examiners will be blinded to SeeMe outputs.

  From Day 1 until hospital discharge, typically within 45 days.

Secondary Outcomes (7)

• Diagnostic Performance of SeeMe for Subsequent Confirmed Command-Following

  From Day 1 until hospital discharge, typically within 45 days.

• Rate of SeeMe-Detected Responsiveness in Participants Without Observable Command-Following at Enrollment

  From Day 1 until hospital discharge, typically within 45 days.

• False Positive Rate of SeeMe in Pharmacologically Paralyzed Controls

  During the localized 60-minute window of the surgical anesthesia procedure.

• Correlation Between SeeMe Motor Initiation and Beta-Band Event-Related Desynchronization (ERD)

  At each synchronized recording session from Day 1 through hospital discharge, typically within 45 days.

• Multi-Class Command Classification Performance of the Bidirectional LSTM Model

  Assessed at the end of R61 model development and validation, approximately Year 3.

Study Arms (3)

TBI Patients

EXPERIMENTAL

Patients with Traumatic Brain Injury receiving the SeeMe Auditory Stimulation protocol daily to detect covert motor responses

Diagnostic Test: SeeMe Multimodal Auditory Command Protocol

Healthy Control Cohort

ACTIVE COMPARATOR

Awake, healthy volunteers receiving the SeeMe protocol to establish "ground truth" for normal voluntary motor signatures and algorithm sensitivity.

Diagnostic Test: SeeMe Multimodal Auditory Command Protocol

Sedated/Paralyzed Patients

SHAM COMPARATOR

Patients undergoing general anesthesia and pharmacological paralysis receiving the SeeMe protocol to establish the algorithm's specificity and "noise floor."

Diagnostic Test: SeeMe Multimodal Auditory Command Protocol

Interventions

SeeMe Multimodal Auditory Command Protocol DIAGNOSTIC_TEST

A standardized, computer-controlled auditory stimulation (AS) protocol designed to elicit and quantify microscopic motor responses. Protocol Details: Stimuli: Participants are presented with five distinct auditory commands: 1) 'Stick out your tongue,' 2) 'Open your eyes,' 3) 'Show me a smile,' 4) 'Close your hands,' and 5) a neutral control command ('Today is a sunny day'). Timing: Each command is presented 10 times via single-use headphones with a randomized 30-45 second jittered interval between trials to distinguish stimulus-evoked responses from spontaneous arousal. Data Capture: Responses are captured using high-resolution video (Panasonic HC-2000X) at 0.03s temporal resolution and synchronized millisecond-level EEG/ECoG. Analysis: Displacement heatmaps are generated via facial pore vector analysis and classified using a bidirectional long short-term memory (LSTM) neural network to determine the statistical significance of motor initiation compared to a 15-minute resting base

Also known as: Computer-Vision Based Consciousness Assessment

Healthy Control Cohort; Sedated/Paralyzed Patients; TBI Patients

Eligibility Criteria

• Age: 22 Years - 85 Years
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• Adults (22+) with a history of acute traumatic brain injury
• Documented loss of consciousness with a Glasgow Coma Scale (GCS) score less than or equal to 8 upon hospitalization
• Clinically stable as determined by the primary neurosurgery or ICU team
• Intact auditory pathways as confirmed by BAERs
• Family consent for study participation

You may not qualify if:

• Hearing Impairment confirmed via absence of Brainstem Auditory Evoked Responses (BAERs) that would prevent the patient from hearing the auditory commands
• No legal authorized representative (LAR) available to provide informed consent for the patients in a comatose state
• Any other medical condition that, in the judgment of the investigator, makes participation in the study unsafe.
• Pregnant women
• Any previous history of traumatic brain injury
• Any neurodegenerative disease such as dementia
• Group 2: Healthy Control Cohort
• Adults 22+ with no history of neurological or psychiatric disorders
• Normal baseline neurological examination
• Intact auditory pathways
• Ability to provide informed consent
• Ability to follow simple auditory commands in English
• Hearing Impairment that would prevent the participant from hearing the auditory commands
• Any previous history of severe traumatic brain injury (TBI)
• Any neurodegenerative disease (e.g., dementia)

Sponsors & Collaborators

• Stony Brook University: lead
• National Institute of Mental Health (NIMH): collaborator

Study Sites (1)

Stony Brook University Hospital

Stony Brook, New York, 11794, United States

RECRUITING

MeSH Terms

Conditions

Brain Injuries, Traumatic; Consciousness Disorders; Coma

Condition Hierarchy (Ancestors)

Brain Injuries; Brain Diseases; Central Nervous System Diseases; Nervous System Diseases; Craniocerebral Trauma; Trauma, Nervous System; Wounds and Injuries; Neurobehavioral Manifestations; Neurologic Manifestations; Signs and Symptoms; Pathological Conditions, Signs and Symptoms; Neurocognitive Disorders; Mental Disorders; Unconsciousness

Central Study Contacts

Sima Mofakham, PhD

CONTACT

631-444-1278; sima.mofakham@stonybrookmedicine.edu

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: NON RANDOMIZED
• Masking: SINGLE
• Who Masked: OUTCOMES ASSESSOR
• Purpose: DIAGNOSTIC
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Associate Professor of Neurosurgery

Study Record Dates

• First Submitted: April 17, 2026
• First Posted: May 1, 2026
• Study Start: March 30, 2026
• Primary Completion (Estimated): December 1, 2029
• Study Completion (Estimated): December 1, 2029
• Last Updated: May 5, 2026

Record last verified: 2026-04

Data Sharing

• IPD Sharing: Will share
• Shared Documents: STUDY PROTOCOL, SAP, ICF, ANALYTIC CODE
• Time Frame: Data will be made available beginning 6 months after the primary manuscript is published and will remain accessible for a period of 5 years. This allows the primary research team time to complete secondary analyses while ensuring the data remains useful for the scientific community during the R33 transition phase.
• Access Criteria: Data will be shared with researchers who provide a methodologically sound proposal and have obtained IRB approval from their home institution. Proposals must be directed to the Principal Investigator (Sima Mofakham). To gain access, data requestors will need to sign a Data Use Agreement (DUA) that strictly prohibits any attempt to re-identify participants or use raw signal data for non-academic purposes.

Locations

US (1)