AI for Parkinson’s Disease and Movement Disorders

Parkinson’s disease (PD) is a progressive neurodegenerative disorder. Its hallmark motor symptoms, such as shuffling gait, rigidity and loss of fine motor control, are accompanied by complications that grow with disease progression and reduced mobility, including fatigue, muscle weakness and respiratory dysfunction; respiratory complications such as pneumonia are among the leading causes of death in advanced PD. Our research applies AI, computer vision and mobile sensing across this spectrum: interpretable video-based gait analysis that recognizes pathological gait patterns such as Parkinsonian gait and backs each diagnosis with measured biomechanical evidence, assistive robots that users with severe motor impairments can direct with their eye gaze and personalize through natural language feedback, and smartphone-based tools that examine the airway and monitor lung function and muscle fatigue at home.

ProGait

July 2025
ProGait is a multi-purpose video dataset aimed to support multiple vision tasks on prosthesis users, including Video Object Segmentation, 2D Human Pose Estimation, and Gait Analysis.

ProGait provides 412 video clips from four above-knee amputees when testing multiple newly-fitted prosthetic legs through walking trials, and depicts the presence, contours, poses, and gait patterns of human subjects with transfemoral prosthetic legs. It is also one of the two benchmarks used to evaluate DrGait’s interpretable gait analysis.

ProGait dataset example

Acoustic Waveform Respiratory Evaluation (AWARE)

January 2024
The Acoustic Waveform Respiratory Evaluation (AWARE) dataset consists of a group of human airway measurements, produced by our integrated AI and sensing systems for smart pulmonary telemedicine.

This dataset contains airway measurements of 382 human subjects, including patients with various pulmonary diseases and healthy control subjects, recruited from the Children’s Hospital of Pittsburgh during the past 3 years. Each measurement reconstructs the airway’s cross-sectional area from the mouth through the pharynx and glottis to the trachea, covering the upper airway segment in which obstruction has been reported in people with PD.

AWARE Setup

Gaze-Guided Intent Prediction for Individuals with Severe Motor Impairments
We bring gaze-guided task suggestion for assistive robots into longitudinal clinical deployments with people living with ALS, spinal cord injuries or Parkinson’s disease. By tracking how motor degeneration affects egocentric gaze and head coordination over time, we adapt our filtering and visual-language algorithms to tolerate pathological gaze noise such as jitter, saccadic intrusions and fixational drift.
September 2026 In arXiv
PDF
Gaze-Guided Intent Prediction for Individuals with Severe Motor Impairments
DrGait: Biomechanically Grounded Visual Reasoning for Interpretable Clinical Gait Analysis
DrGait turns a vision-language model into a clinical planner for video-based gait analysis: instead of judging raw video, it verifies each diagnostic hypothesis with biomechanical measurements from 3D body meshes and 2D poses, and produces an interpretable, evidence-backed report. On the GAVD benchmark it recognizes Parkinsonian gait with 94.1% accuracy, close to human experts (96.1%).
September 2026 In arXiv
PDF
DrGait: Biomechanically Grounded Visual Reasoning for Interpretable Clinical Gait Analysis
Low-Burden LLM-Based Preference Learning: Personalizing Assistive Robots from Natural Language Feedback for Users with Paralysis
We developed an LLM-based framework that personalizes assistive robots using natural language feedback, significantly reducing the physical and cognitive fatigue associated with traditional methods. Grounded in clinical reasoning, the system translates unstructured user input into safe, deterministic control policies for individuals with profound motor impairments.
August 2026 In RO-MAN 2026
PDF Cite Code DOI
Low-Burden LLM-Based Preference Learning: Personalizing Assistive Robots from Natural Language Feedback for Users with Paralysis
Out-Clinic Pulmonary Disease Evaluation via Acoustic Sensing and Multi-Task Learning on Commodity Smartphones
This work presents AWARE, a new sensing and AI system that supports accurate and reliable pulmonary function tests (PFT) using commodity smartphones. AWARE uses a smartphone to transmit acoustic signals and reconstructs the profile of human airway based on the analysis of reflected acoustic waves captured from the smartphone’s microphone. The subject’s pulmonary condition is then evaluated by a multi-task learning model to detect airway obstruction and inflammation and identify pulmonary diseases like asthma or COPD.
November 2022 In SenSys'22
PDF Cite DOI
Out-Clinic Pulmonary Disease Evaluation via Acoustic Sensing and Multi-Task Learning on Commodity Smartphones
MyoMonitor: Evaluating Muscle Fatigue with Commodity Smartphones
We present a new technique to mimic an active sonar system with the smartphone’s built-in microphone and speaker, and use this sonar system to evaluate muscle fatigue from the muscle’s surface characteristics that can be measured from the transmitted acoustic signal.
March 2021 In Smart Health
PDF Cite DOI Code
MyoMonitor: Evaluating Muscle Fatigue with Commodity Smartphones
SpiroSonic: monitoring human lung function via acoustic sensing on commodity smartphones
The first work that allows commodity smartphones to be used as a portable spirometer and provide accurate lung function test results on par with clinical-grade spirometers. This is a collaborative work with the Children’s Hospital of Pittsburgh, and could also potentially contribute to in-home evaluation of COVID-19 risks by allowing convenient out-of-clinic lung function evaluation.
April 2020 In MobiCom'20
PDF Cite Video DOI
SpiroSonic: monitoring human lung function via acoustic sensing on commodity smartphones