Gaze-Guided Intent Prediction for Individuals with Severe Motor Impairments

Abstract

For individuals living with severe motor impairments, such as ALS, spinal cord injuries, or Parkinson’s disease, eye gaze serves as one of the few remaining modalities for independent interaction. While eye-tracking technology offers a fast, hands-free way to direct assistive robots, existing models are trained on healthy populations and frequently break down when exposed to clinical gaze abnormalities, such as involuntary jitter, saccadic intrusions, or fixational drift. This project bridges that gap by deploying our gaze-guided task suggestion framework into real-world, longitudinal clinical environments. By tracking egocentric gaze and head coordination during everyday activities, we evaluate how motor degeneration impacts gaze-based intent prediction over time. We use these insights to adapt our filtering and visual-language algorithms to tolerate pathological gaze noise. The resulting system enables robust, low-burden robot control that preserves patient agency without inducing cognitive or physical fatigue.

Keshav Shankar

PhD Student

Wei Gao

Associate Professor

Associate Professor at University of Pittsburgh