Mobile and Connected Health

Recent technical advances of sensing, computation and communication on mobile and embedded devices, such as smartphones and wearables, highlights the possibility of pervasive monitoring and unobtrusive diagnostics of various acute or chronic diseases, as convenient yet low-cost alternatives of medical-grade methods without any involvement of clinicians. Our research aims to fully unleash such potential of today’s mobile and embedded devices towards accurate, efficient yet cost-effective solutions to mobile and connected health, by employing modern AI tools and developing new AI algorithms to properly extract biomarkers from the mobile sensory data and provide sufficient interpretability to the extracted biomarkers. Currently, our integrated sensing and AI systems have been widely applied to various clinical applications including pulmonary telemedicine, post-discharge heart failure risk evaluation and mitigation, and orthopedic disease evaluation.

ProGait

July 2025
ProGait is a multi-purpose video dataset aimed to support multiple vision tasks on prosthesis users, including Video Object Segmentation, 2D Human Pose Estimation, and Gait Analysis.

ProGait provides 412 video clips from four above-knee amputees when testing multiple newly-fitted prosthetic legs through walking trials, and depicts the presence, contours, poses, and gait patterns of human subjects with transfemoral prosthetic legs.

ProGait dataset example

Acoustic Waveform Respiratory Evaluation (AWARE)

January 2024
The Acoustic Waveform Respiratory Evaluation (AWARE) dataset consists of a group of human airway measurements, produced by our integrated AI and sensing systems for smart pulmonary telemedicine.

This dataset contains airway measurements of 382 human subjects, including patients with various pulmonary diseases and healthy control subjects, recruited from the Children’s Hospital of Pittsburgh during the past 3 years.

AWARE Setup

ProGait: A Multi-Purpose Video Dataset and Benchmark for Transfemoral Prosthesis Users
Vision-based machine learning (ML) methods offer a scalable and non-invasive solution to gait analysis, but face challenges in correctly detecting and analyzing prosthesis, due to their unique appearances and new movement patterns. In this paper, we aim to bridge this gap by introducing a multi-purpose dataset, namely ProGait, to support multiple vision tasks including Video Object Segmentation, 2D Human Pose Estimation, and Gait Analysis (GA). ProGait provides 412 video clips from four above-knee amputees when testing multiple newly-fitted prosthetic legs through walking trials, and depicts the presence, contours, poses, and gait patterns of human subjects with transfemoral prosthetic legs. Alongside the dataset itself, we also present benchmark tasks and fine-tuned baseline models to illustrate the practical application and performance of the ProGait dataset.
July 2025 In ICCV 2025
PDF Cite Code DOI Dataset
ProGait: A Multi-Purpose Video Dataset and Benchmark for Transfemoral Prosthesis Users
PTEase: Objective Airway Examination for Pulmonary Telemedicine using Commodity Smartphones
The first mobile health system that turns a commodity smartphone into a fully functional pulmonary examination device to measure the internal physiological conditions of human airways, such as airway caliber, obstruction and possible inflammation. Information about these airway conditions could provide vital clues for precise and objective pulmonary disease evaluation.
June 2023 In MobiSys'23
PDF Cite Code Video DOI
PTEase: Objective Airway Examination for Pulmonary Telemedicine using Commodity Smartphones
Out-Clinic Pulmonary Disease Evaluation via Acoustic Sensing and Multi-Task Learning on Commodity Smartphones
This work prsents AWARE, a new sensing and AI system that supports accurate and reliable pulmonary function tests (PFT) using commodity smartphones. AWARE uses a smartphone to transmit acoustic signals and reconstructs the profile of human airway based on the analysis of reflected acoustic waves captured from the smartphone’s microphone. The subject’s pulmonary condition is then evaluated by a multi-task learning model to detect airway obstruction and inflammation and identify pulmonary diseases like asthma or COPD.
November 2022 In SenSys'22
PDF Cite DOI
Out-Clinic Pulmonary Disease Evaluation via Acoustic Sensing and Multi-Task Learning on Commodity Smartphones
MyoMonitor: Evaluating Muscle Fatigue with Commodity Smartphones
We present a new technique to mimic an active sonar system with the smartphone’s built-in microphone and speaker, and use this sonar system to evaluate muscle fatigue from the muscle’s surface characteristics that can be measured from the transmitted acoustic signal.
March 2021 In Smart Health
PDF Cite DOI Code
MyoMonitor: Evaluating Muscle Fatigue with Commodity Smartphones
SpiroSonic: monitoring human lung function via acoustic sensing on commodity smartphones
The first work that allows commodity smartphones to be used as a portable spirometer and provide accuracy lung function test results on par with clinical-grade spirometers. This is a collaborative work with the Children’s Hospital of Pittsburgh, and could also potentially contribute to in-home evaluation of COVID-19 risks by allowing convenient out-of-clinic lung function evaluation.
April 2020 In MobiCom'20
PDF Cite Video DOI
SpiroSonic: monitoring human lung function via acoustic sensing on commodity smartphones