These tools will no longer be maintained as of December 31, 2024. Archived website can be found here. PubMed4Hh GitHub repository can be found here. Contact NLM Customer Service if you have questions.


Pubmed for Handhelds

PUBMED FOR HANDHELDS

Search MEDLINE/PubMed

  • Title: Multicenter Evaluation of Machine-Learning Continuous Pulse Rate Algorithm on Wrist-Worn Device.
    Author: Chen W, Cordero R, Lever Taylor J, Pangallo DR, Picard RW, Cruz M, Regalia G.
    Journal: Digit Biomark; 2024; 8(1):218-228. PubMed ID: 39670276.
    Abstract:
    INTRODUCTION: Though wrist-worn photoplethysmography (PPG) sensors play an important role in long-term and continuous heart rhythm monitoring, signals measured at the wrist are contaminated by more intense motion artifacts compared to other body locations. Machine learning (ML)-based algorithms can improve long-term pulse rate (PR) tracking but are associated with more stringent regulatory requirements when intended for clinical use. This study aimed to evaluate the accuracy of a digital health technology using wrist-worn PPG sensors and an ML-based algorithm to measure PR continuously. METHODS: Volunteers were enrolled in three independent clinical trials and concurrently monitored with the investigational device and FDA-cleared electrocardiography (ECG) devices during supervised protocols representative of real-life activities. The primary acceptance threshold was an accuracy root-mean-square (ARMS) ≤3 beats per minute (bpm) or 5 bpm under no-motion and motion conditions, respectively. Bias, mean absolute error (MAE), mean absolute percentage error (MAPE), limits of agreement (LoA), and Pearson and Lin's concordance correlation coefficients (⍴ and CCC) were also computed. Subgroup and outlier analyses were conducted to examine the effect of site, skin tone, age, sex, body mass index (BMI), and health status on PR accuracy. RESULTS: Collectively, 16,915 paired observations between the device and the reference ECG were analyzed from 157 subjects (male: 49.04%, age mean: 43 years, age range: 19-83 years, BMI mean: 26.4, BMI range: 17.5-52, Fitzpatrick class V-IV: 22.9%, cardiovascular condition: 24%). The PR output attained an accuracy of 1.67 bpm under no-motion (n = 5,621 min) and 4.39 bpm under motion (n = 11,294 min), satisfying the acceptance thresholds. Bias and LoA (lower, upper LoA) were -0.09 (-3.36, 3.17) bpm under no-motion and 0.51 (-8.05, 9.06) bpm under motion. MAE was 0.6 bpm in no-motion and 1.77 bpm in motion, and MAPE was 0.86% in no-motion and 2.05% in motion, with ⍴ and CCC >0.98 in both conditions. ARMS values met the clinical acceptance threshold in all relevant subgroups at each clinical site separately, excluding male subjects under motion conditions (ARMS = 5.41 bpm), with more frequent and larger outliers due to stronger forearm contractions. However, these mostly occurred in isolation and, therefore would not impact the clinical utility or usability of the device for its intended use of retrospective review and trend analysis (⍴ and CCC >0.97 and MAPE = 2.61%). CONCLUSION: The analytical validation conducted in this study demonstrated clinical-grade accuracy and generalizability of ML-based continuous PR estimations across a full range of physical motions, health conditions, and demographic variables known to confound PPG signals, paving the way for device usage by populations most likely to benefit from continuous PR monitoring. Monitoring heart rate continuously and unobtrusively is vital for managing various health conditions and for advancing clinical research. Traditional methods for tracking heart rate, like electrocardiography monitors, are often bulky and inconvenient for continuous use. Optical sensors embedded in wristbands conveniently measure PR, a proxy of heart rate. However, the accuracy of these devices for clinical care or research purposes has been poorly established to date, with limitations including a lack of data on accuracy in motion conditions and across demographic factors such as skin tone, as well as a lack of established methodological and accuracy standards from medical device regulators. This validation study aimed to test the accuracy of a new wrist-worn medical device that analyzes optical sensor data with a ML algorithm to continuously monitor PR. The study involved 157 participants who were monitored with the wristband and an electrocardiography monitor simultaneously during different activities, such as resting, walking, and activities of daily living. To make sure that the ML algorithm was equally accurate in different patient populations, data were collected from a range of participants (three independent sites, varying demographic and clinical traits), and accuracy was evaluated using multiple performance indicators and subgroup analyses. Results demonstrated clinical-grade accuracy and generalizability of the device, paving the way for device usage by populations most likely to benefit from continuous PR monitoring. This study significantly advances efforts to transparently report on the methodologies and performance of digital health technologies for PR monitoring.
    [Abstract] [Full Text] [Related] [New Search]