Inside Google's Passive Heart Rate Monitoring Technology

Google has released research on technology that could measure heart rate during normal smartphone use. The method operates without active user participation.

Resting heart rate acts as a biomarker of cardiovascular health and long-term risk. According to the research, high resting heart rate links to adverse cardiovascular events and certain conditions.

Around five billion people worldwide own a smartphone capable of monitoring their resting heart rate. This could expand access to health tracking.

Developing the monitoring model

Google demonstrated in 2022 how smartphones can measure heart rate when a user places a finger over the camera. The new research introduces passive heart rate monitoring that enables tracking in the background during smartphone use.

The system uses the front-facing camera to capture video of the user's face. Deep learning estimates heart rate with a mean absolute percentage error of less than 10%.

According to Google, this meets industry standards for people of all skin tones. The company tested the technology across different demographics.

Eric S Teasley, Product Manager and Ming-Zer Poh, Staff Research Scientist at Google Research, say: "To our knowledge, PHRM marks the first large-scale demonstration of passive HR and daily RHR monitoring during everyday smartphone use.

"As the only rPPG (remote photoplethysmography) method to meet heart rate accuracy standards for people of all skin tones – even in unpredictable real-world conditions – it sets a new standard for the field. It also represents the first use of rPPG to estimate daily RHR, achieving wearable-level accuracy across all skin tones.

"By combining an understanding of user habits with cutting-edge deep learning techniques and an inclusive design, we've developed a smartphone-based heart rate monitoring system that enables wearable-like heart health insights."

How the technology works

Passive heart rate monitoring measures heart rate by sensing fluctuation in how light interacts with skin each time blood pulses through it. The system uses on-device software that processes short clips of facial video.

Temporal shift convolutional neural networks predict heart rate from the processed clips. The technology operates through remote photoplethysmography.

Previous studies underrepresented people with dark skin tones. Melanin makes the signals more challenging for cameras to detect.

Google researchers developed passive heart rate monitoring with more than 350,000 video clips from nearly 700 participants. The team tested participants in both laboratory and real-world settings.

Model training focused on the most challenging and complex cases. Researchers used the Monk Skin Tone Scale to ensure representation.

Participants with light skin tones comprised at least 25% of the datasets. Medium skin tone participants comprised at least 25%.

Dark skin tone participants comprised at least 33% of the datasets. According to Google, this makes the study the largest and most diverse remote photoplethysmography study to date.

Testing across different conditions

Google researchers trained passive heart rate monitoring to handle various conditions in laboratory settings. The team recorded facial video and simultaneous electrocardiogram data from 365 study participants.

According to the research, passive heart rate monitoring outperformed 15 of the leading published remote photoplethysmography models on the same test. The team also trained the model on real-world data.

A free-living study involved 231 participants. These participants installed a custom data collection app on their phones and used them normally.

Participants wore an electrocardiogram chest strap and a Fitbit tracker during the study. The app recorded an average of 231 video clips per day.

Future research could explore optimising camera exposure. The team could also increase success when there is excessive head movement from participants.

Google intends to make its data and modelling resources available to qualified researchers. This could enable further research in the field.