Benchmarking The Atlas Wristband's Heart Rate Monitor

Heart rate monitoring has been integral to professional athlete and gym rat training regimens for years. Now heart rate is widely accepted as a must-have metric for anyone trying to maintain health and wellness or improve overall fitness. Heart rate can give you real-time insight into workout intensity as well as post-workout info such as calories burned and much more.

We've received many inquiries regarding the accuracy of the heart rate monitor (HRM) built into the Atlas Wristband. The short answer is that it is extremely accurate. We are going to go into detail about just how accurate in this article. However, it must be noted that HRM accuracy is dependent on many external factors such as body-type, density of hair on the wrist, skin tone, tattoos, and things such as high motion, contraction of your forearm muscle, and improper wear.

In this article, we will share what goes into developing a quality wrist-based optical heart rate monitor and will share testing results and tips as to how to get the most out of the HRM built right into the Atlas Wristband Super-Tracker. We've also included a brief comparison with category leader Fitbit, which we found particularly relevant given all of the recent news.

Heart Rate Sensors
The Atlas Wristband has a state of the art optical heart rate sensor designed right into the tracker module. (see image below) The core technology is very similar to the finger clip gadget with the red light that you may have seen at the doctor's office. This is called a "pulse oximeter" and just recently over the last 2-3 years, it has become available on the wrist. The wrist is notorious for being one of the most challenging places to extract pulse rate due to the poor signal quality and high amounts of noise. Atlas has tackled this challenge by designing our sensor from the ground up to be one of the most accurate wrist-based optical heart rate monitors on the market, especially for active people. Designing an accurate optical heart rate monitor requires a breadth of knowledge and expertise ranging from mechanical engineering, optical engineering, electrical front end, software and algorithms, and finally manufacturing and quality control. Given these demands, Atlas partnered with Valencell, an expert in the field of biometrics, to assist in creating an industry-leading optical heart rate monitor for the Atlas Wristband.

How it Works
The capillaries in your wrist expand and contract based on the volume of blood in your vessels at any given time - the rate at which they expand and contract is equivalent to your heart rate in beats per minute (BPM). The sensor works by emitting light into the capillaries in your wrist and measuring the amount of light reflected back. The light reflected back is proportional to the amount of blood in your capillaries. Our software then detects the peaks and valleys of the volume of blood in your capillaries, to determine your heart rate. You can learn more about this from our partner here.



High Tech Monitoring for High-Intensity Athletes
The Atlas Wristband optical heart rate technology is designed specifically for high motion environments. Our sensors are EKG accurate when resting, and achieve accuracy comparable to a chest strap when active (see below). The Atlas Wristband uses inertial motion sensors in combination with the heart rate sensor to cancel out excessive motion for a more accurate reading. This is a form of Biometric Signal Extraction called Motion Artifact Removal. This is where most heart rate monitoring technologies fail. When you start sprinting or doing pushups, most devices will lose the heart rate signal due to the crazy amount of motion artifact noise. Additional challenges arise when certain motions have significant forearm muscle activation. We've solved many of the issues with motion to provide one of the most accurate wrist-based heart rate sensors on the market and we have every intention to keep making it better. Additionally, to compensate for a wide range of human body types and demographics, we use both green and amber LEDs to maximize accuracy across different skin tones.

Getting the Best Results
The positioning of the heart rate sensor can have a large impact on the accuracy of the measurement. The wristband technology is designed specifically for the wrist of the wearer. The ideal location of the Atlas Wristband's sensor is high up on the wrist of your left arm. The higher up the wrist towards the forearm and elbow, the better. For best results tighten the strap so that the sensor module is stable and does not slide on the skin, but not so tight as to constrict blood circulation. Less motion of the device relative to your wrist means a more accurate sensor.



Ensure the sensor face (on the bottom of the sensor medallion in the previous picture) is lying directly against the skin with no gap.

The sensor will still perform when worn in positions near the recommended position, however, the closer to the recommended position, the more likely the sensor is to perform accurately.

Experimental Setup
To assess the accuracy of the Atlas Wristband's HRM we need to be able to compare our output to the actual heart rate of the wearer. We used the Polar H7 chest-strap to provide this ground truth heart rate reading because chest-straps are an older technology, regarded by most to be more accurate than their optical counterparts. For this post, we focused on assessing accuracy while running. Our data collections process was highly scientific; we strapped the Atlas Wristband and Polar H7 onto some poor soul and put them to work. To track heart rate while running we placed the Atlas Wristband in Heart Rate Mode.

NB: Making liberal use of Heart Rate Mode is a great way to get the most out of the Atlas Wristband as it allows you to monitor heart rate and caloric burn when doing activities which aren't currently recognized. But don't fret, the Atlas Wristband tracks your heart rate in all workout modes!

Collecting Chest Strap HR Data
We used an app available on the Google Play store called BLE Heart Rate Monitor to collect the data from the 3rd party device (the chest strap) and export it to a CSV file. You can use any app that supports the Bluetooth Low Energy Heart Rate Profile (HRP). There is also ANT+ (a wireless protocol similar to Bluetooth) heart rate monitors, but you will need a different way of logging the data.

Collecting Atlas HR Data
Atlas Wearables has developed an open web-based API to make the transfer of information as seamless as possible. You can find documentation for our API here and can request your own sid and secret using the instructions provided on that page.

With a user account, sid and secret in hand, retrieving heart rate data from Atlas' API is simple. Below is a minimal example using Python and the requests library.


Comparison
Now that we've got the data we can actually do some analysis. Because we're interested in understanding how well the heart rate monitor works, the primary quantity of interest will be the deviation between the BPM reported at each point in time by the Atlas Wristband and whatever reference we're comparing with. Two common metrics will be used to assess this deviation, in both cases treating the output of the chest-strap as the true BPM. We'll denote the output of the reference device as y1,…,yT and the output of the Atlas Wristband as ŷ 1,…,ŷ T.

Mean Absolute Error
The first metric we'll use is Mean Absolute Error (MAE). MAE is simply the average of the absolute difference between the true BPM (reference device) and predicted BPM (Atlas Wristband) at each time.


We chose MAE as it intuitive and gives equal weight to all errors regardless of magnitude, unlike something like Root Mean Squared Error which gives higher weight to large magnitude errors.

Coefficient of Determination
The second metric we'll use is the Coefficient of Determination, commonly denoted as R2. R2 is a common metric used in regression analysis and normally described as measuring the proportion of variance explained by a predictive model. If you've ever taken an introductory level statistics courses you probably discussed R2. Although the use of R2 as a measure of a predictive accuracy is criticized by many (see Section 3 of these lecture notes by Cosma Shalizi for further discussion), we include it here due to its ubiquity.

R2 is derived from the ratio of two quantities: the total sum of squares denoted SStot, and the residual sum of squares denoted SSres.


In general higher R2 corresponds to a better fit. For example, if yt=ŷ t for all t then SSres=0 and R2=1.

Results
Overall we were quite pleased with the performance of the Atlas Wristband's heart rate monitor. The MAE was below 2.5 BPMs and R2 was greater than 0.9 in all of our tests. As can be seen in the images below the BPMs reported by the Atlas Wristband closely follow the inflection points of the more reliable chest-strap.

Finally, given all of the recent press questioning Fitbit's accuracy, we did a quick comparison (yes, only sample size of one) of our technology to Fitbit. In this case, our test subject went on a run with the Atlas Wristband on their left wrist, and the Fitbit Surge on the right, both worn snug, similar to the recommendation on both Atlas' and Fitbit's knowledge base. Check it out below, you be the judge!



Final Thoughts
The results reported above demonstrate the Atlas Wristband's HRM is comparable in accuracy to a chest-strap, however, there are several things that should be pointed out. First, optical heart rate monitoring is a relatively young consumer technology and there can be significant variation in accuracy depending on factors such as user, position, and intensity of activity. In the future, we plan to present benchmark results of the Atlas Wristband's HRM in settings such as strength training and high-intensity interval training. The second thing that should be noted is that a chest-strap HRM is also an indirect measure of BPMs, and is therefore not guaranteed to be 100% accurate.

That being said we hope this post has been useful for those interested in how the Atlas stacks up to other options for monitoring heart rate. If you decide to do any of your own experiments drop us a line, we'd love to hear about it.