ACE-Sponsored Study Evaluates Accuracy of Popular Fitness Trackers

Editor’s Note: Today, fitness trackers are ubiquitous, but one question continues to plague users: Are the data they track, such as steps and calories burned, accurate? Recently, a study sponsored by the American Council on Exercise (ACE) evaluated five popular fitness trackers on the market to determine their true value. VCBeat has compiled the contents of the research report to help you gain a deeper understanding of the actual performance currently achievable by fitness trackers.

In modern society, there is widespread recognition of the value of healthier lifestyles. A growing number of people are turning to fitness trackers to monitor metrics such as calorie expenditure and sleep quality, helping them adhere to their exercise regimens and achieve their fitness and health goals.

It is estimated that a total of 19 million activity trackers were put into use in 2014, and this figure is expected to multiply in the coming years. According to a recent report by Juniper Research, the usage of sports trackers, or fitness wearables, is projected to be three times higher in 2018 than in 2014.

These new technologies are undeniably cool, and some are even enjoyable to use; however, few published studies to date have validated the accuracy and efficacy of these devices. How accurate are fitness trackers in predicting caloric expenditure or counting steps? In light of their growing popularity, the American Council on Exercise (ACE) recruited a team of researchers from the Department of Clinical Exercise Physiology at the University of Wisconsin–La Crosse to evaluate five popular fitness trackers currently on the market.

Five popular activity trackers on the market were selected as subjects for this study: Nike+ Fuelband ($99–$149), Fitbit Ultra ($99), Jawbone UP ($99), BodyMedia FitCore ($99), and Adidas miCoach ($199). (BodyMedia has been acquired by Jawbone.)

Researchers recruited 10 healthy men and 10 healthy women, aged 18 to 44 years, to participate in the study. They were divided into two groups: one group had their energy expenditure measured, while the other had their step count measured. The two study arms were conducted concurrently using the same protocol.

In addition to wearing activity trackers, the subjects also wore portable metabolic gas analyzers and NL-2000I pedometers. The reliability of these two sets of devices has been demonstrated, enabling accurate measurements of energy expenditure and step counts. While wearing all the aforementioned devices, each subject performed a series of different exercises simultaneously; the test was conducted in two separate phases, each lasting 50 minutes.
The exercise protocol for Phase 1 consisted of walking and running on a level treadmill. Each participant first walked at a self-selected pace for 20 minutes, followed by a 10-minute rest period, and then ran at a self-selected pace for 20 minutes.

The second phase of the exercise program was conducted on an elliptical trainer, a device that simultaneously engages both the arms and legs. Participants first performed 20 minutes of elliptical training at a self-selected intensity. After a rest period, they proceeded with sport-specific agility training, which included agility ladder drills, basketball free throws, suspension training, and half-court layup drills.

After completing the two stages of exercise, the energy expenditure values and step counts recorded by each device will be compared with those recorded by the portable metabolic analyzer.

Reliable Step Count Tracking Data

“Step counts from activity trackers are fairly reliable,” said lead researcher Caitlin Stackpool, M.Sc., “but their accuracy varies depending on the type of exercise performed. All five activity trackers showed errors of less than 10% when counting steps during treadmill walking and running, as well as during elliptical training (Tables 1 and 2).”

However, during agility training, activity trackers exhibited significant errors. Stackpool explained that this may be due to the more complex movement trajectories involved. Activity trackers may fail to record smaller or faster steps; furthermore, since many activity trackers are worn on the arm or wrist, limited arm movement can also lead to inaccurate step counts (among the five activity trackers evaluated in this study, only Fitbit was not a wrist-worn device).

Wide Range of Errors in Energy Expenditure Estimation

Estimates of energy expenditure provided by activity trackers are less accurate (Tables 3 and 4). “Recording energy expenditure is a complex process that involves applying regression equations to the data collected by the device using its built-in software,” explains John P. Porcari, Ph.D., Director of the Clinical Exercise Physiology Program at the University of Wisconsin–La Crosse. This may explain why there is greater variability in calorie expenditure readings across different activity trackers.

Table 3. Comparison of Caloric Expenditure Recorded by Portable Metabolic Gas Analyzers and Activity Trackers

The discrepancy between actual measurements and calorie expenditure predictions by activity tracking devices ranges from 13% to 60%, with some devices overestimating and others underestimating. Among the five devices evaluated, none accurately recorded calorie expenditure. Therefore, if consumers need to track calorie expenditure, fitness trackers may not be their best option.

Activity trackers are better suited for low-intensity exercise.

Researchers advise consumers to first consider what information they wish to track when selecting an activity tracker. If the goal is to record step count, Jawbone UP appears to be the optimal choice. For those more concerned with calorie expenditure, the best option will vary depending on the type of physical activity engaged in.

“Most devices are very good at measuring traditional step counts,” Porcari said. “Once you engage in activities beyond traditional exercise, such as sports-related movements, the step counting of activity tracking devices becomes less accurate.”

“These activity trackers are best suited for low-intensity activities, such as walking,” Stackpool added, noting that these devices are particularly well-suited for beginners. “They provide a way for users to assess their current fitness level, helping them set goals and strive for improvement.”

What about calorie expenditure? “Predicting calorie expenditure is highly complex,” Porcari explained. “For example, the range of arm motion during exercise varies from person to person. Some individuals are more efficient, while others exhibit greater variability. For most devices, the accuracy in estimating calorie expenditure likely falls short by 10% to 15%, due to the substantial biological variability.”

However, this does not mean that using fitness trackers offers no benefits. Stackpool continued, “Fitness trackers can inform people of their activity levels throughout the day. Sitting for 90% of the day and then exercising for 30 minutes does not truly make a person ‘active.’”

By wearing wearable devices throughout the day, individuals can determine whether they need to engage in more physical activity during the day. Research indicates that “individuals who wore activity trackers were 30%–40% more active than those who did not.” Therefore, while these activity-tracking devices may not necessarily be highly accurate in measuring caloric expenditure, they do effectively encourage people to get up and move their bodies.

Stackpool said: “Do everything you can to get moving, no matter how. If wearing an activity tracker helps you achieve this, your best bet is to choose a device that feels comfortable, is easy to use, and has the additional features you want.”(To stay updated on the latest information about internet healthcare startups, please follow VCBeat’s WeChat official account: vcbeat. We also welcome you to engage with us on topics of interest, contact us via WeChat, and share your startup projects or related research insights.)