Objective: Recent technological advances have led to the proliferation of ambulatory devices for non-invasively assessing cardiac activity. While these devices have exciting implications for conducting research outside the laboratory, it is critical that this increased mobility does not compromise data quality. As a test case, we assess the efficacy of Empatica’s E4, a high-end wristband device designed to assess Heart Rate Variability (HRV) through the use of photoplethysmography. Approach: We compare the E4 to traditional, wired electrocardiogram measures across a variety of conditions, including seated, supine, and standing baselines, as well as typing and grip strength tasks. Most importantly, we introduce and demonstrate the efficacy of a new method for determining the amount of error in HRV estimates derived from the E4 and a technique for adjusting error tolerance. Main Results: Results indicate that the E4 is severely compromised by motion artifact, resulting in a high percentage of missing data across all conditions except seated and supine baselines. Employing error adjustment yielded more robust results, but at the cost of significantly reducing sample size where motion artifact was present. Significance: These results call into question the wristband’s efficacy as an HRV measurement tool in most in-vivo conditions. We recommend that researchers interested in using photoplethysmography-based HRV devices use caution and evaluate the data quality using methods for error detection and tolerance, such as the one presented here.Keywords: heart rate variability, ambulatory photoplethysmography, electrocardiogram.