Uncovering Systematic Bias in Ratings across Categories

We often quantify a behaviour by counting the number of times it occurs within a specific, short observation period. Measuring behaviour in such a way is typically unavoidable but induces error. This error acts to systematically reduce effect sizes, including metrics of particular interest to behavioural and evolutionary ecologists such as R2, repeatability (intra-class correlation, ICC) and heritability. Through introducing a null model, the Poisson process, for modelling the frequency of behaviour, we give a mechanistic explanation of how this problem arises and demonstrate how it makes comparisons between studies and species problematic, because the magnitude of the error depends on how frequently the behaviour has been observed (e.g. as a function of the observation period) as well as how biologically variable the behaviour is. Importantly, the degree of error is predictable and so can be corrected for. Using the example of parental provisioning rate in birds, we assess the applicability of our null model for modelling the frequency of behaviour. We then review recent literature and demonstrate that the error is rarely accounted for in current analyses. We highlight the problems that arise from this and provide solutions. We further discuss the biological implications of deviations from our null model, and highlight the new avenues of research that they may provide. Adopting our recommendations into analyses of behavioural counts will improve the accuracy of estimated effect sizes and allow meaningful comparisons to be made between studies.

Download Full-text

Are perceived prevalences of infection also biased and how? Lessons from large epidemics of mosquito-borne diseases in tropical regions

10.31234/osf.io/zh84a ◽

2017 ◽

Author(s):

Jocelyn Raude

Keyword(s):

Infectious Diseases ◽

Cognitive Biases ◽

Adult Population ◽

Systematic Bias ◽

Tropical Regions ◽

Telephone Surveys ◽

Behavioral Decision ◽

Risk Estimates ◽

Representative Samples ◽

Actual Prevalence

Objectives: Although people have been repeatedly found to underestimate the frequency of risks to health from common diseases, we still do not know much about reasons for this systematic bias, which is also referred to as “primary bias” in the literature. In this study, we take advantage of a series of large epidemics of mosquito-borne diseases to examine the accuracy of judgments of risk frequencies. In this aim, we assessed the perceived versus the observed prevalence of infection by zika, chikungunya or dengue fever during these outbreaks, as well as their variations among different subpopulations and epidemiological settings.Design: We used data drawn from 4 telephone surveys, conducted between 2006 and 2016, among representative samples of the adult population in tropical regions (Reunion, Martinique, and French Guiana). The participants were asked to estimate the prevalence of these infections by using a natural frequency scale.Results: The surveys showed that (1) most people greatly overestimated the prevalence of infection by arbovirus, (2) these risk overestimations fell considerably as the actual prevalence of these diseases increased, (3) the better-educated and male participants consistently yielded less inaccurate risk estimates across epidemics, and (4) that these biases in the perception of prevalence of these infectious diseases are relatively well predicted by probability weighting function.Conclusions: These findings suggest that the cognitive biases that affect perception of prevalence of acute infectious diseases are not fundamentally different from those that characterize other types of probabilistic judgments observed in the field of behavioral decision-making. They also indicate that numeracy may play a considerable role in people’s ability to transform epidemiological observations from their social environment to more accurate risk estimates.

Download Full-text

An analytical method reduces noise bias in motor adaptation analysis

Scientific Reports ◽

10.1038/s41598-021-88688-5 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Daniel H. Blustein ◽

Ahmed W. Shehata ◽

Erin S. Kuylenstierna ◽

Kevin B. Englehart ◽

Jonathon W. Sensinger

Keyword(s):

Motor Planning ◽

Motor Adaptation ◽

Motor Command ◽

Systematic Bias ◽

Sources Of Information ◽

Noise Masking ◽

Stochastic Signal ◽

Movement Data ◽

Motor Error ◽

Stochastic Signal Processing

AbstractWhen a person makes a movement, a motor error is typically observed that then drives motor planning corrections on subsequent movements. This error correction, quantified as a trial-by-trial adaptation rate, provides insight into how the nervous system is operating, particularly regarding how much confidence a person places in different sources of information such as sensory feedback or motor command reproducibility. Traditional analysis has required carefully controlled laboratory conditions such as the application of perturbations or error clamping, limiting the usefulness of motor analysis in clinical and everyday environments. Here we focus on error adaptation during unperturbed and naturalistic movements. With increasing motor noise, we show that the conventional estimation of trial-by-trial adaptation increases, a counterintuitive finding that is the consequence of systematic bias in the estimate due to noise masking the learner’s intention. We present an analytic solution relying on stochastic signal processing to reduce this effect of noise, producing an estimate of motor adaptation with reduced bias. The result is an improved estimate of trial-by-trial adaptation in a human learner compared to conventional methods. We demonstrate the effectiveness of the new method in analyzing simulated and empirical movement data under different noise conditions.

Download Full-text

Reporting on one's behavior: a survey experiment on the nonvalidity of self-reported COVID-19 hygiene-relevant routine behaviors

Behavioural Public Policy ◽

10.1017/bpp.2021.13 ◽

2021 ◽

pp. 1-18

Author(s):

Pelle Guldborg Hansen ◽

Erik Gahner Larsen ◽

Caroline Drøgemüller Gundersen

Keyword(s):

Measurement Error ◽

Crucial Role ◽

Test Validity ◽

Self Report ◽

Systematic Bias ◽

Survey Experiment ◽

Policy Decisions ◽

Policy Responses ◽

Randomized Controlled ◽

Self Reports

Abstract Surveys based on self-reported hygiene-relevant routine behaviors have played a crucial role in policy responses to the COVID-19 pandemic. In this article, using anchoring to test validity in a randomized controlled survey experiment during the COVID-19 pandemic, we demonstrate that asking people to self-report on the frequency of routine behaviors are prone to significant measurement error and systematic bias. Specifically, we find that participants across age, gender, and political allegiance report higher (lower) frequencies of COVID-19-relevant behaviors when provided with a higher (lower) anchor. The results confirm that such self-reports should not be regarded as behavioral data and should primarily be used to inform policy decisions if better alternatives are not available. To this end, we discuss the use of anchoring as a validity test relative to self-reported behaviors as well as viable alternatives to self-reports when seeking to behaviorally inform policy decisions.

Download Full-text

Comparison between photoplethysmographic heart rate monitor from Polar Vantage M and Polar V800 with H10 chest strap while running on a treadmill: Validation of the Polar Precision PrimeTM photoplestimographic system

Proceedings of the Institution of Mechanical Engineers Part P Journal of Sports Engineering and Technology ◽

10.1177/1754337120976659 ◽

2020 ◽

pp. 175433712097665

Author(s):

Emilio J Ruiz-Malagón ◽

Santiago A Ruiz-Alias ◽

Felipe García-Pinillos ◽

Gabriel Delgado-García ◽

Victor M Soto-Hermoso

Keyword(s):

Heart Rate ◽

Random Error ◽

Maximum Speed ◽

Systematic Bias ◽

Long Distance ◽

Time Intervals ◽

Heart Rate Monitor ◽

Calibration Time

Chest bands have been the most used device to monitor heart rate during running. However, some runners feel uncomfortable with the use of bands due to the friction and pressure exerted on the chest. Thus, the aim of this study was to determine if the photoplethysmography (PPG) system Polar Precision Prime used in the Polar Vantage M watch could replace chest bands (Polar V800-H10) to monitor heart rate with the same precision. A group of 37 people, middle-distance and long-distance professional runners, participated in this study. The submaximal speed was determined using 50% of the participants’ maximum speed in the height of their season. The Polar Vantage M reported high correlation ( r > 0.84) and high ICC (ICC > 0.86) when comparing its heart rate monitor with the Polar V800 synchronised with H10 chest strap during recording intervals of more than 2 min. The systematic bias and random error were very small (<1 bpm), especially for the 600 s recording interval (0.26 ± 5.10 bpm). Nevertheless, the error increased for 10 s (−5.13 ± 9.20 bpm), 20 s (−8.65 ± 12.60 bpm) and 30 s (−10.71 ± 14.99 bpm) time intervals. In conclusion, the PPG Polar Precision Prime included in the Polar Vantage M demonstrates that it could be a valid alternative to chest bands for monitoring heart rate while running, taking into account some usage considerations, good strap adjustment and an initial calibration time during the first 2–3 min.

Download Full-text

Time to Be SHY? Some Comments on Sleep and Synaptic Homeostasis

Neural Plasticity ◽

10.1155/2012/415250 ◽

2012 ◽

Vol 2012 ◽

pp. 1-12 ◽

Cited By ~ 56

Author(s):

Giulio Tononi ◽

Chiara Cirelli

Keyword(s):

Synaptic Strength ◽

Universal Function ◽

Systematic Bias ◽

Synaptic Homeostasis ◽

The Face ◽

Essential Function ◽

Function Of Sleep ◽

The Brain

Sleep must serve an essential, universal function, one that offsets the risk of being disconnected from the environment. The synaptic homeostasis hypothesis (SHY) is an attempt to identify this essential function. Its core claim is that sleep is needed to reestablish synaptic homeostasis, which is challenged by the remarkable plasticity of the brain. In other words, sleep is “the price we pay for plasticity.” In this issue, M. G. Frank reviewed several aspects of the hypothesis and raised several issues. The comments below provide a brief summary of the motivations underlying SHY and clarify that SHY is a hypothesis not about specific mechanisms, but about a universal, essential function of sleep. This function is the preservation of synaptic homeostasis in the face of a systematic bias toward a net increase in synaptic strength—a challenge that is posed by learning during adult wake, and by massive synaptogenesis during development.

Download Full-text

Population-based input function for TSPO quantification and kinetic modeling with [11C]-DPA-713

EJNMMI Physics ◽

10.1186/s40658-021-00381-8 ◽

2021 ◽

Vol 8 (1) ◽

Author(s):

Mercy I. Akerele ◽

Sara A. Zein ◽

Sneha Pandya ◽

Anastasia Nikolopoulou ◽

Susan A. Gauthier ◽

...

Keyword(s):

Healthy Volunteers ◽

Kinetic Analysis ◽

Kinetic Modeling ◽

Input Function ◽

Brain Regions ◽

Population Based ◽

Patient Specific ◽

Systematic Bias ◽

Altman Analysis ◽

Bland Altman Analysis

Abstract Introduction Quantitative positron emission tomography (PET) studies of neurodegenerative diseases typically require the measurement of arterial input functions (AIF), an invasive and risky procedure. This study aims to assess the reproducibility of [11C]DPA-713 PET kinetic analysis using population-based input function (PBIF). The final goal is to possibly eliminate the need for AIF. Materials and methods Eighteen subjects including six healthy volunteers (HV) and twelve Parkinson disease (PD) subjects from two [11C]-DPA-713 PET studies were included. Each subject underwent 90 min of dynamic PET imaging. Five healthy volunteers underwent a test-retest scan within the same day to assess the repeatability of the kinetic parameters. Kinetic modeling was carried out using the Logan total volume of distribution (VT) model. For each data set, kinetic analysis was performed using a patient-specific AIF (PSAIF, ground-truth standard) and then repeated using the PBIF. PBIF was generated using the leave-one-out method for each subject from the remaining 17 subjects and after normalizing the PSAIFs by 3 techniques: (a) Weightsubject×DoseInjected, (b) area under AIF curve (AUC), and (c) Weightsubject×AUC. The variability in the VT measured with PSAIF, in the test-retest study, was determined for selected brain regions (white matter, cerebellum, thalamus, caudate, putamen, pallidum, brainstem, hippocampus, and amygdala) using the Bland-Altman analysis and for each of the 3 normalization techniques. Similarly, for all subjects, the variabilities due to the use of PBIF were assessed. Results Bland-Altman analysis showed systematic bias between test and retest studies. The corresponding mean bias and 95% limits of agreement (LOA) for the studied brain regions were 30% and ± 70%. Comparing PBIF- and PSAIF-based VT estimate for all subjects and all brain regions, a significant difference between the results generated by the three normalization techniques existed for all brain structures except for the brainstem (P-value = 0.095). The mean % difference and 95% LOA is −10% and ±45% for Weightsubject×DoseInjected; +8% and ±50% for AUC; and +2% and ± 38% for Weightsubject×AUC. In all cases, normalizing by Weightsubject×AUC yielded the smallest % bias and variability (% bias = ±2%; LOA = ±38% for all brain regions). Estimating the reproducibility of PBIF-kinetics to PSAIF based on disease groups (HV/PD) and genotype (MAB/HAB), the average VT values for all regions obtained from PBIF is insignificantly higher than PSAIF (%difference = 4.53%, P-value = 0.73 for HAB; and %difference = 0.73%, P-value = 0.96 for MAB). PBIF also tends to overestimate the difference between PD and HV for HAB (% difference = 32.33% versus 13.28%) and underestimate it in MAB (%difference = 6.84% versus 20.92%). Conclusions PSAIF kinetic results are reproducible with PBIF, with variability in VT within that obtained for the test-retest studies. Therefore, VT assessed using PBIF-based kinetic modeling is clinically feasible and can be an alternative to PSAIF.

Download Full-text

The mass of the Milky Way out to 100 kpc using halo stars

Monthly Notices of the Royal Astronomical Society ◽

10.1093/mnras/staa3984 ◽

2020 ◽

Author(s):

Alis J Deason ◽

Denis Erkal ◽

Vasily Belokurov ◽

Azadeh Fattahi ◽

Facundo A Gómez ◽

...

Keyword(s):

Distribution Function ◽

Mass Concentration ◽

Milky Way ◽

Function Analysis ◽

Large Magellanic Cloud ◽

Systematic Bias ◽

Cosmological Simulations ◽

Halo Stars ◽

Systematic Effects ◽

Mass Estimate

Abstract We use a distribution function analysis to estimate the mass of the Milky Way out to 100 kpc using a large sample of halo stars. These stars are compiled from the literature, and the vast majority ($\sim \! 98\%$) have 6D phase-space information. We pay particular attention to systematic effects, such as the dynamical influence of the Large Magellanic Cloud (LMC), and the effect of unrelaxed substructure. The LMC biases the (pre-LMC infall) halo mass estimates towards higher values, while realistic stellar halos from cosmological simulations tend to underestimate the true halo mass. After applying our method to the Milky Way data we find a mass within 100 kpc of M( < 100kpc) = 6.07 ± 0.29(stat.) ± 1.21(sys.) × 1011M⊙. For this estimate, we have approximately corrected for the reflex motion induced by the LMC using the Erkal et al. model, which assumes a rigid potential for the LMC and MW. Furthermore, stars that likely belong to the Sagittarius stream are removed, and we include a 5% systematic bias, and a 20% systematic uncertainty based on our tests with cosmological simulations. Assuming the mass-concentration relation for Navarro-Frenk-White haloes, our mass estimate favours a total (pre-LMC infall) Milky Way mass of M200c = 1.01 ± 0.24 × 1012M⊙, or (post-LMC infall) mass of M200c = 1.16 ± 0.24 × 1012 M⊙ when a 1.5 × 1011M⊙ mass of a rigid LMC is included.

Download Full-text

Long-Term Observations of Microwave Brightness Temperatures over a Metropolitan Area: Comparison of Radiometric Data and Spectra Simulated with the Use of Radiosonde Measurements

Remote Sensing ◽

10.3390/rs13112061 ◽

2021 ◽

Vol 13 (11) ◽

pp. 2061

Author(s):

Mikhail V. Belikovich ◽

Mikhail Yu. Kulikov ◽

Dmitry S. Makarov ◽

Natalya K. Skalyga ◽

Vitaly G. Ryskin ◽

...

Keyword(s):

Radiative Transfer ◽

Elevation Angle ◽

Radiative Transfer Model ◽

Transfer Model ◽

Systematic Bias ◽

Tropospheric Temperature ◽

Absorption Model ◽

60 Ghz ◽

Error Budget ◽

The Difference

Ground-based microwave radiometers are increasingly used in operational meteorology and nowcasting. These instruments continuously measure the spectra of downwelling atmospheric radiation in the range 20–60 GHz used for the retrieval of tropospheric temperature and water vapor profiles. Spectroscopic uncertainty is an important part of the retrieval error budget, as it leads to systematic bias. In this study, we analyze the difference between observed and simulated microwave spectra obtained from more than four years of microwave and radiosonde observations over Nizhny Novgorod (56.2° N, 44° E). We focus on zenith-measured and elevation-scanning data in clear-sky conditions. The simulated spectra are calculated by a radiative transfer model with the use of radiosonde profiles and different absorption models, corresponding to the latest spectroscopy research. In the case of zenith-measurements, we found a systematic bias (up to ~2 K) of simulated spectra at 51–54 GHz. The sign of bias depends on the absorption model. A thorough investigation of the error budget points to a spectroscopic nature of the observed differences. The dependence of the results on the elevation angle and absorption model can be explained by the basic properties of radiative transfer and by cloud contamination at elevation angles.

Download Full-text

The Systematic Bias of Entropy Calculation in the Multi-Scale Entropy Algorithm

Entropy ◽

10.3390/e23060659 ◽

2021 ◽

Vol 23 (6) ◽

pp. 659

Author(s):

Jue Lu ◽

Ze Wang

Keyword(s):

Time Series ◽

Standard Deviation ◽

Coarse Graining ◽

Sample Entropy ◽

Systematic Bias ◽

Original Time Series ◽

Multi Scale ◽

Calculation Results ◽

Surrogate Measure ◽

Original Time

Entropy indicates irregularity or randomness of a dynamic system. Over the decades, entropy calculated at different scales of the system through subsampling or coarse graining has been used as a surrogate measure of system complexity. One popular multi-scale entropy analysis is the multi-scale sample entropy (MSE), which calculates entropy through the sample entropy (SampEn) formula at each time scale. SampEn is defined by the “logarithmic likelihood” that a small section (within a window of a length m) of the data “matches” with other sections will still “match” the others if the section window length increases by one. “Match” is defined by a threshold of r times standard deviation of the entire time series. A problem of current MSE algorithm is that SampEn calculations at different scales are based on the same matching threshold defined by the original time series but data standard deviation actually changes with the subsampling scales. Using a fixed threshold will automatically introduce systematic bias to the calculation results. The purpose of this paper is to mathematically present this systematic bias and to provide methods for correcting it. Our work will help the large MSE user community avoiding introducing the bias to their multi-scale SampEn calculation results.

Download Full-text