Optimising sampling and analysis protocols in environmental DNA studies

AbstractEcological surveys risk incurring false negative and false positive detections of the target species. With indirect survey methods, such as environmental DNA, such error can occur at two stages: sample collection and laboratory analysis. Here we analyse a large qPCR based eDNA data set using two occupancy models, one of which accounts for false positive error by Griffin et al. (J R Stat Soc Ser C Appl Stat 69: 377–392, 2020), and a second that assumes no false positive error by Stratton et al. (Methods Ecol Evol 11: 1113–1120, 2020). Additionally, we apply the Griffin et al. (2020) model to simulated data to determine optimal levels of replication at both sampling stages. The Stratton et al. (2020) model, which assumes no false positive results, consistently overestimated both overall and individual site occupancy compared to both the Griffin et al. (2020) model and to previous estimates of pond occupancy for the target species. The inclusion of replication at both stages of eDNA analysis (sample collection and in the laboratory) reduces both bias and credible interval width in estimates of both occupancy and detectability. Even the collection of > 1 sample from a site can improve parameter estimates more than having a high number of replicates only within the laboratory analysis.

Download Full-text

Error Rates in Competency Testing When Test Retaking Is Permitted

Journal of Educational Statistics ◽

10.3102/10769986015001039 ◽

1990 ◽

Vol 15 (1) ◽

pp. 39-52 ◽

Cited By ~ 1

Author(s):

Huynh Huynh

Keyword(s):

Conditional Probability ◽

False Positive ◽

False Negative ◽

Error Rates ◽

Rasch Models ◽

Positive Error ◽

False Positive Error ◽

Competency Testing ◽

Negative Error ◽

False Negative Error

False positive and false negative error rates are studied for competency testing where examinees are permitted to retake the test if they fail to pass. Formulae are provided for the beta-binomial and Rasch models, and estimates based on these two models are compared for several typical situations. Although Rasch estimates are expected to be more accurate than beta-binomial estimates, differences among them are found not to be substantial in a number of practical situations. Under relatively general conditions and when test retaking is permitted, the probability of making a false negative error is zero. Under the same situation, and given that an examinee is a true nonmaster, the conditional probability of making a false positive error for this examinee is one.

Download Full-text

Apophenia as the Disposition to False Positives: A Unifying Framework for the Openness-Psychoticism Dimension

10.31234/osf.io/d9wkc ◽

2019 ◽

Author(s):

Scott D. Blain ◽

Julia Longenecker ◽

Rachael Grazioplene ◽

Bonnie Klimes-Dougan ◽

Colin G. DeYoung

Keyword(s):

False Positive ◽

Psychotic Disorders ◽

Structural Equation Models ◽

False Negative ◽

Error Rates ◽

Self Report ◽

Auditory Signal ◽

Positive Symptoms ◽

Positive Error ◽

False Positive Error

Positive symptoms of schizophrenia and its extended phenotype—often termed psychoticism or positive schizotypy—are characterized by the inclusion of novel, erroneous mental contents. One promising framework for explaining positive symptoms involves “apophenia,” conceptualized here as a disposition toward false positive errors. Apophenia and positive symptoms have shown relations to Openness to Experience, and all of these constructs involve tendencies toward pattern seeking. Nonetheless, few studies have investigated the relations between psychoticism and non-self-report indicators of apophenia, let alone the role of normal personality variation. The current research used structural equation models to test associations between psychoticism, openness, intelligence, and non-self-report indicators of apophenia comprising false positive error rates on a variety of computerized tasks. In Sample 1, 1193 participants completed digit identification, theory of mind, and emotion recognition tasks. In Sample 2, 195 participants completed auditory signal detection and semantic word association tasks. Openness and psychoticism were positively correlated. Self-reported psychoticism, openness, and their shared variance were positively associated with apophenia, as indexed by false positive error rates, whether or not intelligence was controlled for. Apophenia was not associated with other personality traits, and openness and psychoticism were not associated with false negative errors. Standardized regression paths from openness-psychoticism to apophenia were in the range of .61 to .75. Findings provide insights into the measurement of apophenia and its relation to personality and psychopathology. Apophenia and pattern seeking may be promising constructs for unifying openness with the psychosis spectrum and for providing an explanation of positive symptoms. Results are discussed in the context of possible adaptive characteristics of apophenia, as well as potential risk factors for the development of psychotic disorders.

Download Full-text

Feasibility of FT-NIR spectroscopy and Vis/NIR hyperspectral imaging for sorting unsound chestnuts

Italus Hortus ◽

10.26353/j.itahort/2020.1.0318 ◽

2020 ◽

Vol 27 ◽

pp. 3-18

Author(s):

Giacomo Bedini ◽

Giorgia Bastianelli ◽

Swathi Sirisha Nallan Chakravartula ◽

Carmen Morales-Rodríguez ◽

Luca Rossini ◽

...

Keyword(s):

Hyperspectral Imaging ◽

Error Rate ◽

False Positive ◽

Total Error ◽

False Negative ◽

Nir Spectroscopy ◽

Positive Error ◽

False Positive Error ◽

Negative Error ◽

False Negative Error

Authors explored the potential use of Vis/NIR hyperspectral imaging (HSI) and Fourier-transform Near-Infrared (FT-NIR) spectroscopy to be used as in-line tools for the detection of unsound chestnut fruits (i.e. infected and/or infested) in comparison with the traditional sorting technique. For the intended purpose, a total of 720 raw fruits were collected from a local company. Chestnut fruits were preliminarily classified into sound (360 fruits) and unsound (360 fruits) batches using a proprietary floating system at the facility along with manual selection performed by expert workers. The two batches were stored at 4 ± 1 °C until use. Samples were left at ambient temperature for at least 12 h before measurements. Subsequently, fruits were subjected to non-destructive measurements (i.e. spectral analysis) immediately followed by destructive analyses (i.e. microbiological and entomological assays). Classification models were trained using the Partial Least Squares Discriminant Analysis (PLS-DA) by pairing the spectrum of each fruit with the categorical information obtained from its destructive assay (i.e., sound, Y = 0; unsound, Y = 1). Categorical data were also used to evaluate the classification performance of the traditional sorting method. The performance of each PLS-DA model was evaluated in terms of false positive error (FP), false negative error (FN) and total error (TE) rates. The best result (8% FP, 14% FN, 11% TE) was obtained using Savitzky-Golay first derivative with a 5-points window of smoothing on the dataset of raw reflectance spectra scanned from the hilum side of fruit using the Vis/NIR HSI setup. This model showed similarity in terms of False Negative error rate with the best one computed using data from the FT-NIR setup (i.e. 15% FN), which, however, had the lowest global performance (17% TE) due to the highest False Positive error rate (19%). Finally, considering that the total error rate committed by the traditional sorting system was about 14.5% with a tendency of misclassifying unsound fruits, the results indicate the feasibility of a rapid, in-line detection system based on spectroscopic measurements.

Download Full-text

Response Error and the Medicaid Undercount in the Current Population Survey

10.31235/osf.io/ch8qt ◽

2017 ◽

Author(s):

James Noon ◽

Leticia Fernandez ◽

Sonya Rastogi

Keyword(s):

False Positive ◽

Current Population Survey ◽

Population Survey ◽

False Negative ◽

Positive Error ◽

False Positive Error ◽

Response Error ◽

Medicaid Enrollment ◽

False Negative Error ◽

Uninsured Population

The Current Population Survey Annual Social and Economic Supplement (CPS ASEC) is an important source for estimates of the uninsured population. Previous research has shown that survey estimates produce an undercount of beneficiaries compared to Medicaid enrollment records. We extend past work by examining the Medicaid undercount in the 2007-2011 CPS ASEC compared to enrollment data from the Medicaid Statistical Information System for calendar years 2006-2010. By linking individuals across datasets, we analyze two types of response error regarding Medicaid enrollment – false negative error and false positive error. We use regression analysis to identify factors associated with these two types of response error in the 2011 CPS ASEC. We find that the Medicaid undercount was between 22 and 31 percent from 2007 to 2011. In 2011, the false negative rate was 40 percent, and 27 percent of Medicaid reports in CPS ASEC were false positives. False negative error is associated with the duration of enrollment in Medicaid, enrollment in Medicare and private insurance, and Medicaid enrollment in the survey year. False positive error is associated with enrollment in Medicare and shared Medicaid coverage in the household. We discuss implications for survey reports of health insurance coverage and for estimating the uninsured population.

Download Full-text

An Rshiny app for modelling environmental DNA data: accounting for false positive and false negative observation error

10.1101/2020.12.09.417600 ◽

2020 ◽

Author(s):

Alex Diana ◽

Eleni Matechou ◽

Jim E. Griffin ◽

Andrew S. Buxton ◽

Richard A. Griffiths

Keyword(s):

Statistical Method ◽

False Positive ◽

False Negative ◽

Environmental Dna ◽

Bayesian Variable Selection ◽

Observation Error ◽

List Type ◽

Data Set ◽

A Site ◽

User Friendly

AbstractEnvironmental DNA (eDNA) surveys have become a popular tool for assessing the distribution of species. However, it is known that false positive and false negative observation error can occur at both stages of eDNA surveys, namely the field sampling stage and laboratory analysis stage.We present an RShiny app that implements the Griffin et al. (2019) statistical method, which accounts for false positive and false negative errors in both stages of eDNA surveys. Following Griffin et al. (2019), we employ a Bayesian approach and perform efficient Bayesian variable selection to identify important predictors for the probability of species presence as well as the probabilities of observation error at either stage.We demonstrate the RShiny app using a data set on great crested newts collected by Natural England in 2018 and we identify water quality, pond area, fish presence, macrophyte cover, frequency of drying as important predictors for species presence at a site.The state-of-the-art statistical method that we have implemented is the only one that has specifically been developed for the purposes of modelling false negatives and false positives in eDNA data. Our RShiny app is user-friendly, requires no prior knowledge of R and fits the models very efficiently. Therefore, it should be part of the tool-kit of any researcher or practitioner who is collecting or analysing eDNA data.

Download Full-text

Sample size determination for comparing two therapies taking both the α (type I or false positive) error and the β (type II or false negative) error into consideration

Applied Mathematics and Computation ◽

10.1016/0096-3003(77)90028-5 ◽

1977 ◽

Vol 3 (2) ◽

pp. 169-174

Author(s):

Vrudhula K. Murthy ◽

L.Julian Haywood

Keyword(s):

Sample Size ◽

False Positive ◽

False Negative ◽

Sample Size Determination ◽

Type I ◽

Type Ii ◽

Size Determination ◽

Positive Error ◽

False Positive Error ◽

False Negative Error

Download Full-text

Detection Games under Fully Active Adversaries

Entropy ◽

10.3390/e21010023 ◽

2018 ◽

Vol 21 (1) ◽

pp. 23 ◽

Cited By ~ 2

Author(s):

Benedetta Tondi ◽

Neri Merhav ◽

Mauro Barni

Keyword(s):

False Positive ◽

False Negative ◽

Test Sequence ◽

Positive Error ◽

False Positive Error ◽

Bayesian Game ◽

Equilibrium Payoff ◽

Error Exponents ◽

Exponential Decay Rate

We study a binary hypothesis testing problem in which a defender must decide whether a test sequence has been drawn from a given memoryless source P 0 , while an attacker strives to impede the correct detection. With respect to previous works, the adversarial setup addressed in this paper considers an attacker who is active under both hypotheses, namely, a fully active attacker, as opposed to a partially active attacker who is active under one hypothesis only. In the fully active setup, the attacker distorts sequences drawn both from P 0 and from an alternative memoryless source P 1 , up to a certain distortion level, which is possibly different under the two hypotheses, to maximize the confusion in distinguishing between the two sources, i.e., to induce both false positive and false negative errors at the detector, also referred to as the defender. We model the defender–attacker interaction as a game and study two versions of this game, the Neyman–Pearson game and the Bayesian game. Our main result is in the characterization of an attack strategy that is asymptotically both dominant (i.e., optimal no matter what the defender’s strategy is) and universal, i.e., independent of P 0 and P 1 . From the analysis of the equilibrium payoff, we also derive the best achievable performance of the defender, by relaxing the requirement on the exponential decay rate of the false positive error probability in the Neyman–Pearson setup and the tradeoff between the error exponents in the Bayesian setup. Such analysis permits characterizing the conditions for the distinguishability of the two sources given the distortion levels.

Download Full-text

Forensic science evidence: Naive estimates of false positive error rates and reliability

10.31234/osf.io/anxur ◽

2020 ◽

Author(s):

Kristy Martire ◽

Agnes Bali ◽

Kaye Ballantyne ◽

Gary Edmond ◽

Richard Kemp ◽

...

Keyword(s):

Forensic Science ◽

False Positive ◽

Error Rates ◽

Forensic Sciences ◽

Positive Error ◽

False Positive Error ◽

Reliability Estimates ◽

Positive Report ◽

Science Evidence ◽

Science Disciplines

We do not know how often false positive reports are made in a range of forensic science disciplines. In the absence of this information it is important to understand the naive beliefs held by potential jurors about forensic science evidence reliability. It is these beliefs that will shape evaluations at trial. This descriptive study adds to our knowledge about naive beliefs by: 1) measuring jury-eligible (lay) perceptions of reliability for the largest range of forensic science disciplines to date, over three waves of data collection between 2011 and 2016 (n = 674); 2) calibrating reliability ratings with false positive report estimates; and 3) comparing lay reliability estimates with those of an opportunity sample of forensic practitioners (n = 53). Overall the data suggest that both jury-eligible participants and practitioners consider forensic evidence highly reliable. When compared to best or plausible estimates of reliability and error in the forensic sciences these views appear to overestimate reliability and underestimate the frequency of false positive errors. This result highlights the importance of collecting and disseminating empirically derived estimates of false positive error rates to ensure that practitioners and potential jurors have a realistic impression of the value of forensic science evidence.

Download Full-text