scholarly journals Failure to Apply Signal Detection Theory to the Montreal Battery of Evaluation of Amusia May Misdiagnose Amusia

2012 ◽  
Vol 30 (5) ◽  
pp. 480-496 ◽  
Author(s):  
Molly J. Henry ◽  
J. Devin McAuley
Keyword(s):  
Signal Detection ◽  
Response Bias ◽  
Signal Detection Theory ◽  
Roc Curves ◽  
Detection Theory ◽  
Performance Measure ◽  
Evaluation Of Performance ◽  
N 93 ◽  
Response Biases ◽  
Large Response

This article considers a signal detection theory (SDT) approach to evaluation of performance on the Montreal Battery of Evaluation of Amusia (MBEA). One hundred fifty-five individuals completed the original binary response version of the MBEA (n = 62) or a confidence rating version (MBEA-C; n = 93). Confidence ratings afforded construction of empirical receiver operator characteristic (ROC) curves and derivation of bias-free performance measures against which we compared the standard performance metric, proportion correct (PC), and an alternative signal detection metric, d ′. Across the board, PC was tainted by response bias and underestimated performance as indexed by Az, a nonparametric ROC-based performance measure. Signal detection analyses further revealed that some individuals performing worse than the standard PC-based cutoff for amusia diagnosis showed large response biases. Given that PC is contaminated by response bias, this suggests the possibility that categorizing individuals as having amusia or not, using a PC-based cutoff, may inadvertently misclassify some individuals with normal perceptual sensitivity as amusic simply because they have large response biases. In line with this possibility, a comparison of amusia classification using d ′- and PC-based cutoffs showed potential misclassification of 33% of the examined cases.

scholarly journals Predictive cues reduce but do not eliminate intrinsic response bias

2018 ◽  
Author(s):  
Mingjia Hu ◽  
Dobromir Rahnev
Keyword(s):  
Signal Detection ◽  
Response Bias ◽  
Signal Detection Theory ◽  
Detection Theory ◽  
Stimulus Category ◽  
Biasing Effects ◽  
Original Size ◽  
Response Biases ◽  
Criterion Location

Predictive cues induce large changes in people’s choices by biasing responses towards the expected stimulus category. At the same time, even in the absence of predictive cues, humans often exhibit substantial intrinsic response biases. Despite the ubiquity of both of these biasing effects, it remains unclear how predictive cues interact with intrinsic bias. To understand the nature of this interaction, we examined data across three previous experiments that featured a combination of neutral cues (revealing intrinsic biases) and predictive cues. We found that predictive cues decreased the intrinsic bias to about half of its original size. This result held both when bias was quantified as the criterion location estimated using signal detection theory and as the probability of choosing a particular stimulus category. Our findings demonstrate that predictive cues reduce but do not eliminate intrinsic response bias, testifying to both the malleability and rigidity of intrinsic biases.

scholarly journals Using Signal Detection Theory to Better Understand Cognitive Fatigue

2021 ◽  
Vol 11 ◽  
Author(s):  
Glenn R. Wylie ◽  
Bing Yao ◽  
Joshua Sandry ◽  
John DeLuca
Keyword(s):  
Signal Detection ◽  
Response Bias ◽  
Signal Detection Theory ◽  
Subjective Experience ◽  
Memory Task ◽  
Objective Measure ◽  
Detection Theory ◽  
Behavioral Measure ◽  
Cognitive Fatigue ◽  
Fatigue Criterion

When we are fatigued, we feel that our performance is worse than when we are fresh. Yet, for over 100 years, researchers have been unable to identify an objective, behavioral measure that covaries with the subjective experience of fatigue. Previous work suggests that the metrics of signal detection theory (SDT)—response bias (criterion) and perceptual certainty (d’)—may change as a function of fatigue, but no work has yet been done to examine whether these metrics covary with fatigue. Here, we investigated cognitive fatigue using SDT. We induced fatigue through repetitive performance of the n-back working memory task, while functional magnetic resonance imaging (fMRI) data was acquired. We also assessed cognitive fatigue at intervals throughout. This enabled us to assess not only whether criterion and d’ covary with cognitive fatigue but also whether similar patterns of brain activation underlie cognitive fatigue and SDT measures. Our results show that both criterion and d’ were correlated with changes in cognitive fatigue: as fatigue increased, subjects became more conservative in their response bias and their perceptual certainty declined. Furthermore, activation in the striatum of the basal ganglia was also related to cognitive fatigue, criterion, and d’. These results suggest that SDT measures represent an objective measure of cognitive fatigue. Additionally, the overlap and difference in the fMRI results between cognitive fatigue and SDT measures indicate that these measures are related while also separate. In sum, we show the relevance of SDT measures in the understanding of fatigue, thus providing researchers with a new set of tools with which to better understand the nature and consequences of cognitive fatigue.

Vigilance and Signal Detection Theory: An Evaluation of Response Bias Measures

1994 ◽  
Author(s):  
Judi E. See ◽  
Joel S. Warm ◽  
William N. Dember ◽  
Steven R. Howe
Keyword(s):  
Signal Detection ◽  
Response Bias ◽  
Signal Detection Theory ◽  
Detection Theory

A Computer Program Demonstrating the Effect of the Payoff Matrix on the Signal Detection Measure, Beta

1994 ◽  
Vol 21 (2) ◽  
pp. 113-115 ◽  
Author(s):  
Janet D. Larsen
Keyword(s):  
Computer Program ◽  
Signal Detection ◽  
False Alarm ◽  
Response Bias ◽  
Signal Detection Theory ◽  
Visual Display ◽  
Payoff Matrix ◽  
Detection Theory ◽  
Basic Concepts ◽  
Hit Rates

The computer program described in this article demonstrates the basic concepts of signal detection theory and illustrates the effect of changes in the payoff matrix on beta, a measure of response bias. Observers attempt to identify the trials on which there was a signal in a noisy visual display. They earn or lose points for correct or incorrect answers. After three sets of 100 trials, with a different payoff matrix for each set, the program provides observers with their hit rates, false alarm rates, d' scores, and beta scores for the three sets of trials, as well as the total points they have earned for the 300 trials. The program runs on MS-DOS machines and does not require a computer with graphics capabilities.

Analysis of Diagnostic Tasks in Accounting Research Using Signal Detection Theory

2005 ◽  
Vol 17 (1) ◽  
pp. 149-173 ◽  
Author(s):  
Robert J. Ramsay ◽  
Richard M. Tubbs
Keyword(s):  
Theoretical Model ◽  
Signal Detection ◽  
Response Bias ◽  
Signal Detection Theory ◽  
Detection Theory ◽  
Accounting Research ◽  
Accounting Judgments ◽  
Percentage Correct

Many accounting judgments are diagnostic tasks in which accountants, auditors, managers, or investors discriminate among possible states and decide which one exists. To measure the accuracy of such decisions, most accounting research employs percentage correct, a measure proven to be invalid and unreliable, primarily because it does not control for response bias. This paper describes Signal Detection Theory (SDT), a theoretical model of diagnostic tasks that has been supported empirically in many fields. SDT provides superior measures of accuracy and response bias. We discuss the benefits of employing SDT in accounting research and describe an SDT-based reanalysis of data related to two published accounting studies that results in revised conclusions and important additional insights.

Rediscovering the Past: Gustav Fechner and Signal Detection Theory

1994 ◽  
Vol 5 (6) ◽  
pp. 335-340 ◽  
Author(s):  
Stephen W. Link
Keyword(s):  
Signal Detection ◽  
Response Bias ◽  
Experimental Psychology ◽  
Signal Detection Theory ◽  
Detection Theory ◽  
Empirical Results ◽  
The Past

The origins of experimental psychology are found in the theory of discrimination created by Gustav Fechner The theory advanced by Fechner was the first theory of mental judgment that applied the ideas of Gaussian error to human discrimination Despite its powerful theoretical and empirical results, the theory vanished Later embellishments, such as the addition of response bias found in signal detection theory, gave such theories new life, but Fechner never received the credit due to the founder of psychological detection theory

Vigilance and Signal Detection Theory: An Empirical Evaluation of Five Measures of Response Bias

1997 ◽  
Vol 39 (1) ◽  
pp. 14-29 ◽  
Author(s):  
Judi E. See ◽  
Joel S. Warm ◽  
William N. Dember ◽  
Steven R. Howe
Keyword(s):  
Signal Detection ◽  
Response Bias ◽  
Signal Detection Theory ◽  
Empirical Evaluation ◽  
Vigilance Task ◽  
Parametric Model ◽  
Detection Theory ◽  
Nonparametric Model ◽  
Effective Measure ◽  
Bias Index

Three experiments were conducted to determine which of five response bias indices (β, c, B", B'H and B"D) defined by the theory of signal detection provides the most effective measure of the observer's willingness to respond in a vigilance task. The results indicated that the traditional parametric bias index β was an inadequate measure of response bias in every respect, whereas the newer parametric measure c was the most effective of all five indices. When the three nonparametric measures (B", B'H' and B"D) were examined separately, B"D emerged as the most effective nonparametric index. We recommend that vigilance researchers use c rather than β to measure bias when a parametric model is involved and B"D instead of B" and B' H when a nonparametric model is used.

Sign in / Sign up

Export Citation Format

Share Document