The Pseudoscientific Structure of “Perceptual Decision-Making” and other “Signal Detection Theory"-based Research (or How Science Works When It Works)

2021 ◽  
Author(s):  
Lydia Maria Maniatis
Keyword(s):  
Decision Making ◽  
Signal Detection ◽  
Signal Detection Theory ◽  
Ad Hoc ◽  
Natural World ◽  
Detection Theory ◽  
Perceptual Decision Making ◽  
Research Activity ◽  
Perceptual Decision ◽  
Visual Neuroscience

The assumptions and formulas of “Signal Detection Theory” (SDT) dominate psychophysics and neuroscience, and serve as the basis of visual neuroscience under the rubric of “perceptual decision-making.” Here, I discuss how the overly simple, ad hoc assumptions of SDT served to rationalize the chronic failure of traditional psychophysics to achieve reliable results; how the constraints on outcomes imposed by the traditional methods combined with SDT to artificially immunize core assumptions from empirical challenge; and how consequently, research activity has been reduced to a seemingly uncomplicated - yet still non-replicable - matter of mere measurement and correlation. I contrast the structure of this ever-barren approach to the structure of research that respects reality and expands our knowledge of the natural world.

scholarly journals When uncertain, does human self-motion decision-making fully utilize complete information?

2018 ◽  
Vol 119 (4) ◽  
pp. 1485-1496 ◽  
Author(s):  
Torin K. Clark ◽  
Yongwoo Yi ◽  
Raquel C. Galvan-Garza ◽  
María Carolina Bermúdez Rey ◽  
Daniel M. Merfeld
Keyword(s):  
Decision Making ◽  
Signal Detection ◽  
Signal Detection Theory ◽  
Complete Information ◽  
Detection Theory ◽  
Forced Choice ◽  
Perceptual Decision Making ◽  
Perceptual Decision ◽  
Available Information ◽  
Self Motion

When forced to choose humans often feel uncertain. Investigations of human perceptual decision-making often employ signal detection theory, which assumes that even when uncertain all available information is fully utilized. However, other studies have suggested or assumed that, when uncertain, human subjects guess totally at random, ignoring available information. When uncertain, do humans simply guess totally at random? Or do humans fully utilize complete information? Or does behavior fall between these two extremes yielding “above chance” performance without fully utilizing complete information? While it is often assumed complete information is fully utilized, even when uncertain, to our knowledge this has never been experimentally confirmed. To answer this question, we combined numerical simulations, theoretical analyses, and human studies performed using a self-motion direction-recognition perceptual decision-making task (did I rotate left or right?). Subjects were instructed to make forced-choice binary (left/right) and trinary (left/right/uncertain) decisions when cued following each stimulus. Our results show that humans 1) do not guess at random when uncertain and 2) make binary and trinary decisions equally well. These findings show that humans fully utilize complete information when uncertain for our perceptual decision-making task. This helps unify signal detection theory and other models of forced-choice decision-making which allow for uncertain responses. NEW & NOTEWORTHY Humans make many perceptual decisions every day. But what if we are uncertain? While many studies assume that humans fully utilize complete information, other studies have suggested and/or assumed that when we're uncertain and forced to decide, information is not fully utilized. While humans tend to perform above chance when uncertain, no earlier study has tested whether available information is fully utilized. Our results show that humans make fully informed decisions even when uncertain.

scholarly journals Quorums enable optimal pooling of independent judgements

2018 ◽  
Author(s):  
James Marshall ◽  
Ralf H.J.M. Kurvers ◽  
Jens Krause ◽  
Max Wolf
Keyword(s):  
Decision Making ◽  
Signal Detection ◽  
Signal Detection Theory ◽  
Detection Theory ◽  
Collective Decision ◽  
Majority Voting ◽  
Single Shot ◽  
Collective Decision Making ◽  
Trade Offs ◽  
Speed Accuracy

Majority-voting and the Condorcet Jury Theorem pervade thinking about collective decision-making. Thus, it is typically assumed that majority-voting is the best possible decision mechanism, and that scenarios exist where individually-weak decision-makers should not pool information. Condorcet and its applications implicitly assume that only one kind of error can be made, yet signal detection theory shows two kinds of errors exist, ‘false positives’ and ‘false negatives’. We apply signal detection theory to collective decision-making to show that majority voting is frequently sub-optimal, and can be optimally replaced by quorum decision-making. While quorums have been proposed to resolve within-group conflicts, or manage speed-accuracy trade-offs, our analysis applies to groups with aligned interests undertaking single-shot decisions. Our results help explain the ubiquity of quorum decision-making in nature, relate the use of sub- and super-majority quorums to decision ecology, and may inform the design of artificial decision-making systems.Impact StatementTheory typically assumes that majority voting is optimal; this is incorrect – majority voting is typically sub-optimal, and should be replaced by sub-majority or super-majority quorum voting. This helps explain the prevalence of quorum-sensing in even the simplest collective systems, such as bacterial communities.

Robustness of the a b Signal Detection Theory Model of Decision Making

2007 ◽  
Vol 51 (18) ◽  
pp. 1263-1267
Author(s):  
Ernesto A. Bustamante ◽  
Brittany L. Anderson ◽  
Amy R. Thompson ◽  
James P. Bliss ◽  
Mark W. Scerbo
Keyword(s):  
Decision Making ◽  
Monte Carlo ◽  
Monte Carlo Simulations ◽  
Signal Detection ◽  
Signal Detection Theory ◽  
Classical Model ◽  
Empirical Evaluation ◽  
Detection Theory ◽  
Theory Model ◽  
Extreme Responses

Bustamante, Fallon, and Bliss (2006) showed that the a b Signal Detection Theory (SDT) model was more parsimonious, generalizable, and applicable than the classical SDT model. Additionally, they demonstrated that both models provided statistically equivalent and uncorrelated measures of sensitivity and bias under ideal conditions. The purpose of this research was to show the robustness of the a b model for handling extreme responses. We conducted an empirical evaluation of operators' decision-making and two Monte Carlo simulations. Results from the empirical study showed that the a b model provided equivalent yet independent measures of decision-making accuracy and bias, whereas the classical model failed to provide independent measures in the presence of extreme responses. The Monte Carlo simulations showed a similar trend for the superiority of the a b model. Results from this research provide evidence to support the use of the a b model instead of the classical model.

The Application of Signal Detection Theory to Decision-Making in Forensic Science

2001 ◽  
Vol 46 (2) ◽  
pp. 14962J ◽  
Author(s):  
Victoria L. Phillips ◽  
Michael J. Saks ◽  
Joseph L. Peterson
Keyword(s):  
Decision Making ◽  
Signal Detection ◽  
Forensic Science ◽  
Signal Detection Theory ◽  
Detection Theory

Differential Threshold of Length and Response Criterion for Inspecting Irregular Objects

1997 ◽  
Vol 85 (2) ◽  
pp. 723-735
Author(s):  
Chia-Fen Chi ◽  
Chin-Lung Chen
Keyword(s):  
Decision Making ◽  
Signal Detection ◽  
Signal Detection Theory ◽  
Detection Theory ◽  
Decision Time ◽  
Visual Sensitivity ◽  
Differential Threshold ◽  
Human Beings ◽  
Experimental Conditions ◽  
Defective Items

This research investigated human visual sensitivity and bias in inspecting irregular objects. A preliminary study was conducted using the method of constants to determine the threshold value for judgment of size. A factorial experiment was conducted using payoffs, rate of defective items, and detectability in the signal-detection theory as the factors. In total, eight experimental conditions were tested. 10 college students were recruited as subjects. Each subject was asked to compare 40 teapot shapes to a standard teapot shape under eight experimental conditions. Defective shapes were generated by lengthening the vertical dimension of a standard teapot shape by a factor of 1.01 and 1.04 for ‘low’ and ‘high’ detectability. The decision time and responses of ‘identical’ or ‘different’ were collected under all experimental conditions. Analysis indicates that the decision-making strategy used to inspect this irregular object was very close to maximizing the accuracy of decision-making by considering the rate of defective items. This result is different from most research findings in signal-detection theory in which responses of human beings are similar to degraded Bayes optimizers. The standard deviation of the signal distribution was about 1.30 and 1.41 times that of the noise distributions for ‘low’ and ‘high’ detectability.

scholarly journals Perceptual Anticipation in a Shoot/Don’t Shoot Task

2019 ◽  
Vol 63 (1) ◽  
pp. 1358-1362
Author(s):  
Dakota Scott ◽  
Joel Suss
Keyword(s):  
College Students ◽  
Decision Making ◽  
Law Enforcement ◽  
Signal Detection ◽  
Cell Phone ◽  
Signal Detection Theory ◽  
Detection Theory ◽  
Perceptual Judgments ◽  
A Cell

Signal Detection Theory (SDT) has been applied to examine expertise-related differences in perceptual judgments of deceptive and non-deceptive movements in sport (e.g., handball, soccer). Deceptive actions in sport-related tasks (i.e., faking in rugby, fake passes in basketball) affects anticipation performance in both novice and expert athletes (i.e., more incorrect responses in deceptive actions compared to incorrect responses in non-deceptive actions); however, experts still outperform novices when facing deceptive actions in sport-related tasks (Güldenpenning, Kunde, & Weigelt, 2017). To date, this approach has not yet been applied to shoot/don’t shoot scenarios in law enforcement. To address this issue, we filmed actors pulling out either a weapon (i.e., gun) or a non-weapon (i.e., cell phone). We then edited the videos to create temporally-occluded stimuli. College students observed the videos and indicated whether the object was a weapon or a non-weapon. We conducted two experiments: across both we found that participants’ responses were more likely to be correct at later occlusion points, when the object was fully observable. We also found that when the object was fully observable, participants were more likely to identify the object as a gun rather than a cell phone. The results can inform the design of decision-making training for police.

scholarly journals The wisdom of crowds for visual search

2017 ◽  
Vol 114 (21) ◽  
pp. E4306-E4315 ◽  
Author(s):  
Mordechai Z. Juni ◽  
Miguel P. Eckstein
Keyword(s):  
Decision Making ◽  
Visual Search ◽  
Signal Detection ◽  
Signal Detection Theory ◽  
Detection Theory ◽  
Majority Voting ◽  
Weighted Averaging ◽  
Simple Majority ◽  
Wisdom Of Crowds ◽  
The Wisdom Of Crowds

Decision-making accuracy typically increases through collective integration of people’s judgments into group decisions, a phenomenon known as the wisdom of crowds. For simple perceptual laboratory tasks, classic signal detection theory specifies the upper limit for collective integration benefits obtained by weighted averaging of people’s confidences, and simple majority voting can often approximate that limit. Life-critical perceptual decisions often involve searching large image data (e.g., medical, security, and aerial imagery), but the expected benefits and merits of using different pooling algorithms are unknown for such tasks. Here, we show that expected pooling benefits are significantly greater for visual search than for single-location perceptual tasks and the prediction given by classic signal detection theory. In addition, we show that simple majority voting obtains inferior accuracy benefits for visual search relative to averaging and weighted averaging of observers’ confidences. Analysis of gaze behavior across observers suggests that the greater collective integration benefits for visual search arise from an interaction between the foveated properties of the human visual system (high foveal acuity and low peripheral acuity) and observers’ nonexhaustive search patterns, and can be predicted by an extended signal detection theory framework with trial to trial sampling from a varying mixture of high and low target detectabilities across observers (SDT-MIX). These findings advance our theoretical understanding of how to predict and enhance the wisdom of crowds for real world search tasks and could apply more generally to any decision-making task for which the minority of group members with high expertise varies from decision to decision.

Sign in / Sign up

Export Citation Format

Share Document