Higher-order learning

We design a novel experiment to study how subjects update their beliefs about the beliefs of others. Three players receive sequential signals about an unknown state of the world. Player 1 reports her beliefs about the state; Player 2 simultaneously reports her beliefs about the beliefs of Player 1; Player 3 simultaneously reports her beliefs about the beliefs of Player 2. We say that beliefs exhibit higher-order learning if the beliefs of Player k about the beliefs of Player $$k-1$$ k - 1 become more accurate as more signals are observed. We find that some of the predicted dynamics of higher-order beliefs are reflected in the data; in particular, higher-order beliefs are updated more slowly with private than public information. However, higher-order learning fails even after a large number of signals is observed. We argue that this result is driven by base-rate neglect, heterogeneity in updating processes, and subjects’ failure to correctly take learning rules of others into account.

Is Automation of Statistical Reasoning a Suitable Mindware in a Base-Rate Neglect Task?

Until recently, studies within the dual-process approach were mainly focused on group differences in processing, and individual differences were neglected. However, individual differences have proven to be a significant factor in conflict detection efficiency and the overall success in base-rate neglect and similar tasks. This should be taken into consideration within the framework of the Hybrid Model of Dual Processing. New tendencies in the development of this model have focused attention on the degree of mindware instantiation as a predictor of base-rate neglect task efficiency. This study aimed to examine the relationship between mindware and base-rate neglect task efficiency and to test and explore the relationship between base-rate response frequency and conflict detection efficiency and the degree of mindware instantiation. All participants solved base-rate neglect tasks, made judgments of confidence in their responses, and solved the Statistical Reasoning Test, Cognitive Reflection Test and Numeracy Scale. We used the Statistical Reasoning Test as a measure of mindware instantiation. The degree of mindware instantiation was found to be the only significant predictor of base-rate neglect task efficiency and the results showed that participants with a higher degree of mindware instantiation generally made more base-rate responses. No correlation was found between the degree of mindware instantiation and conflict detection efficiency. These findings support the hypothesis that the power of logical intuition depends on the individual’s degree of mindware instantiation. Therefore, the results of this research indicate the importance of further research into the role of statistical reasoning in base-rate neglect task efficiency. However, we discuss that there are some methodological limitations in this research which might explain why the degree of mindware instantiation had no relationship with conflict efficiency.

Top Ten Behavioral Biases in Project Management: An Overview

Behavioral science has witnessed an explosion in the number of biases identified by behavioral scientists, to more than 200 at present. This article identifies the 10 most important behavioral biases for project management. First, we argue it is a mistake to equate behavioral bias with cognitive bias, as is common. Cognitive bias is half the story; political bias the other half. Second, we list the top 10 behavioral biases in project management: (1) strategic misrepresentation, (2) optimism bias, (3) uniqueness bias, (4) the planning fallacy, (5) overconfidence bias, (6) hindsight bias, (7) availability bias, (8) the base rate fallacy, (9) anchoring, and (10) escalation of commitment. Each bias is defined, and its impacts on project management are explained, with examples. Third, base rate neglect is identified as a primary reason that projects underperform. This is supported by presentation of the most comprehensive set of base rates that exist in project management scholarship, from 2,062 projects. Finally, recent findings of power law outcomes in project performance are identified as a possible first stage in discovering a general theory of project management, with more fundamental and more scientific explanations of project outcomes than found in conventional theory.

Experts use base rates in real-world sequential decisions

Human behavior is often assumed to be irrational, full of errors, and affected by cognitive biases. One of these biases is base-rate neglect, which happens when the base rates of a specific category are not considered when making decisions. We argue here that while naïve subjects demonstrate base-rate neglect in laboratory conditions, experts tested in the real world do use base rates. Our explanation is that lab studies use single questions, whereas, in the real world, most decisions are sequential in nature, leading to a more realistic test of base-rate use. One decision that lends itself to testing base-rate use in real life occurs in beach volleyball—specifically, deciding to whom to serve to win the game. Analyzing the sequential choices in expert athletes in more than 1,300 games revealed that they were sensitive to base rates and adapted their decision strategies to the performance of the opponent. Our data describes a threshold at which players change their strategy and use base rates. We conclude that the debate over whether decision makers use base rates should be shifted to real-world tests, and the focus should be on when and how base rates are used.

De-Biasing Legal Factfinders

This chapter examines the psychological studies of biases and de-biasing measures in human decision-making with special reference to adjudicative factfinding. Research shows that factfinders are prone to cognitive biases (such as anchoring, framing, base-rate neglect, and confirmation bias) as well as social biases. Driven by this research, multiple studies have examined the extent to which those biases can be mitigated by de-biasing measures like “consider the opposite” and “give reasons.” After a brief overview of the research, the author points to the problematic evidential basis and identifies future research needs, and concludes that empirical research on de-biasing measures has so far delivered less than one would hope for.

Base rate neglect and conservatism in probabilistic reasoning: Insights from eliciting full distributions

Bayesian statistics offers a normative description for how a person should combine their original beliefs (i.e., their priors) in light of new evidence (i.e., the likelihood). Previous research suggests that people tend to under-weight both their prior (base rate neglect) and the likelihood (conservatism), although this varies by individual and situation. Yet this work generally elicits people's knowledge as single point estimates (e.g., x has 5% probability of occurring) rather than as a full distribution. Here we demonstrate the utility of eliciting and fitting full distributions when studying these questions. Across three experiments, we found substantial variation in the extent to which people showed base rate neglect and conservatism, which our method allowed us to measure for the first time simultaneously at the level of the individual. We found that while most people tended to disregard the base rate, they did so less when the prior was made explicit. Although many individuals were conservative, there was no apparent systematic relationship between base rate neglect and conservatism within individuals. We suggest that this method shows great potential for studying human probabilistic reasoning.

Question-answer dynamics and confirmation theory in reasoning by representativeness

We show that probabilistic decision-making behavior characteristic of reasoning by representativeness or typicality arises in minimalistic settings lacking many of the features previously thought to be necessary conditions for the phenomenon. Specifically, we develop a version of a classical experiment by Kahneman and Tversky (1973) on base-rate neglect, where participants have full access to the probabilistic distribution, conveyed entirely visually and without reliance on familiar stereotypes, rich descriptions, or individuating information. We argue that the notion of evidential support as studied in (Bayesian) confirmation theory offers a good account of our experimental findings, as has been proposed for related data points from the representativeness literature. In a nutshell, when faced with competing alternatives to choose from, humans are sometimes less interested in picking the option with the highest probability of being true (posterior probability), and instead choose the option best supported by available evidence. We point out that this theoretical avenue is descriptively powerful, but has an as-yet unclear explanatory dimension. Building on approaches to reasoning from linguistic semantics, we propose that the chief trigger of confirmation-theoretic mechanisms in deliberate reasoning is a linguistically-motivated tendency to interpret certain experimental setups as intrinsically contrastive, in a way best cashed out by modern linguistic semantic theories of questions. These questions generate pragmatic pressures for interpreting surrounding information as having been meant to help answer the question, which will naturally give rise to confirmation-theoretic effects, very plausibly as a byproduct of iterated Bayesian update as proposed by modern Bayesian theories of relevance-based reasoning in pragmatics. Our experiment provides preliminary but tantalizing evidence in favor of this hypothesis, as participants displayed significantly more confirmation-theoretic behavior in a condition that highlighted the question-like, contrastive nature of the task.

Cognitive Biases: Mistakes or Missing Stakes?

Despite decades of research on heuristics and biases, evidence on the effect of large incentives on cognitive biases is scant. We test the effect of incentives on four widely documented biases: base-rate neglect, anchoring, failure of contingent thinking, and intuitive reasoning. In laboratory experiments with 1,236 college students in Nairobi, we implement three incentive levels: no incentives, standard lab payments, and very high incentives. We find that very high stakes increase response times by 40% but improve performance only very mildly or not at all. In none of the tasks do very high stakes come close to de-biasing participants.

On the generality and cognitive basis of base-rate neglect

ABSTRACTBase rate neglect refers to people’s apparent tendency to underweight or even ignore base rate information when estimating posterior probabilities for events, such as the probability that a person with a positive cancer-test outcome actually does have cancer. While many studies have replicated the effect, there has been little variation in the structure of the reasoning problems used in those studies. In particular, most experiments have used extremely low base rates, high hit rates, and low false alarm rates. As a result, it is unclear whether the effect is a general phenomenon in human probabilistic reasoning or an anomaly that applies only to a small subset of reasoning problems. Moreover, previous studies have focused on describing empirical patterns of the effect and not so much on the underlying strategies. Here, we address these limitations by testing participants on a broader problem space and modelling their response at a single-participant level. We find that the empirical patterns that have served as evidence for base-rate neglect generalize to the larger problem space. At the level of individuals, we find evidence for large variability in how sensitive participants are to base rates, but with two distinct groups: those who largely ignore base rates and those who almost perfectly account for it. This heterogeneity is reflected in the cognitive modeling results, which reveal that there is not a single strategy that best captures the data for all participants. The overall best model is a variant of the Bayesian model with too conservative priors, tightly followed by a linear-additive integration model. Surprisingly, we find very little evidence for earlier proposed heuristic models. Altogether, our results suggest that the effect known as “base-rate neglect” generalizes to a large set of reasoning problems, but may need a reinterpretation in terms of the underlying cognitive mechanisms.