scholarly journals Bayesian inference with incomplete knowledge explains perceptual confidence and its deviations from accuracy

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Koosha Khalvati ◽  
Roozbeh Kiani ◽  
Rajesh P. N. Rao
Keyword(s):  
Bayesian Inference ◽  
Sensory Information ◽  
Apparent Difference ◽  
Incomplete Knowledge ◽  
Markov Decision ◽  
Subjective Confidence ◽  
Opposing Effects ◽  
Hidden States ◽  
Partially Observable ◽  
Computational Processes

AbstractIn perceptual decisions, subjects infer hidden states of the environment based on noisy sensory information. Here we show that both choice and its associated confidence are explained by a Bayesian framework based on partially observable Markov decision processes (POMDPs). We test our model on monkeys performing a direction-discrimination task with post-decision wagering, demonstrating that the model explains objective accuracy and predicts subjective confidence. Further, we show that the model replicates well-known discrepancies of confidence and accuracy, including the hard-easy effect, opposing effects of stimulus variability on confidence and accuracy, dependence of confidence ratings on simultaneous or sequential reports of choice and confidence, apparent difference between choice and confidence sensitivity, and seemingly disproportionate influence of choice-congruent evidence on confidence. These effects may not be signatures of sub-optimal inference or discrepant computational processes for choice and confidence. Rather, they arise in Bayesian inference with incomplete knowledge of the environment.

scholarly journals Bayesian Inference with Incomplete Knowledge Explains Perceptual Confidence and its Deviations from Accuracy

2020 ◽  
Author(s):  
Koosha Khalvati ◽  
Roozbeh Kiani ◽  
Rajesh P. N. Rao
Keyword(s):  
Bayesian Inference ◽  
Sensory Information ◽  
Apparent Difference ◽  
Incomplete Knowledge ◽  
Markov Decision ◽  
Subjective Confidence ◽  
Opposing Effects ◽  
Hidden States ◽  
Partially Observable ◽  
Computational Processes

AbstractIn perceptual decisions, subjects infer hidden states of the environment based on noisy sensory information. Here we show that both choice and its associated confidence are explained by a Bayesian framework based on partially observable Markov decision processes (POMDPs). We test our model on monkeys performing a direction-discrimination task with post-decision wagering, demonstrating that the model explains objective accuracy and predicts subjective confidence. Further, we show that the model replicates well-known discrepancies of confidence and accuracy, including the hard-easy effect, opposing effects of stimulus volatility on confidence and accuracy, dependence of confidence ratings on simultaneous or sequential reports of choice and confidence, apparent difference between choice and confidence sensitivity, and seemingly disproportionate influence of choice-congruent evidence on confidence. These effects may not be signatures of sub-optimal inference or discrepant computational processes for choice and confidence. Rather, they arise in Bayesian inference with incomplete knowledge of the environment.

scholarly journals Bayesian Inference of Other Minds Explains Human Choices in Group Decision Making

2018 ◽  
Author(s):  
Koosha Khalvati ◽  
Seongmin A. Park ◽  
Saghar Mirbagheri ◽  
Remi Philippe ◽  
Mariateresa Sestito ◽  
...  
Keyword(s):  
Decision Making ◽  
Bayesian Inference ◽  
Markov Decision Processes ◽  
Group Decision Making ◽  
Group Decision ◽  
Decision Processes ◽  
Other Minds ◽  
Markov Decision ◽  
The Mind ◽  
Partially Observable

AbstractTo make decisions in a social context, humans have to predict the behavior of others, an ability that is thought to rely on having a model of other minds known as theory of mind. Such a model becomes especially complex when the number of people one simultaneously interacts is large and the actions are anonymous. Here, we show that in order to make decisions within a large group, humans employ Bayesian inference to model the “mind of the group,” making predictions of others’ decisions while also considering the effects of their own actions on the group as a whole. We present results from a group decision making task known as the Volunteers Dilemma and demonstrate that a Bayesian model based on partially observable Markov decision processes outperforms existing models in quantitatively explaining human behavior. Our results suggest that in group decision making, rather than acting based solely on the rewards received thus far, humans maintain a model of the group and simulate the group’s dynamics into the future in order to choose an action as a member of the group.

scholarly journals Modeling other minds: Bayesian inference explains human choices in group decision-making

2019 ◽  
Vol 5 (11) ◽  
pp. eaax8783 ◽  
Author(s):  
Koosha Khalvati ◽  
Seongmin A. Park ◽  
Saghar Mirbagheri ◽  
Remi Philippe ◽  
Mariateresa Sestito ◽  
...  
Keyword(s):  
Decision Making ◽  
Bayesian Inference ◽  
Group Decision Making ◽  
Group Decision ◽  
Other Minds ◽  
Group Interactions ◽  
Markov Decision ◽  
The Mind ◽  
Partially Observable ◽  
Large Groups

To make decisions in a social context, humans have to predict the behavior of others, an ability that is thought to rely on having a model of other minds known as “theory of mind.” Such a model becomes especially complex when the number of people one simultaneously interacts with is large and actions are anonymous. Here, we present results from a group decision-making task known as the volunteer’s dilemma and demonstrate that a Bayesian model based on partially observable Markov decision processes outperforms existing models in quantitatively predicting human behavior and outcomes of group interactions. Our results suggest that in decision-making tasks involving large groups with anonymous members, humans use Bayesian inference to model the “mind of the group,” making predictions of others’ decisions while also simulating the effects of their own actions on the group’s dynamics in the future.

Automatic landmark discovery for learning agents under partial observability

2019 ◽  
Vol 34 ◽  
Author(s):  
Alper Demіr ◽  
Erkіn Çіlden ◽  
Faruk Polat
Keyword(s):  
Learning Algorithm ◽  
Multiple Instance Learning ◽  
Learning Performance ◽  
Learning Context ◽  
Solution Quality ◽  
Learning Agents ◽  
Markov Decision ◽  
Hidden States ◽  
Partially Observable ◽  
Diverse Density

Abstract In the reinforcement learning context, a landmark is a compact information which uniquely couples a state, for problems with hidden states. Landmarks are shown to support finding good memoryless policies for Partially Observable Markov Decision Processes (POMDP) which contain at least one landmark. SarsaLandmark, as an adaptation of Sarsa(λ), is known to promise a better learning performance with the assumption that all landmarks of the problem are known in advance. In this paper, we propose a framework built upon SarsaLandmark, which is able to automatically identify landmarks within the problem during learning without sacrificing quality, and requiring no prior information about the problem structure. For this purpose, the framework fuses SarsaLandmark with a well-known multiple-instance learning algorithm, namely Diverse Density (DD). By further experimentation, we also provide a deeper insight into our concept filtering heuristic to accelerate DD, abbreviated as DDCF (Diverse Density with Concept Filtering), which proves itself to be suitable for POMDPs with landmarks. DDCF outperforms its antecedent in terms of computation speed and solution quality without loss of generality. The methods are empirically shown to be effective via extensive experimentation on a number of known and newly introduced problems with hidden state, and the results are discussed.

scholarly journals Factorized Asymptotic Bayesian Policy Search for POMDPs

2017 ◽  
Author(s):  
Masaaki Imaizumi ◽  
Ryohei Fujimaki
Keyword(s):  
Bayesian Inference ◽  
Model Selection ◽  
Policy Search ◽  
Total Rewards ◽  
Partially Observed ◽  
Markov Decision ◽  
Selection For ◽  
Hidden States ◽  
Flexible Model

This paper proposes a novel direct policy search (DPS) method with model selection for partially observed Markov decision processes (POMDPs). DPSs have been standard for learning POMDPs due to their computational efficiency and natural ability to maximize total rewards. An important open challenge for the best use of DPS methods is model selection, i.e., determination of the proper dimensionality of hidden states and complexity of policy functions, to mitigate overfitting in highly-flexible model representations of POMDPs. This paper bridges Bayesian inference and reward maximization and derives marginalized weighted log-likelihood~(MWL) for POMDPs which takes both advantages of Bayesian model selection and DPS. Then we propose factorized asymptotic Bayesian policy search (FABPS) to explore the model and the policy which maximizes MWL by expanding recently-developed factorized asymptotic Bayesian inference. Experimental results show that FABPS outperforms state-of-the-art model selection methods for POMDPs, with respect both to model selection and to expected total rewards.

Optimal adaptive inspection and maintenance for redundant systems

2021 ◽  
pp. 1748006X2110201
Author(s):  
Chaochao Lin ◽  
Matteo Pozzi
Keyword(s):  
Engineering Systems ◽  
Discounted Cost ◽  
Markov Decision ◽  
Inspection And Maintenance ◽  
And Performance ◽  
Partially Observable ◽  
Series Systems ◽  
Selection Of ◽  
Redundant Systems

Optimal exploration of engineering systems can be guided by the principle of Value of Information (VoI), which accounts for the topological important of components, their reliability and the management costs. For series systems, in most cases higher inspection priority should be given to unreliable components. For redundant systems such as parallel systems, analysis of one-shot decision problems shows that higher inspection priority should be given to more reliable components. This paper investigates the optimal exploration of redundant systems in long-term decision making with sequential inspection and repairing. When the expected, cumulated, discounted cost is considered, it may become more efficient to give higher inspection priority to less reliable components, in order to preserve system redundancy. To investigate this problem, we develop a Partially Observable Markov Decision Process (POMDP) framework for sequential inspection and maintenance of redundant systems, where the VoI analysis is embedded in the optimal selection of exploratory actions. We investigate the use of alternative approximate POMDP solvers for parallel and more general systems, compare their computation complexities and performance, and show how the inspection priorities depend on the economic discount factor, the degradation rate, the inspection precision, and the repair cost.

POMDP-Based Coding of Child–Robot Interaction within a Robot-Assisted ASD Diagnostic Protocol

2018 ◽  
Vol 15 (02) ◽  
pp. 1850011 ◽  
Author(s):  
Frano Petric ◽  
Damjan Miklić ◽  
Zdenko Kovačić
Keyword(s):  
Humanoid Robots ◽  
Autism Diagnostic Observation Schedule ◽  
Autism Spectrum ◽  
Diagnostic Process ◽  
Sequential Decision ◽  
Robot Assisted ◽  
Robot Interaction ◽  
Fine Grained ◽  
Markov Decision ◽  
Partially Observable

The existing procedures for autism spectrum disorder (ASD) diagnosis are often time consuming and tiresome both for highly-trained human evaluators and children, which may be alleviated by using humanoid robots in the diagnostic process. Hence, this paper proposes a framework for robot-assisted ASD evaluation based on partially observable Markov decision process (POMDP) modeling, specifically POMDPs with mixed observability (MOMDPs). POMDP is broadly used for modeling optimal sequential decision making tasks under uncertainty. Spurred by the widely accepted autism diagnostic observation schedule (ADOS), we emulate ADOS through four tasks, whose models incorporate observations of multiple social cues such as eye contact, gestures and utterances. Relying only on those observations, the robot provides an assessment of the child’s ASD-relevant functioning level (which is partially observable) within a particular task and provides human evaluators with readable information by partitioning its belief space. Finally, we evaluate the proposed MOMDP task models and demonstrate that chaining the tasks provides fine-grained outcome quantification, which could also increase the appeal of robot-assisted diagnostic protocols in the future.

Sign in / Sign up

Export Citation Format

Share Document