Human Belief State-Based Exploration and Exploitation in an Information-Selective Symmetric Reversal Bandit Task

Humans often face sequential decision-making problems, in which information about the environmental reward structure is detached from rewards for a subset of actions. In the current exploratory study, we introduce an information-selective symmetric reversal bandit task to model such situations and obtained choice data on this task from 24 participants. To arbitrate between different decision-making strategies that participants may use on this task, we developed a set of probabilistic agent-based behavioral models, including exploitative and explorative Bayesian agents, as well as heuristic control agents. Upon validating the model and parameter recovery properties of our model set and summarizing the participants’ choice data in a descriptive way, we used a maximum likelihood approach to evaluate the participants’ choice data from the perspective of our model set. In brief, we provide quantitative evidence that participants employ a belief state-based hybrid explorative-exploitative strategy on the information-selective symmetric reversal bandit task, lending further support to the finding that humans are guided by their subjective uncertainty when solving exploration-exploitation dilemmas.

The Fragmented Mind

Mental fragmentation is the thesis that the mind is fragmented, or compartmentalized. Roughly, this means that an agent’s overall belief state is divided into several sub-states—fragments. These fragments need not make for a consistent and deductively closed belief system. The thesis of mental fragmentation became popular through the work of philosophers like Christopher Cherniak, David Lewis, and Robert Stalnaker in the 1980s. Recently, it has attracted great attention again. This volume is the first collection of essays devoted to the topic of mental fragmentation. It features important new contributions by leading experts in the philosophy of mind, epistemology, and philosophy of language. Opening with an accessible Introduction providing a systematic overview of the current debate, the fourteen essays cover a wide range of issues: foundational issues and motivations for fragmentation, the rationality or irrationality of fragmentation, fragmentation’s role in language, the relationship between fragmentation and mental files, and the implications of fragmentation for the analysis of implicit attitudes.

Fragmented but Rational

Is it a failure of rationality for one’s belief state to be compartmentalized into fragments that don’t cohere with each other? More generally, is being fragmented itself a kind of rational failure? This chapter discusses both questions, favoring negative answers.

Distribution Bisimilarity via the Power of Convex Algebras

Probabilistic automata (PA), also known as probabilistic nondeterministic labelled transition systems, combine probability and nondeterminism. They can be given different semantics, like strong bisimilarity, convex bisimilarity, or (more recently) distribution bisimilarity. The latter is based on the view of PA as transformers of probability distributions, also called belief states, and promotes distributions to first-class citizens. We give a coalgebraic account of distribution bisimilarity, and explain the genesis of the belief-state transformer from a PA. To do so, we make explicit the convex algebraic structure present in PA and identify belief-state transformers as transition systems with state space that carries a convex algebra. As a consequence of our abstract approach, we can give a sound proof technique which we call bisimulation up-to convex hull. Comment: Full (extended) version of a CONCUR 2017 paper, minor revision of the LMCS submission

Wireless Body Area Network Control Policies for Energy-Efficient Health Monitoring

Wireless body area networks (WBANs) have strong potential in the field of health monitoring. However, the energy consumption required for accurate monitoring determines the time between battery charges of the wearable sensors, which is a key performance factor (and can be critical in the case of implantable devices). In this paper, we study the inherent trade-off between the power consumption of the sensors and the probability of misclassifying a patient’s health state. We formulate this trade-off as a dynamic problem, in which at each step, we can choose to activate a subset of sensors that provide noisy measurements of the patient’s health state. We assume that the (unknown) health state follows a Markov chain, so our problem is formulated as a partially observable Markov decision problem (POMDP). We show that all the past measurements can be summarized as a belief state on the true health state of the patient, which allows tackling the POMDP problem as an MDP on the belief state. Then, we empirically study the performance of a greedy one-step look-ahead policy compared to the optimal policy obtained by solving the dynamic program. For that purpose, we use an open-source Continuous Glucose Monitoring (CGM) dataset of 232 patients over six months and extract the transition matrix and sensor accuracies from the data. We find that the greedy policy saves ≈50% of the energy costs while reducing the misclassification costs by less than 2% compared to the most accurate policy possible that always activates all sensors. Our sensitivity analysis reveals that the greedy policy remains nearly optimal across different cost parameters and a varying number of sensors. The results also have practical importance, because while the optimal policy is too complicated, a greedy one-step look-ahead policy can be easily implemented in WBAN systems.

Strong knowledge, weak belief?

According to the knowledge norm of belief (Williamson in Knowledge and its limits, Oxford University Press, Oxford, p. 47, 2000), one should believe p only if one knows p. However, it can easily seem that the ordinary notion of belief is much weaker than the knowledge norm would have it. It is possible to rationally believe things one knows to be unknown (Hawthorne et al. in Philos Stud 173:1393–1404, 2016; McGlynn in Noûs 47:385–407, 2013, Whiting in Chan (ed) The aim of belief, Oxford University Press, Oxford, 2013). One response to this observation is to develop a technical notion of ‘outright’ belief. A challenge for this line of response is to find a way of getting a grip on the targeted notion of belief. In order to meet this challenge, I characterize ‘outright’ belief in this paper as the strongest belief state implied by knowledge. I show that outright belief so construed allows this notion to play important theoretical roles in connection with knowledge, assertion and action.

Wireless Body Area Network Control Policies for Energy-Efficient Health Monitoring

Wireless body area networks (WBANs) have strong potential in the field of health monitoring. However, the energy consumption required for accurate monitoring limits the time between battery charges of the wearable sensors, which is a key performance factor (and can be critical in the case of implantable devices). In this paper, we study the inherent trade-off between the power consumption of the sensors and the probability of misclassifying a patient’s health state. We formulate this trade-off as a dynamic problem, in which at each step we can choose to activate a subset of sensors that provide noisy measurements of the patient’s health state. We assume that the (unknown) health state follows a Markov chain, so our problem is formulated as a partially observable Markov decision problem (POMDP). We show that all the past measurements can be summarized as a belief state on the true health state of the patient, which allows tackling the POMDP problem as an MDP on the belief state. We then empirically study the performance of a greedy one-step look-ahead policy compared to the optimal policy obtained by solving the dynamic program. For that purpose, we use an open-source Continuous Glucose Monitoring (CGM) data set of 232 patients over six months and extract the transition matrix and sensor accuracies from the data. We find that the greedy policy saves ~50% of the energy costs while reducing the misclassification costs by less than 2% compared to the most accurate policy possible that always activates all sensors. Our sensitivity analysis reveals that the greedy policy remains nearly optimal across different cost parameters and a varying number of sensors. The results also have practical importance, because while the optimal policy is too complicated, a greedy one-step look-ahead policy can be easily implemented in WBAN systems.

Serial reproduction reveals the geometry of visuospatial representations

An essential function of the human visual system is to locate objects in space and navigate the environment. Due to limited resources, the visual system achieves this by combining imperfect sensory information with a belief state about locations in a scene, resulting in systematic distortions and biases. These biases can be captured by a Bayesian model in which internal beliefs are expressed in a prior probability distribution over locations in a scene. We introduce a paradigm that enables us to measure these priors by iterating a simple memory task where the response of one participant becomes the stimulus for the next. This approach reveals an unprecedented richness and level of detail in these priors, suggesting a different way to think about biases in spatial memory. A prior distribution on locations in a visual scene can reflect the selective allocation of coding resources to different visual regions during encoding (“efficient encoding”). This selective allocation predicts that locations in the scene will be encoded with variable precision, in contrast to previous work that has assumed fixed encoding precision regardless of location. We demonstrate that perceptual biases covary with variations in discrimination accuracy, a finding that is aligned with simulations of our efficient encoding model but not the traditional fixed encoding view. This work demonstrates the promise of using nonparametric data-driven approaches that combine crowdsourcing with the careful curation of information transmission within social networks to reveal the hidden structure of shared visual representations.

Ignorance, Introspection, and Epistemic Modals

Embedded epistemic modals are infelicitous under desire predicates when they are anchored to the belief state of the attitude holder (see, esp., Anand & Hacquard 2013). We present two ways of deriving this observation from an inde- pendently motivated property of desire predicates (Heim 1992; von Fintel 1999).