Pathologies of Decision-Making

This chapter explores the pathophysiology of neural conditions related to the neural network of decision-making. If humans are not fully rational, they manage to pretend to be most of the time. Some individuals are distinguished by traits that influence their decision-making, such as impulsiveness, procrastination, and stubbornness. These behaviours are so common that they are not considered pathological. There are, however, cases in which the decision-making system is dysfunctional enough for this irrationality to go beyond socially acceptable norms. This is the field of neurological and psychiatric syndromes of decision-making. The chapter then examines in detail obsessive-compulsive disorders (OCDs), Tourette’s syndrome, Parkinson's disease, and hyperdopaminergic syndromes. It also describes the deep brain stimulation paradox.

A Ghost in the Machine

This chapter describes the neurobiological approach of decision-making. Until the late 1980s, ignoring the work of experimental economists and behaviourists, electrophysiologists restricted themselves to the study of sensory and motor function, believing it to be impossible for them to access cognitive processes. In 1989, William Newsome and Anthony Movshon broke the dogma while studying the role of neurons in the medio-temporal area of the cortex (an associative visual area) in the visual discrimination of macaques. They became the first researchers who were able to correlate decision-making with a pattern of electrophysiological activity in neurons. This correlation, which they called psychometric–neurometric pairing, became the backbone of all subsequent studies into the neurobiology of decision-making. The chapter then looks at the development of functional MRI, and presents a normative approach to decision-making and learning.

A Hierarchy of Decision-Making

This chapter explores the flexibility of the neural network described in the previous chapters. It also shows that the anterior part of the brain can be subdivided into five functional loops that underlie different executive functions. These five major loops are the motor loop, the oculomotor loop, the prefrontal loop, the orbitofrontal loop, and the cingular loop. The first two circuits deal with the learning and decision-making processes of the motor domain. The prefrontal and frontal circuits are involved in cognitive processes. Finally, the cingular circuit is involved in episodic memory, regulation of emotions, and modulation of mood. Therefore, one can already see a certain hierarchical order, underpinned by anatomical realities: the mood, emotions, and personal history of the subject (the memory) will condition the cognitive functions that will influence motor behaviours. This hierarchy can be concretized by direct interactions between the different loops, of which anatomical evidence has been demonstrated several times.

Introduction to Information Transfer in the Nervous System

This chapter discusses the modalities of information transfer in the nervous system. The nervous system is organised around specialised cells called neurons, which work as integration units that transform all received information into new information. The neurons generate unitary electric pulses of invariant form and duration called action potentials or spikes. Neurons have an intrinsic firing frequency that is their frequency of producing spikes when they are not influenced. The chapter then considers the two major families of neurotransmitters. In general, a neuron releases only one type of neurotransmitter belonging to one of these two families. The first family is that of excitatory neurotransmitters; the neurons that release them are naturally called excitatory neurons. When they bind with postsynaptic receptors, they have a facilitating effect on the production of action potentials. Meanwhile, inhibitory neurons release neurotransmitters whose binding with postsynaptic receptors decreases the discharge frequency of the postsynaptic neuron. The chapter also describes a special family of neurotransmitters: the neuro-modulators.

Free Will?

This chapter focuses on the issue of free will. Free will is defined as the ability to choose independently from any exogenous determinations. The seminal experiment on this subject was proposed by Benjamin Libet in the early eighties. His aim was to highlight that free will is only an illusion. This chapter describes the debate around the experiences initiated by Libet and the exact nature of the phenomenon observed. It then highlights the difference between intentionality and agency. Agency is related to who is at the origin of the action. Intentionality is related to the consciousness of acting and is therefore directly related to moral responsibility. Finally, the chapter argues that neuroscience may help but will be insufficient alone to conclude the possibility or impossibility of free will.

Bias and Heuristics

This chapter addresses the cognitive bias and heuristics of judgement. It also considers possible underlying neural mechanisms. Economist Herbert Simon introduced the notion of heuristics in judgement to define the approximate rational rules upon which individuals rely to make decisions. Experimental psychologists Daniel Kahneman and Amos Tversky transformed this notion of heuristics by highlighting the cognitive biases that influence judgements. From his work with Tversky, Kahneman elaborated the two-systems theory. According to him, human decision-making is the result of a competition between a fast, automatic system (System 1) that is prone to make mistakes and a slower, more demanding but also more reliable one (System 2). Both systems use heuristics, but the second compensates with anticipation. This chapter then looks at initial bias and beliefs. It also explains the anchoring effect, as well as the dilution effect. Anchoring is the excessive influence of a first impression on judgements.

The Machine-Learning Approach of Reinforcement Learning

This chapter assesses alternative approaches of reinforcement learning that are developed by machine learning. The initial goal of this branch of artificial intelligence, which appeared in the middle of the twentieth century, was to develop and implement algorithms that allow a machine to learn. Originally, they were computers or more or less autonomous robotic automata. As artificial intelligence has developed and cross-fertilized with neuroscience, it has begun to be used to model the learning and decision-making processes for biological agents, broadening the meaning of the word ‘machine’. Theoreticians of this discipline define several categories of learning, but this chapter only deals with those which are related to reinforcement learning. To understand how these algorithms work, it is necessary first of all to explain the Markov chain and the Markov decision-making process. The chapter then goes on to examine model-free reinforcement learning algorithms, the actor-critic model, and finally model-based reinforcement learning algorithms.

From Pallium to Cortex

This chapter assesses what the development of the cortex brings to the behavioural capacity of vertebrates, culminating with the theory of mind in humans. Two behavioural characteristics, related to each other, distinguish mammals from other vertebrates. The first is a period of dependence of infant mammals that can last several years. The second is play. Although the function of this activity is still debated, it is certain that it has a central role in the learning of foraging and social behaviour. Another property that is almost exclusive to mammals is the ability to recognize oneself. These specific behavioural features are correlated with a general increase in the encephalization quotient and also the ratio between the size of the telencephalon and the rest of the brain. This chapter then goes on to look at the unique capacities of humans, including language, the ability to anticipate, and consciousness.

The Lamprey’s Dilemma

This chapter focuses on the neural network, demonstrating how the principles described in the previous chapter are implemented in vertebrates, taking as a blueprint the oldest one: the lamprey. The reticulospinal neurons belong to the reticular formation located in the brainstem of the lamprey. These reticulospinal neurons act as the effector system. Apart from the peripheral input that comes back from the spinal cord, the reticular formation receives, among other things, input from the diencephalon and specifically the thalamus. This structure allows interfacing between sensory stimuli (visual, auditory, and olfactory) and the motor system. The other very important targets of the thalamus in the lamprey are the basal ganglia. The chapter then goes on to explain the diencephalic and telencephalic loops.

The Decision-Making Engine

This chapter presents an upgrade of the neural network by implementing the reward prediction error. It then compares the final product with the actor-critic model and discusses the similarities and differences. Reinforcement learning algorithms, more specifically actor-critic models, are currently very successful in the field of decision-making. They are notably related to properties of dopaminergic neurons which have not yet been addressed in previous chapters. It has been demonstrated that dopaminergic neurons respond when the subject receives a reward or when the subject associates a conditional stimulus with the reward, and that this response to the stimulus is proportional to the utility function of the reward. In fact, dopaminergic neurons behave exactly like a process that computes temporal difference. The amplitude of their response when the reward is administered is proportional to the difference between the expected utility at time and the reward actually obtained at the moment, i.e. the temporal difference. This chapter then assesses whether the telencephalic loop is an actor-critic system.