scholarly journals Prediction Error-Based Action Policy Learning for Quadcopter Flight Control

2021 ◽  
Vol 12 (1) ◽  
pp. 47
Author(s):  
Jamal Shams Khanzada ◽  
Wasif Muhammad ◽  
Muhammad Jehanzeb Irshad
Keyword(s):  
Flight Control ◽  
Prediction Error ◽  
Predictive Coding ◽  
Control Policy ◽  
Ground Truth ◽  
Effective Control ◽  
Training Time ◽  
Complementary Approach ◽  
Autopilot Systems ◽  
Action Spaces

Quadcopters are finding their place in everything from transportation, delivery, hospitals, and to homes in almost every part of daily life. In places where human intervention for quadcopter flight control is impossible, it becomes necessary to equip drones with intelligent autopilot systems so that they can make decisions on their own. All previous reinforcement learning (RL)-based efforts for quadcopter flight control in complex, dynamic, and unstructured environments remained unsuccessful during the training phase in avoiding the trend of catastrophic failures by naturally unstable quadcopters. In this work, we propose a complementary approach for quadcopter flight control using prediction error as an effective control policy reward in the sensory space instead of rewards from unstable action spaces alike in conventional RL approaches. The proposed predictive coding biased competition using divisive input modulation (PC/BC-DIM) neural network learns prediction error-based flight control policy without physically actuating quadcopter propellers, which ensures its safety during training. The proposed network learned flight control policy without any physical flights, which reduced the training time to almost zero. The simulation results showed that the trained agent reached the destination accurately. For 20 quadcopter flight trails, the average path deviation from the ground truth was 1.495 and the root mean square (RMS) of the goal reached 1.708.

#3083 Fibromyalgia and myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS): an interoceptive predictive coding model of pain and fatigue expression

2021 ◽  
Vol 92 (8) ◽  
pp. A3.3-A4
Author(s):  
Harriet Sharp ◽  
Kristy Themelis ◽  
Marisa Amato ◽  
Andrew Barritt ◽  
Kevin Davies ◽  
...  
Keyword(s):  
Prediction Error ◽  
Predictive Coding ◽  
Chronic Fatigue ◽  
Self Report ◽  
Subjective Experiences ◽  
Double Blind ◽  
Pain Thresholds ◽  
Pressure Pain Thresholds ◽  
Interoceptive Accuracy ◽  
Bodily States

IntroductionThe aetiology and pathophysiology of fibromyalgia and ME/CFS are poorly characterised but altered inflammatory, autonomic and interoceptive processes have been implicated. Interoception has been conceptualised as a predictive coding process; where top-down prediction signals compare to bottom-up afferents, resulting in prediction error signals indicating mismatch between expected and actual bodily states. Chronic dyshomeostasis and elevated interoceptive prediction error signals have been theorised to contribute to the expression of pain and fatigue in fibromyalgia and ME/CFS.Objectives/AimsTo investigate how altered interoception and prediction error relates to baseline expression of pain and fatigue in fibromyalgia and ME/CFS and in response to an inflammatory challenge.MethodsSixty-five patients with fibromyalgia and/or ME/CFS diagnosis and 26 matched controls underwent baseline assessment: self-report questionnaires assessing subjective pain and fatigue and objective measurements of pressure-pain thresholds. Participants received injections of typhoid (inflammatory challenge) or saline (placebo) in a randomised, double-blind, crossover design, then completed heartbeat tracking task (assessing interoceptive accuracy). Porges Body Questionnaire assessed interoceptive sensibility. Interoceptive prediction error (IPE) was calculated as discrepancy between objective accuracy and subjective sensibility.ResultsPatients with fibromyalgia and ME/CFS had significantly higher IPE (suggesting tendency to over-estimate interoceptive ability) and interoceptive sensibility, despite no differences in interoceptive accuracy. IPE and sensibility correlated positively with all self-report fatigue and pain measures, and negatively with pain thresholds. Following inflammatory challenge, IPE correlated negatively with the mismatch between subjective and objective measures of pain induced by inflammation.ConclusionsThis is the first study to reveal altered interoception processes in patients with fibromyalgia and ME/CFS, who are known to have dysregulated autonomic function. Notably, we found elevated IPE in patients, correlating with their subjective experiences of pain and fatigue. We hypothesise a predictive coding model, where mismatch between expected and actual internal bodily states (linked to autonomic dysregulation) results in prediction error signalling which could be metacognitively interpreted as chronic pain and fatigue. Our results demonstrate potential for further exploration of interoceptive processing in patients with fibromyalgia and ME/CFS, aiding understanding of these poorly defined conditions and providing potential new targets for diagnostic and therapeutic intervention.

scholarly journals Smoking as a Common Modulator of Sensory Gating and Reward Learning in Individuals with Psychotic Disorders

2021 ◽  
Vol 11 (12) ◽  
pp. 1581
Author(s):  
Alexis E. Whitton ◽  
Kathryn E. Lewandowski ◽  
Mei-Hua Hall
Keyword(s):  
Bipolar Disorder ◽  
Prediction Error ◽  
Psychotic Disorders ◽  
Predictive Coding ◽  
Sensory Gating ◽  
Event Related Potential ◽  
Reward Learning ◽  
Smoking History ◽  
Impaired Performance ◽  
Psychotic Bipolar Disorder

Motivational and perceptual disturbances co-occur in psychosis and have been linked to aberrations in reward learning and sensory gating, respectively. Although traditionally studied independently, when viewed through a predictive coding framework, these processes can both be linked to dysfunction in striatal dopaminergic prediction error signaling. This study examined whether reward learning and sensory gating are correlated in individuals with psychotic disorders, and whether nicotine—a psychostimulant that amplifies phasic striatal dopamine firing—is a common modulator of these two processes. We recruited 183 patients with psychotic disorders (79 schizophrenia, 104 psychotic bipolar disorder) and 129 controls and assessed reward learning (behavioral probabilistic reward task), sensory gating (P50 event-related potential), and smoking history. Reward learning and sensory gating were correlated across the sample. Smoking influenced reward learning and sensory gating in both patient groups; however, the effects were in opposite directions. Specifically, smoking was associated with improved performance in individuals with schizophrenia but impaired performance in individuals with psychotic bipolar disorder. These findings suggest that reward learning and sensory gating are linked and modulated by smoking. However, disorder-specific associations with smoking suggest that nicotine may expose pathophysiological differences in the architecture and function of prediction error circuitry in these overlapping yet distinct psychotic disorders.

scholarly journals Prediction-error signals to violated expectations about person identity and head orientation are doubly-dissociated across the dorsal and ventral visual streams

2018 ◽  
Author(s):  
Jonathan E. Robinson ◽  
Will Woods ◽  
Sumie Leung ◽  
Jordy Kaufman ◽  
Michael Breakspear ◽  
...  
Keyword(s):  
Error Detection ◽  
Visual Processing ◽  
Prediction Error ◽  
Predictive Coding ◽  
Angular Gyrus ◽  
Head Orientation ◽  
Visual Stream ◽  
Person Identity ◽  
Violated Expectations ◽  
Double Dissociations

AbstractPredictive coding theories of perception suggest the importance of constantly updated internal models of the world in predicting future sensory inputs. One implication of such models is that cortical regions whose function is to resolve particular stimulus attributes should also signal prediction violations with respect to those same stimulus attributes. Previously, through carefully designed experiments, we have demonstrated early-mid latency EEG/MEG prediction-error signals in the dorsal visual stream to violated expectations about stimulus orientation/trajectory, with localisations consistent with cortical areas processing motion and orientation. Here we extend those methods to simultaneously investigate the predictive processes in both dorsal and ventral visual streams. In this MEG study we employed a contextual trajectory paradigm that builds expectations using a series of image presentations. We created expectations about both face orientation and identity, either of which can subsequently be violated. Crucially this paradigm allows us to parametrically test double dissociations between these different types of violations. The study identified double dissociations across the type of violation in the dorsal and ventral visual streams, such that the right fusiform gyrus showed greater evidence of prediction-error signals to Identity violations than to Orientation violations, whereas the left angular gyrus and postcentral gyrus showed the opposite pattern of results. Our results suggest comparable processes for error checking and context updating in high-level expectations instantiated across both perceptual streams. Perceptual prediction-error signalling is initiated in regions associated with the processing of different stimulus properties.Significance StatementVisual processing occurs along ‘what’ and ‘where’ information streams that run, respectively along the ventral and dorsal surface of the posterior brain. Predictive coding models of perception imply prediction-error detection processes that are instantiated at the level where particular stimulus attributes are parsed. This implies that, for instance, when considering face stimuli, signals arising through violated expectations about the person identity of the stimulus should localise to the ventral stream, whereas signals arising through violated expectations about head orientation should localise to the dorsal stream. We test this in a magnetoencephalography source localisation study. The analysis confirmed that prediction-error signals to identity versus head-orientation occur with similar latency, but activate doubly-dissociated brain regions along ventral and dorsal processing streams.

scholarly journals Dynamics of bed bug infestations and control under disclosure policies

2019 ◽  
Vol 116 (13) ◽  
pp. 6473-6481 ◽  
Author(s):  
Sherrie Xie ◽  
Alison L. Hill ◽  
Chris R. Rehmann ◽  
Michael Z. Levy
Keyword(s):  
Cost Benefit ◽  
Control Policy ◽  
Spillover Effects ◽  
The United States ◽  
Effective Control ◽  
Bed Bugs ◽  
Bed Bug ◽  
Highly Nonlinear ◽  
Public Health Practitioners ◽  
And Control

Bed bugs have reemerged in the United States and worldwide over recent decades, presenting a major challenge to both public health practitioners and housing authorities. A number of municipalities have proposed or initiated policies to stem the bed bug epidemic, but little guidance is available to evaluate them. One contentious policy is disclosure, whereby landlords are obligated to notify potential tenants of current or prior bed bug infestations. Aimed to protect tenants from leasing an infested rental unit, disclosure also creates a kind of quarantine, partially and temporarily removing infested units from the market. Here, we develop a mathematical model for the spread of bed bugs in a generalized rental market, calibrate it to parameters of bed bug dispersion and housing turnover, and use it to evaluate the costs and benefits of disclosure policies to landlords. We find disclosure to be an effective control policy to curb infestation prevalence. Over the short term (within 5 years), disclosure policies result in modest increases in cost to landlords, while over the long term, reductions of infestation prevalence lead, on average, to savings. These results are insensitive to different assumptions regarding the prevalence of infestation, rate of introduction of bed bugs from other municipalities, and the strength of the quarantine effect created by disclosure. Beyond its application to bed bugs, our model offers a framework to evaluate policies to curtail the spread of household pests and is appropriate for systems in which spillover effects result in highly nonlinear cost–benefit relationships.

EXISTENCE OF OPTIMAL STATIONARY POLICIES IN FINITE DYNAMIC PROGRAMS WITH NONNEGATIVE REWARDS

2001 ◽  
Vol 15 (4) ◽  
pp. 557-564 ◽  
Author(s):  
Rolando Cavazos-Cadena ◽  
Raúl Montes-de-Oca
Keyword(s):  
Control Policy ◽  
Stationary Policy ◽  
Reward Function ◽  
Total Reward ◽  
Dynamic Programs ◽  
Finite State ◽  
Markov Decision ◽  
Optimal Stationary Policy ◽  
Action Spaces ◽  
Discounted Criterion

This article concerns Markov decision chains with finite state and action spaces, and a control policy is graded via the expected total-reward criterion associated to a nonnegative reward function. Within this framework, a classical theorem guarantees the existence of an optimal stationary policy whenever the optimal value function is finite, a result that is obtained via a limit process using the discounted criterion. The objective of this article is to present an alternative approach, based entirely on the properties of the expected total-reward index, to establish such an existence result.

scholarly journals Prediction suppression and surprise enhancement in monkey inferotemporal cortex

2017 ◽  
Vol 118 (1) ◽  
pp. 374-382 ◽  
Author(s):  
Suchitra Ramachandran ◽  
Travis Meyer ◽  
Carl R. Olson
Keyword(s):  
Visual System ◽  
Strong Evidence ◽  
Prediction Error ◽  
Predictive Coding ◽  
Neutral Condition ◽  
Training Period ◽  
Inferotemporal Cortex ◽  
Limited Evidence ◽  
Fixed Sequence ◽  
Area Te

Exposing monkeys, over the course of days and weeks, to pairs of images presented in fixed sequence, so that each leading image becomes a predictor for the corresponding trailing image, affects neuronal visual responsiveness in area TE. At the end of the training period, neurons respond relatively weakly to a trailing image when it appears in a trained sequence and, thus, confirms prediction, whereas they respond relatively strongly to the same image when it appears in an untrained sequence and, thus, violates prediction. This effect could arise from prediction suppression (reduced firing in response to the occurrence of a probable event) or surprise enhancement (elevated firing in response to the omission of a probable event). To identify its cause, we compared firing under the prediction-confirming and prediction-violating conditions to firing under a prediction-neutral condition. The results provide strong evidence for prediction suppression and limited evidence for surprise enhancement. NEW & NOTEWORTHY In predictive coding models of the visual system, neurons carry signed prediction error signals. We show here that monkey inferotemporal neurons exhibit prediction-modulated firing, as posited by these models, but that the signal is unsigned. The response to a prediction-confirming image is suppressed, and the response to a prediction-violating image may be enhanced. These results are better explained by a model in which the visual system emphasizes unpredicted events than by a predictive coding model.

scholarly journals Placing meta-stable states of consciousness within the predictive coding hierarchy: The deceleration of the accelerated prediction error

2018 ◽  
Vol 63 ◽  
pp. 123-142 ◽  
Author(s):  
Amirali Shirazibeheshti ◽  
Jennifer Cooke ◽  
Srivas Chennu ◽  
Ram Adapa ◽  
David K. Menon ◽  
...  
Keyword(s):  
Prediction Error ◽  
Predictive Coding ◽  
Stable States ◽  
States Of Consciousness

The Design and Evaluation of Complex Systems: Application to a Man-Machine Interface for Aerial Navigation

1979 ◽  
Vol 23 (1) ◽  
pp. 75-79 ◽  
Author(s):  
Dennis B. Beringer
Keyword(s):  
Flight Control ◽  
Tracking Error ◽  
Data Entry ◽  
Digital Data ◽  
Training Time ◽  
Subsequent Performance ◽  
Navigation Data ◽  
Wide Range ◽  
Dynamic Map ◽  
Aerial Navigation

Systematic and economic design and evaluation strategies were applied to a computer-generated 4-D aerial navigation system. During the evaluation each of 24 experienced instrument pilots received training in a PLATO-based digital flight simulator using either a keyboard entry/static map, keyboard entry/dynamic map, or touch entry/dynamic map system. Tasks performed during the execution of an area navigation course included continuous flight control, navigation data updating, digital data entry, and amended course plotting. Digital data entry training time was comparable for all three systems but the touch-map proved superior for the plotting tasks, greatly reducing training and task execution times while virtually eliminating errors. Subsequent performance evaluation showed that the touch-map reduced flight path tracking error, increased processing rates on a digit-cancelling secondary task, and increased the accuracy of manual plotting operations. It was concluded that a touch entry system could significantly reduce cockpit workload across a wide range of operational environments.

Robust Flight Control System Design with Cerebellar Model Articulation Controller

2011 ◽  
Vol 145 ◽  
pp. 579-582
Author(s):  
Y.J. Huang ◽  
T.C. Kuo ◽  
C. Y. Chen ◽  
B.W. Hong ◽  
P. C. Wu
Keyword(s):  
Flight Control ◽  
Control Method ◽  
Controller Design ◽  
Effective Control ◽  
Wind Gust ◽  
Cerebellar Model Articulation Controller ◽  
Pd Controller ◽  
Robust Tracking ◽  
Robust Tracking Control ◽  
On Line

This paper presents a robust proportional-derivative (PD) based cerebellar model articulation controller (CMAC) for vertical take-off and landing flight control systems. It is known that PD control is a simple and effective control method. However, it does not ensure the robustness if it is used alone for uncertain systems. CMAC can be used for robust control. However, it requires training patterns for tuning some weighting factors. A novel CMAC incorporating with a PD controller design is proposed in this paper. Successful on-line training and recalling process of CMAC accompanying the PD controller was developed. The advantage of the proposed method is mainly the robust tracking performance against aerodynamic parametric variation and external wind gust. Even when the PD controller is not designed well, the CMAC is capable of doing a robust tracking control through on-line recalling and training procedures.

Sign in / Sign up

Export Citation Format

Share Document