<p>Robots are entering our daily lives from self-driving cars to health-care robots. Historically, pre-programmed robots were vulnerable to changing conditions in daily life, primarily because of a lack of ability to generate novel, non-preset flexible solutions. Thus there is a need for robotics to incorporate adaptation, which is a trait of higher order natural species. This adaptation allows higher-order natural species to change their behaviours and internal mechanisms based on experience with often dynamic environment. The ability to adapt emerged through evolutionary processes. Evolutionary Robotics is an approach to create autonomous robots that are capable of automatically generating artificial behaviors and morphologies to achieve adaptation. Evolutionary robotics has the potential to automatically synthesize controllers for real autonomous robots and generate solutions to complete tasks in the uncertain real-world. Compared to the inflexibility of pre-programmed robots, evolutionary robots are able to learn flexible solutions to given tasks through evolutionary methods. Cognitive robotics, a branch of artificial cognitive systems research, is such an attempt to create autonomous robots by applying bio-inspired methods. As the robot interacts with environment, an underlying cognitive system can learn its own solutions toward task completion. This learning-solution-from-interaction approach, also termed as a Reinforcement Learning (RL) approach, is widely applied in cognitive robotics to learn the solutions automatically. Ideally, the solutions can emerge in the cognitive system through the trial-and-error process of the RL approach without introducing human bias. This thesis aims to develop an evolutionary cognitive architecture (system) for a robot that can learn adaptive solutions to complete tasks. Inspired by emotion theories, this work proposes Affective Computing Multilayer Cognitive Architecture (ACMCA), a universal cognitive architecture, which is able to learn diverse solutions. Extending from previous work, ACMCA has a five-layer structure, where each layer aims to achieve different components of the solutions. The position of this thesis is that introducing a novel emotion inspired multilayer architecture that produces task solutions through subsumption operations and underlying appropriate machine learning algorithms will allow a robot to complete admissible tasks. ACMCA’s five layers are: primary reinforcer layer, secondary reinforcer layer, core affect state layer, strategy layer, and behaviour layer. This five-layer decomposition also meets the traditional decomposition of a mobile control system into functional modules (e.g. perception, modelling, planning, task execution, and motor control). Each layer contains computing nodes as functional modules that process various Stimuli, Actions, and their consequential Outcomes of the cognitive system. In this work, 17 computing nodes and their connections in ACMCA represent the solutions that a mobile robot has learned to complete navigation tasks in complex scenarios. Inspired by the Constructive Theory 1 and the robotic subsumption system, this work proposes a contingency-based subsumption approach to construct ACMCA. This contingency is termed Stimuli-Action-Outcome Contingency (SAOC), which is extended from the Action-Outcome (AO) contingency of Construction Theory. SAOCs are represented by “if-then” rules, termed SAOC rules, which encapsulate Stimuli, Actions, and their consequential Outcomes, providing clear symbolic interpretations. That is, the symbolic meaning of a SAOC rule can be interpreted as: if the input stimulus is perceived, the output action will be advocated as a cognitive response, expecting the outcome of the action with an estimation of relevance. As low-level computing nodes encapsulate Stimulus, Actions, and Outcomes, high-level computing nodes can subsume these low-level ones through the form of SAOC rules. Therefore, the proposed ACMCA can be constructed by subsumption layers of Stimuli-Action-Outcome Contingency (SAOC) rules. This work applies machine learning techniques to facilitate ACMCA’s real-world robotic implementation. This work selects Accuracy-based Learning Classifier Systems (XCS) algorithms as the underlying machine learning techniques that are deployed at computing nodes for the contingency-based subsumption operations. The mitosis approach of XCS and the XCS with a Combined Reward method (XCSCR) are two novel variants of XCS algorithm. They are proposed to amend two challenges that occur when the standard XCS approaches are applied for robotic applications. The mitosis approach introduces an accuracy pressure into the algorithm’s evolutionary process, improving the algorithms’ performance in robotic applications where noisy interferences exist. The XCSCR enables the policy to emerge earlier and more frequently than the existing benchmark approaches in multistep problems. Therefore, a robot with the XCSCR can handle a multistep scenario more effectively than those with the benchmarked algorithms. This work conducts five experiments to test the capability of ACMCA and its underlying algorithms in learning solutions for robotic navigation tasks. The five experiments are conducted as follows: reflex-learning, IR-tuning, deliberation-establishing, emotion model, and combined reward assignment. As the results of the experiments, three different affective patterns have emerged in the first three experiments, an emotion model has emerged in the fourth experiments, and the fifth experiment explores ACMCA’s potential implementation in the life-long learning scenario. These results demonstrate that ACMCA, a novel emotion inspired multilayer architecture, can produce task solutions through contingency-based subsumption operations and underlying appropriate machine learning algorithms, allowing a robot to complete admissible tasks through evolutionary processes. The contingency-based subsumption operations can establish three contingencies and one emotion model between the subsumed components by multiple RL agents which deploy the proposed mitosis approach of XCS algorithms. These three emotion patterns and emotion model can consistently improve the robot’s navigation performance with interpretable explanations. These two variants of XCS algorithms can amend shortfalls of the standard XCS approach in real-world robotic implementations. It has been demonstrated that the diverse solutions learned by ACMCA improve the navigation performance of the robot in terms of higher flexibility, reduction in continuous collisions and shorter navigation time consumption.</p>
Optimal Poisson Cognitive System with Markov Learning Model
