DEEP REINFORCEMENT LEARNING WITH SPARSE DISTRIBUTED MEMORY FOR “WATER WORLD” PROBLEM SOLVING

Context. Machine learning is one of the actively developing areas of data processing. Reinforcement learning is a class of machine learning methods where the problem involves mapping the sequence of environmental states to agent’s actions. Significant progress in this area has been achieved using DQN-algorithms, which became one of the first classes of stable algorithms for learning using deep neural networks. The main disadvantage of this approach is the rapid growth of RAM in real-world tasks. The approach proposed in this paper can partially solve this problem. Objective. The aim is to develop a method of forming the structure and nature of access to the sparse distributed memory with increased information content to improve reinforcement learning without additional memory. Method. A method of forming the structure and modification of sparse distributed memory for storing previous transitions of the actor in the form of prototypes is proposed. The method allows increasing the informativeness of the stored data and, as a result, to improve the process of creating a model of the studied process by intensifying the learning of the deep neural network. Increasing the informativeness of the stored data is the result of this sequence of actions. First, we compare the new transition and the last saved transition. To perform this comparison, this method introduces a rate estimate for the distance between transitions. If the distance between the new transition and the last saved transition is smaller than the specified threshold, the new transition is written in place of the previous one without increasing the amount of memory. Otherwise, we create a new prototype in memory while deleting the prototype that has been stored in memory the longest. Results. The work of the proposed method was studied during the solution of the popular “Water World” test problem. The results showed a 1.5-times increase in the actor’s survival time in a hostile environment. This result was achieved by increasing the informativeness of the stored data without increasing the amount of RAM. Conclusions. The proposed method of forming and modifying the structure of sparse distributed memory allowed to increase the informativeness of the stored data. As a result of this approach, improved reinforcement learning parameters on the example of the “Water World” problem by increasing the accuracy of the model of the physical process represented by a deep neural network.

Cognitive function synthesis: preliminary results

Purpose This paper aims to introduce the cognitive function synthesis (CFS) conceptual framework to artificial general intelligence. CFS posits that at the “core” of intelligence in hybrid architectures, “interdependent” cognitive functions are synthesised through the interaction of various associative memory (AM)-based systems. This synthesis could form an interface layer between deliberative/symbolic and reactive/sub-symbolic layers in hybrid cognitive architectures. Design/methodology/approach A CFS conceptual framework, specifying an arrangement of AMs, was presented. The framework was executed using sparse distributed memory. Experiments were performed to investigate CFS autonomous extraction, consciousness and imagination. Findings Autonomous extraction was achieved using data from a Wi-Fi camera with the CFS auto-associative AM handling “Sensor Data”. However, noise reduction degraded the extracted image. An environment, simulated in V-REP 3.3.1, was used to investigate consciousness and imagination. CFS displayed consciousness by successfully tracking/anticipating the object position with over 90 per cent congruence. CFS imagination was seen by its predicting two time steps into the future. Originality/value Preliminary results demonstrate the plausibility of CFS claims for autonomous extraction, consciousness and imagination.