scholarly journals GateRL: Automated Circuit Design Framework of CMOS Logic Gates Using Reinforcement Learning

Electronics ◽  
2021 ◽  
Vol 10 (9) ◽  
pp. 1032
Author(s):  
Hyoungsik Nam ◽  
Young In Kim ◽  
Jina Bae ◽  
Junhee Lee
Keyword(s):  
Reinforcement Learning ◽  
Circuit Design ◽  
Logic Gates ◽  
Connection Matrix ◽  
Design Framework ◽  
Network Layers ◽  
Learning Speed ◽  
Fully Connected ◽  
Automated Circuit Design ◽  
Circuit Configuration

This paper proposes a GateRL that is an automated circuit design framework of CMOS logic gates based on reinforcement learning. Because there are constraints in the connection of circuit elements, the action masking scheme is employed. It also reduces the size of the action space leading to the improvement on the learning speed. The GateRL consists of an agent for the action and an environment for state, mask, and reward. State and reward are generated from a connection matrix that describes the current circuit configuration, and the mask is obtained from a masking matrix based on constraints and current connection matrix. The action is given rise to by the deep Q-network of 4 fully connected network layers in the agent. In particular, separate replay buffers are devised for success transitions and failure transitions to expedite the training process. The proposed network is trained with 2 inputs, 1 output, 2 NMOS transistors, and 2 PMOS transistors to design all the target logic gates, such as buffer, inverter, AND, OR, NAND, and NOR. Consequently, the GateRL outputs one-transistor buffer, two-transistor inverter, two-transistor AND, two-transistor OR, three-transistor NAND, and three-transistor NOR. The operations of these resultant logics are verified by the SPICE simulation.

scholarly journals Reinforcement Learning using Convolutional Neural Network for Game Prediction

2020 ◽  
Vol 9 (8) ◽  
pp. 425-430
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Computer Games ◽  
Elevated Level ◽  
Q Learning ◽  
The Third ◽  
Level Information ◽  
Simple Neural Network ◽  
Fully Connected ◽  
Deep Learning Model

The paper presents a Deep learning model for playing computer games with elevated level information utilizing Reinforcement learning learning. The games are activity restricted (like snakes, catcher, air-bandit and so on.). The implementation is progressive in three parts. The first part deals with a simple neural network, the second one with Deep Q network and further to increase the accuracy and speed of the algorithm, the third part consists of a model consisting of convolution neural network for image processing and giving outputs from the fully connected layers so as to estimate the probability of an action being taken based on information extracted from inputs where we apply Q-learning to determine the best possible move. The results are further analysed and compared to provide an overview of the improvements in each methods.

scholarly journals Large-Scale Person Re-Identification Based on Deep Hash Learning

Entropy ◽  
2019 ◽  
Vol 21 (5) ◽  
pp. 449 ◽  
Author(s):  
Xian-Qin Ma ◽  
Chong-Chong Yu ◽  
Xiu-Xin Chen ◽  
Lan Zhou
Keyword(s):  
Hash Function ◽  
Large Scale ◽  
Research Topic ◽  
Cross Entropy ◽  
Network Layers ◽  
Hash Code ◽  
Image Hash ◽  
Challenging Research ◽  
Fully Connected ◽  
Relationship Of

Person re-identification in the image processing domain has been a challenging research topic due to the influence of pedestrian posture, background, lighting, and other factors. In this paper, the method of harsh learning is applied in person re-identification, and we propose a person re-identification method based on deep hash learning. By improving the conventional method, the method proposed in this paper uses an easy-to-optimize shallow convolutional neural network to learn the inherent implicit relationship of the image and then extracts the deep features of the image. Then, a hash layer with three-step calculation is incorporated in the fully connected layer of the network. The hash function is learned and mapped into a hash code through the connection between the network layers. The generation of the hash code satisfies the requirements that minimize the error of the sum of quantization loss and Softmax regression cross-entropy loss, which achieve the end-to-end generation of hash code in the network. After obtaining the hash code through the network, the distance between the pedestrian image hash code to be retrieved and the pedestrian image hash code library is calculated to implement the person re-identification. Experiments conducted on multiple standard datasets show that our deep hashing network achieves the comparable performances and outperforms other hashing methods with large margins on Rank-1 and mAP value identification rates in pedestrian re-identification. Besides, our method is predominant in the efficiency of training and retrieval in contrast to other pedestrian re-identification algorithms.

scholarly journals Symmetric stacked fast binary counters based on reversible logic

2018 ◽  
Vol 7 (4) ◽  
pp. 2747
Author(s):  
C Santhi ◽  
Dr. Moparthy Gurunadha Babu
Keyword(s):  
Energy Dissipation ◽  
Circuit Design ◽  
Power Dissipation ◽  
Heat Dissipation ◽  
Logic Gates ◽  
Digital Circuit ◽  
Information Loss ◽  
Reversible Logic ◽  
Binary Counter ◽  
Reversible Logic Gates

A Symmetric Stacked Fast Binary counter design is proposed in this paper. In the circuit design, the first phase is occupied by 3-bit stacking circuits, which are further followed by combining circuits. The resultant novel circuit thus becomes a 6-bit stacker. A 6:3 counter has been chosen as an example to demonstrate the working of the proposed circuit. The proposed circuit is further implemented by using reversible logic gates. Heat dissipation is a major problem in the designing of a digital circuit. Rolf Landauer has proved that the information loss in a digital circuit is directly proportional to the energy dissipation. The proposed modified Symmetric Stacking counter is implemented using reversible logic gates thus reducing the power dissipation of the circuit. 

Acceleration of Reinforcement Learning with Incomplete Prior Information

2013 ◽  
Vol 17 (5) ◽  
pp. 721-730 ◽  
Author(s):  
Kento Terashima ◽  
◽  
Hirotaka Takano ◽  
Junichi Murata
Keyword(s):  
Reinforcement Learning ◽  
Prior Information ◽  
Search Process ◽  
Trial And Error ◽  
Calculation Time ◽  
Learning Speed ◽  
Incomplete Prior Information

Reinforcement learning is applicable to complex or unknown problems because the solution search process is done by trial-and-error. However, the calculation time for the trial-and-error search becomes larger as the scale of the problem increases. Therefore, in order to decrease calculation time, some methods have been proposed using the prior information on the problem. This paper improves a previously proposed method utilizing options as prior information. In order to increase the learning speed even with wrong options, methods for option correction by forgetting the policy and extending initiation sets are proposed.

Function Expansion of a Chaotic Logic Unit

2010 ◽  
Vol 171-172 ◽  
pp. 283-287
Author(s):  
Yi Yan Sheng ◽  
Wen Bo Liu
Keyword(s):  
Numerical Model ◽  
Circuit Design ◽  
Simulation Analysis ◽  
Circuit Simulation ◽  
Logic Gate ◽  
Logic Gates ◽  
Control Voltage ◽  
Computing Unit ◽  
Nor Gate ◽  
The Impact

Chaos computing is a new circuit design scheme of using chaos computing units to achieve reconfigurable logic gates. The computing unit can function as different kinds of logic gates by changing external parameters. In this paper, the possibilities of expanding the function of a chaotic NOR gate proposed in the literature is studied. The numerical model for the circuit design was built by constructing differential equations fit for Matlab integration mechanism. Besides, numerical model for integrator saturation was built to make results of numerical simulation conform to that of circuit simulation. Analysis of the impact of integrator saturation was done. With the analysis and by changing the control voltage, NAND function was expanded for the original chaotic logic gate that was only able to function as a NOR gate. By adding the function control signal to the input end and setting the voltage of it to different levels, the computing unit becomes a real time reconfigurable one.

scholarly journals STOCHASTIC PSEUDOSPIN NEURAL NETWORK WITH TRIDIAGONAL SYNAPTIC CONNECTIONS

2021 ◽  
pp. 114-122
Author(s):  
R. М. Peleshchak ◽  
V. V. Lytvyn ◽  
О. І. Cherniak ◽  
І. R. Peleshchak ◽  
М. V. Doroshenko
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Connection Matrix ◽  
Synaptic Connections ◽  
Hessenberg Matrix ◽  
Tuning Time ◽  
Hebb Rule ◽  
The Matrix ◽  
Dipole Glass ◽  
Fully Connected

Context. To reduce the computational resource time in the problems of diagnosing and recognizing distorted images based on a fully connected stochastic pseudospin neural network, it becomes necessary to thin out synaptic connections between neurons, which is solved using the method of diagonalizing the matrix of synaptic connections without losing interaction between all neurons in the network. Objective. To create an architecture of a stochastic pseudo-spin neural network with diagonal synaptic connections without loosing the interaction between all the neurons in the layer to reduce its learning time. Method. The paper uses the Hausholder method, the method of compressing input images based on the diagonalization of the matrix of synaptic connections and the computer mathematics system MATLAB for converting a fully connected neural network into a tridiagonal form with hidden synaptic connections between all neurons. Results. We developed a model of a stochastic neural network architecture with sparse renormalized synaptic connections that take into account deleted synaptic connections. Based on the transformation of the synaptic connection matrix of a fully connected neural network into a Hessenberg matrix with tridiagonal synaptic connections, we proposed a renormalized local Hebb rule. Using the computer mathematics system “WolframMathematica 11.3”, we calculated, as a function of the number of neurons N, the relative tuning time of synaptic connections (per iteration) in a stochastic pseudospin neural network with a tridiagonal connection Matrix, relative to the tuning time of synaptic connections (per iteration) in a fully connected synaptic neural network. Conclusions. We found that with an increase in the number of neurons, the tuning time of synaptic connections (per iteration) in a stochastic pseudospin neural network with a tridiagonal connection Matrix, relative to the tuning time of synaptic connections (per iteration) in a fully connected synaptic neural network, decreases according to a hyperbolic law. Depending on the direction of pseudospin neurons, we proposed a classification of a renormalized neural network with a ferromagnetic structure, an antiferromagnetic structure, and a dipole glass.

scholarly journals Pre-training with non-expert human demonstration for deep reinforcement learning

2019 ◽  
Vol 34 ◽  
Author(s):  
Gabriel V. de la Cruz ◽  
Yunshu Du ◽  
Matthew E. Taylor
Keyword(s):  
Reinforcement Learning ◽  
Feature Learning ◽  
Feature Representation ◽  
Superior Performance ◽  
Learning Goals ◽  
Training Time ◽  
Learning Speed ◽  
Small Set ◽  
Data Efficiency ◽  
Human Demonstration

Abstract Deep reinforcement learning (deep RL) has achieved superior performance in complex sequential tasks by using deep neural networks as function approximators to learn directly from raw input images. However, learning directly from raw images is data inefficient. The agent must learn feature representation of complex states in addition to learning a policy. As a result, deep RL typically suffers from slow learning speeds and often requires a prohibitively large amount of training time and data to reach reasonable performance, making it inapplicable to real-world settings where data are expensive. In this work, we improve data efficiency in deep RL by addressing one of the two learning goals, feature learning. We leverage supervised learning to pre-train on a small set of non-expert human demonstrations and empirically evaluate our approach using the asynchronous advantage actor-critic algorithms in the Atari domain. Our results show significant improvements in learning speed, even when the provided demonstration is noisy and of low quality.

Sign in / Sign up

Export Citation Format

Share Document