Location- and Person-Independent Activity Recognition with WiFi, Deep Neural Networks, and Reinforcement Learning

2021 ◽  
Vol 2 (1) ◽  
pp. 1-25
Author(s):  
Yongsen Ma ◽  
Sheheryar Arshad ◽  
Swetha Muniraju ◽  
Eric Torkildson ◽  
Enrico Rantala ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Reinforcement Learning ◽  
Activity Recognition ◽  
Deep Neural Networks ◽  
State Machine ◽  
Recognition Algorithm ◽  
The State ◽  
Neural Architecture ◽  
Learning Agent

In recent years, Channel State Information (CSI) measured by WiFi is widely used for human activity recognition. In this article, we propose a deep learning design for location- and person-independent activity recognition with WiFi. The proposed design consists of three Deep Neural Networks (DNNs): a 2D Convolutional Neural Network (CNN) as the recognition algorithm, a 1D CNN as the state machine, and a reinforcement learning agent for neural architecture search. The recognition algorithm learns location- and person-independent features from different perspectives of CSI data. The state machine learns temporal dependency information from history classification results. The reinforcement learning agent optimizes the neural architecture of the recognition algorithm using a Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM). The proposed design is evaluated in a lab environment with different WiFi device locations, antenna orientations, sitting/standing/walking locations/orientations, and multiple persons. The proposed design has 97% average accuracy when testing devices and persons are not seen during training. The proposed design is also evaluated by two public datasets with accuracy of 80% and 83%. The proposed design needs very little human efforts for ground truth labeling, feature engineering, signal processing, and tuning of learning parameters and hyperparameters.

scholarly journals Hybrid Reinforcement Learning with Expert State Sequences

2019 ◽  
Vol 33 ◽  
pp. 3739-3746 ◽  
Author(s):  
Xiaoxiao Guo ◽  
Shiyu Chang ◽  
Mo Yu ◽  
Gerald Tesauro ◽  
Murray Campbell
Keyword(s):  
Neural Networks ◽  
Reinforcement Learning ◽  
Deep Neural Networks ◽  
Hybrid Approach ◽  
Imitation Learning ◽  
Learning Approaches ◽  
Inference Model ◽  
Learning Agent ◽  
Hybrid Reinforcement ◽  
Policy Optimization

Existing imitation learning approaches often require that the complete demonstration data, including sequences of actions and states, are available. In this paper, we consider a more realistic and difficult scenario where a reinforcement learning agent only has access to the state sequences of an expert, while the expert actions are unobserved. We propose a novel tensor-based model to infer the unobserved actions of the expert state sequences. The policy of the agent is then optimized via a hybrid objective combining reinforcement learning and imitation learning. We evaluated our hybrid approach on an illustrative domain and Atari games. The empirical results show that (1) the agents are able to leverage state expert sequences to learn faster than pure reinforcement learning baselines, (2) our tensor-based action inference model is advantageous compared to standard deep neural networks in inferring expert actions, and (3) the hybrid policy optimization objective is robust against noise in expert state sequences.

scholarly journals Utilizing reinforcement learning and deep neural networks to optimize non-pharmaceutical COVID-19 interventions in Florida

2021 ◽  
Vol 10 (3) ◽  
Author(s):  
Megan Yang ◽  
Leya Joykutty
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Reinforcement Learning ◽  
Deep Neural Networks ◽  
The Neural Network ◽  
Long Term Effect ◽  
Training Samples ◽  
Economy And Education

Under the umbrella of artificial intelligence is machine learning that allows a system to improve through experience without any explicit programs telling it to. It is able to find patterns in massive amounts of data from works, images, numbers, to statistics. One approach to machine learning is neural networks in which the computer learns to finish a task by analyzing training samples. Another approach used in this study is reinforcement learning which manipulates it environment to discover errors and rewards.      This study aimed developed a deep neural network and used reinforcement learning to develop a system that was able to predict whether the cases will increase or decrease, then using that information, was able to predict which actions would most effectively cause a decline in cases while keeping things like economy and education in mind for a better long term effect. These models were made based on Florida using eight different counties’ data including things like mobility, temperature, dates of government actions, etc. Based on this information, data exploration and feature engineering was conducted to add dimensions that would further the accuracy of the neural network. The reinforcement learning model’s actions consisted of first, a shutdown for about two months before reopening schools and allowing things to return to normal. Then interestingly the model decided to keep school operating in a hybrid model with some students going back to school while others continue to study remotely.   

scholarly journals Neural Fourier Energy Disaggregation

Sensors ◽  
2022 ◽  
Vol 22 (2) ◽  
pp. 473
Author(s):  
Christoforos Nalmpantis ◽  
Nikolaos Virtsionis Gkalinikis ◽  
Dimitris Vrakas
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Fourier Transformation ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Limited Resources ◽  
Learning Models ◽  
Neural Architecture ◽  
Energy Disaggregation ◽  
Computationally Expensive

Deploying energy disaggregation models in the real-world is a challenging task. These models are usually deep neural networks and can be costly when running on a server or prohibitive when the target device has limited resources. Deep learning models are usually computationally expensive and they have large storage requirements. Reducing the computational cost and the size of a neural network, without trading off any performance is not a trivial task. This paper suggests a novel neural architecture that has less learning parameters, smaller size and fast inference time without trading off performance. The proposed architecture performs on par with two popular strong baseline models. The key characteristic is the Fourier transformation which has no learning parameters and it can be computed efficiently.

scholarly journals Controlling system based on neural networks with reinforcement learning for robotic manipulator

2020 ◽  
pp. 24-32
Author(s):  
Elena Solovyeva ◽  
Ali Abdullah
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Reinforcement Learning ◽  
Deep Neural Networks ◽  
Industrial Robot ◽  
Robotic Manipulator ◽  
Learning Method ◽  
Numerical Function ◽  
Continuous Space ◽  
Manipulator Robot

Introduction: Due to its advantages, such as high flexibility and the ability to move heavy pieces with high torques and forces, the robotic arm, also named manipulator robot, is the most used industrial robot. Purpose: We improve the controlling quality of a manipulator robot with seven degrees of freedom in the V-REP program's environment using the reinforcement learning method based on deep neural networks. Methods: Estimate the action signal's policy by building a numerical algorithm using deep neural networks. The action-network sends the action's signal to the robotic manipulator, and the critic-network performs a numerical function approximation to calculate the value function (Q-value). Results: We create a model of the robot and the environment using the reinforcement-learning library in MATLAB and connecting the output signals (the action's signal) to a simulated robot in V-REP program. Train the robot to reach an object in its workspace after interacting with the environment and calculating the reward of such interaction. The model of the observations was done using three vision sensors. Based on the proposed deep learning method, a model of an agent representing the robotic manipulator was built using four layers neural network for the actor with four layers neural network for the critic. The agent's model representing the robotic manipulator was trained for several hours until the robot started to reach the object in its workspace in an acceptable way. The main advantage over supervised learning control is allowing our robot to perform actions and train at the same moment, giving the robot the ability to reach an object in its workspace in a continuous space action. Practical relevance: The results obtained are used to control the behavior of the movement of the manipulator without the need to construct kinematic models, which reduce the mathematical complexity of the calculation and provide a universal solution.

scholarly journals DARB: A Density-Adaptive Regular-Block Pruning for Deep Neural Networks

2020 ◽  
Vol 34 (04) ◽  
pp. 5495-5502
Author(s):  
Ren Ao ◽  
Zhang Tao ◽  
Wang Yuhao ◽  
Lin Sheng ◽  
Dong Peiyan ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Structural Characteristics ◽  
The State ◽  
Neural Network Models ◽  
Model Compression ◽  
High Regularity ◽  
Decoding Efficiency

The rapidly growing parameter volume of deep neural networks (DNNs) hinders the artificial intelligence applications on resource constrained devices, such as mobile and wearable devices. Neural network pruning, as one of the mainstream model compression techniques, is under extensive study to reduce the model size and thus the amount of computation. And thereby, the state-of-the-art DNNs are able to be deployed on those devices with high runtime energy efficiency. In contrast to irregular pruning that incurs high index storage and decoding overhead, structured pruning techniques have been proposed as the promising solutions. However, prior studies on structured pruning tackle the problem mainly from the perspective of facilitating hardware implementation, without diving into the deep to analyze the characteristics of sparse neural networks. The neglect on the study of sparse neural networks causes inefficient trade-off between regularity and pruning ratio. Consequently, the potential of structurally pruning neural networks is not sufficiently mined.In this work, we examine the structural characteristics of the irregularly pruned weight matrices, such as the diverse redundancy of different rows, the sensitivity of different rows to pruning, and the position characteristics of retained weights. By leveraging the gained insights as a guidance, we first propose the novel block-max weight masking (BMWM) method, which can effectively retain the salient weights while imposing high regularity to the weight matrix. As a further optimization, we propose a density-adaptive regular-block (DARB) pruning that can effectively take advantage of the intrinsic characteristics of neural networks, and thereby outperform prior structured pruning work with high pruning ratio and decoding efficiency. Our experimental results show that DARB can achieve 13× to 25× pruning ratio, which are 2.8× to 4.3× improvements than the state-of-the-art counterparts on multiple neural network models and tasks. Moreover, DARB can achieve 14.3× decoding efficiency than block pruning with higher pruning ratio.

scholarly journals Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Florian Stelzer ◽  
André Röhm ◽  
Raul Vicente ◽  
Ingo Fischer ◽  
Serhiy Yanchuk
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Single Neuron ◽  
Deep Neural Networks ◽  
Back Propagation ◽  
Local Network ◽  
Multiple Time ◽  
Learning Tools ◽  
Back Propagation Algorithm

AbstractDeep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron’s dynamics. By adjusting the feedback-modulation within the loops, we adapt the network’s connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

scholarly journals An efficient pruning scheme of deep neural networks for Internet of Things applications

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Chen Qi ◽  
Shibo Shen ◽  
Rongpeng Li ◽  
Zhifeng Zhao ◽  
Qing Liu ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Internet Of Things ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Superior Performance ◽  
Compact Structure ◽  
Resource Limited ◽  
Benchmark Datasets ◽  
Iot Devices

AbstractNowadays, deep neural networks (DNNs) have been rapidly deployed to realize a number of functionalities like sensing, imaging, classification, recognition, etc. However, the computational-intensive requirement of DNNs makes it difficult to be applicable for resource-limited Internet of Things (IoT) devices. In this paper, we propose a novel pruning-based paradigm that aims to reduce the computational cost of DNNs, by uncovering a more compact structure and learning the effective weights therein, on the basis of not compromising the expressive capability of DNNs. In particular, our algorithm can achieve efficient end-to-end training that transfers a redundant neural network to a compact one with a specifically targeted compression rate directly. We comprehensively evaluate our approach on various representative benchmark datasets and compared with typical advanced convolutional neural network (CNN) architectures. The experimental results verify the superior performance and robust effectiveness of our scheme. For example, when pruning VGG on CIFAR-10, our proposed scheme is able to significantly reduce its FLOPs (floating-point operations) and number of parameters with a proportion of 76.2% and 94.1%, respectively, while still maintaining a satisfactory accuracy. To sum up, our scheme could facilitate the integration of DNNs into the common machine-learning-based IoT framework and establish distributed training of neural networks in both cloud and edge.

scholarly journals Diversity oriented Deep Reinforcement Learning for targeted molecule generation

2021 ◽  
Vol 13 (1) ◽  
Author(s):  
Tiago Pereira ◽  
Maryam Abbasi ◽  
Bernardete Ribeiro ◽  
Joel P. Arrais
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Reinforcement Learning ◽  
Deep Neural Networks ◽  
Chemical Space ◽  
Biological Properties ◽  
Training Process ◽  
Training Strategy ◽  
Inhibitory Power ◽  
Exploratory Strategy

AbstractIn this work, we explore the potential of deep learning to streamline the process of identifying new potential drugs through the computational generation of molecules with interesting biological properties. Two deep neural networks compose our targeted generation framework: the Generator, which is trained to learn the building rules of valid molecules employing SMILES strings notation, and the Predictor which evaluates the newly generated compounds by predicting their affinity for the desired target. Then, the Generator is optimized through Reinforcement Learning to produce molecules with bespoken properties. The innovation of this approach is the exploratory strategy applied during the reinforcement training process that seeks to add novelty to the generated compounds. This training strategy employs two Generators interchangeably to sample new SMILES: the initially trained model that will remain fixed and a copy of the previous one that will be updated during the training to uncover the most promising molecules. The evolution of the reward assigned by the Predictor determines how often each one is employed to select the next token of the molecule. This strategy establishes a compromise between the need to acquire more information about the chemical space and the need to sample new molecules, with the experience gained so far. To demonstrate the effectiveness of the method, the Generator is trained to design molecules with an optimized coefficient of partition and also high inhibitory power against the Adenosine $$A_{2A}$$ A 2 A and $$\kappa$$ κ opioid receptors. The results reveal that the model can effectively adjust the newly generated molecules towards the wanted direction. More importantly, it was possible to find promising sets of unique and diverse molecules, which was the main purpose of the newly implemented strategy.

scholarly journals Reynolds averaged turbulence modelling using deep neural networks with embedded invariance

2016 ◽  
Vol 807 ◽  
pp. 155-166 ◽  
Author(s):  
Julia Ling ◽  
Andrew Kurzawski ◽  
Jeremy Templeton
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Reynolds Stress ◽  
Network Architecture ◽  
Eddy Viscosity ◽  
Deep Neural Networks ◽  
Test Cases ◽  
Neural Network Architecture ◽  
Stress Anisotropy ◽  
Anisotropy Tensor

There exists significant demand for improved Reynolds-averaged Navier–Stokes (RANS) turbulence models that are informed by and can represent a richer set of turbulence physics. This paper presents a method of using deep neural networks to learn a model for the Reynolds stress anisotropy tensor from high-fidelity simulation data. A novel neural network architecture is proposed which uses a multiplicative layer with an invariant tensor basis to embed Galilean invariance into the predicted anisotropy tensor. It is demonstrated that this neural network architecture provides improved prediction accuracy compared with a generic neural network architecture that does not embed this invariance property. The Reynolds stress anisotropy predictions of this invariant neural network are propagated through to the velocity field for two test cases. For both test cases, significant improvement versus baseline RANS linear eddy viscosity and nonlinear eddy viscosity models is demonstrated.

Inclusion of Multiple Cycling of the Potential into Deep Neural Network Classification of Voltammetric Reaction Mechanisms

2021 ◽  
Author(s):  
Luke Gundry ◽  
Gareth Kennedy ◽  
Alan Bond ◽  
Jie Zhang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Reaction Mechanisms ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Neural Network Classification ◽  
Initial Cycle

The use of Deep Neural Networks (DNNs) for the classification of electrochemical mechanisms based on training with simulations of the initial cycle of potential have been reported. In this paper,...

Sign in / Sign up

Export Citation Format

Share Document