DeepWeave: Accelerating Job Completion Time with Deep Reinforcement Learning-based Coflow Scheduling

To improve the processing efficiency of jobs in distributed computing, the concept of coflow is proposed. A coflow is a collection of flows that are semantically correlated in a multi-stage computation task. A job consists of multiple coflows and can be usually formulated as a Directed-Acyclic Graph (DAG). A proper scheduling of coflows can significantly reduce the completion time of jobs in distributed computing. However, this scheduling problem is proved to be NP-hard. Different from existing schemes that use hand-crafted heuristic algorithms to solve this problem, in this paper, we propose a Deep Reinforcement Learning (DRL) framework named DeepWeave to generate coflow scheduling policies. To improve the inter-coflow scheduling ability in the job DAG, DeepWeave employs a Graph Neural Network (GNN) to process the DAG information. DeepWeave learns from the history workload trace to train the neural networks of the DRL agent and encodes the scheduling policy in the neural networks, which make coflow scheduling decisions without expert knowledge or a pre-assumed model. The proposed scheme is evaluated with a simulator using real-life traces. Simulation results show that DeepWeave completes jobs at least 1.7X faster than the state-of-the-art solutions.

Download Full-text

A Task of Miniature Mobile Robot Learning for Obstacle Avoidance through Neural Networks

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.151.498 ◽

2012 ◽

Vol 151 ◽

pp. 498-502

Author(s):

Jin Xue Zhang ◽

Hai Zhu Pan

Keyword(s):

Neural Networks ◽

Reinforcement Learning ◽

Mobile Robot ◽

Obstacle Avoidance ◽

Robot Learning ◽

Q Learning ◽

Robot Systems ◽

The Neural Networks ◽

Behavior Based

This paper is concerned with Q-learning , a very popular algorithm for reinforcement learning ,for obstacle avoidance through neural networks. The principle tells that the focus always must be on both ecological nice tasks and behaviours when designing on robot. Many robot systems have used behavior-based systems since the 1980’s.In this paper, the Khepera robot is trained through the proposed algorithm of Q-learning using the neural networks for the task of obstacle avoidance. In experiments with real and simulated robots, the neural networks approach can be used to make it possible for Q-learning to handle changes in the environment.

Download Full-text

Analysis of the ReSuMe Learning Process For Spiking Neural Networks

International Journal of Applied Mathematics and Computer Science ◽

10.2478/v10006-008-0011-1 ◽

2008 ◽

Vol 18 (2) ◽

pp. 117-127 ◽

Cited By ~ 16

Author(s):

Filip Ponulak

Keyword(s):

Neural Networks ◽

Learning Process ◽

Learning Algorithm ◽

Real Life ◽

Optimal Solution ◽

Spiking Neural Networks ◽

Fast Learning ◽

Life Tasks ◽

The Neural Networks ◽

The Stability

Analysis of the ReSuMe Learning Process For Spiking Neural NetworksIn this paper we perform an analysis of the learning process with the ReSuMe method and spiking neural networks (Ponulak, 2005; Ponulak, 2006b). We investigate how the particular parameters of the learning algorithm affect the process of learning. We consider the issue of speeding up the adaptation process, while maintaining the stability of the optimal solution. This is an important issue in many real-life tasks where the neural networks are applied and where the fast learning convergence is highly desirable.

Download Full-text

Actor-Critic Traction Control Based on Reinforcement Learning with Open-Loop Training

Modelling and Simulation in Engineering ◽

10.1155/2021/4641450 ◽

2021 ◽

Vol 2021 ◽

pp. 1-10

Author(s):

M. Funk Drechsler ◽

T. A. Fiorentin ◽

H. Göllinger

Keyword(s):

Neural Networks ◽

Reinforcement Learning ◽

Autonomous Driving ◽

Stability Control ◽

Open Loop ◽

Training Process ◽

Slip Ratio ◽

Traction Control ◽

Antilock Braking System ◽

The Neural Networks

The use of actor-critic algorithms can improve the controllers currently implemented in automotive applications. This method combines reinforcement learning (RL) and neural networks to achieve the possibility of controlling nonlinear systems with real-time capabilities. Actor-critic algorithms were already applied with success in different controllers including autonomous driving, antilock braking system (ABS), and electronic stability control (ESC). However, in the current researches, virtual environments are implemented for the training process instead of using real plants to obtain the datasets. This limitation is given by trial and error methods implemented for the training process, which generates considerable risks in case the controller directly acts on the real plant. In this way, the present research proposes and evaluates an open-loop training process, which permits the data acquisition without the control interaction and an open-loop training of the neural networks. The performance of the trained controllers is evaluated by a design of experiments (DOE) to understand how it is affected by the generated dataset. The results present a successful application of open-loop training architecture. The controller can maintain the slip ratio under adequate levels during maneuvers on different floors, including grounds that are not applied during the training process. The actor neural network is also able to identify the different floors and change the acceleration profile according to the characteristics of each ground.

Download Full-text

Unmanned Aerial Vehicle Pitch Control under Delay Using Deep Reinforcement Learning with Continuous Action in Wind Tunnel Test

Aerospace ◽

10.3390/aerospace8090258 ◽

2021 ◽

Vol 8 (9) ◽

pp. 258

Author(s):

Daichi Wada ◽

Sergio A. Araujo-Estrada ◽

Shane Windsor

Keyword(s):

Neural Networks ◽

Reinforcement Learning ◽

Wind Tunnel ◽

Real World ◽

Time Delays ◽

Wind Tunnel Test ◽

The Real ◽

Pitch Control ◽

Controller Performance ◽

The Neural Networks

Nonlinear flight controllers for fixed-wing unmanned aerial vehicles (UAVs) can potentially be developed using deep reinforcement learning. However, there is often a reality gap between the simulation models used to train these controllers and the real world. This study experimentally investigated the application of deep reinforcement learning to the pitch control of a UAV in wind tunnel tests, with a particular focus of investigating the effect of time delays on flight controller performance. Multiple neural networks were trained in simulation with different assumed time delays and then wind tunnel tested. The neural networks trained with shorter delays tended to be susceptible to delay in the real tests and produce fluctuating behaviour. The neural networks trained with longer delays behaved more conservatively and did not produce oscillations but suffered steady state errors under some conditions due to unmodeled frictional effects. These results highlight the importance of performing physical experiments to validate controller performance and how the training approach used with reinforcement learning needs to be robust to reality gaps between simulation and the real world.

Download Full-text

Solar Energetic Particle Event occurrence prediction using Solar Flare Soft X-ray measurements and Machine Learning

Journal of Space Weather and Space Climate ◽

10.1051/swsc/2021043 ◽

2021 ◽

Author(s):

Sigiava Aminalragia-Giamini ◽

Savvas Raptis ◽

Anastasios Anastasiadis ◽

Antonis Tsigkanos ◽

Ingmar Sandberg ◽

...

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Solar Flares ◽

Energetic Particle ◽

Real Life ◽

Solar Energetic Particle ◽

Solar Energetic Particle Event ◽

X Rays ◽

The Neural Networks ◽

Event Occurrence

The prediction of the occurrence of Solar Energetic Particle (SEP) events has been investigated over many years and multiple works have presented significant advances in this problem. The accurate and timely prediction of SEPs is of interest to the scientific community as well as mission designers, operators, and industrial partners due to the threat SEPs pose to satellites, spacecrafts and crewed missions. In this work we present a methodology for the prediction of SEPs from the soft X-rays of solar flares associated with SEPs that were measured in 1 AU. We use an expansive dataset covering 25 years of solar activity, 1988-2013, which includes thousands of flares and more than two hundred identified and catalogued SEPs. Neural networks are employed as the predictors in the model providing probabilities for the occurrence or not of an SEP which are converted to yes/no predictions. The neural networks are designed using current and state-of the-art tools integrating recent advances in the machine learning field. The results of the methodology are extensively evaluated and validated using all the available data and it is shown that we achieve very good levels of accuracy with correct SEP occurrence prediction higher than 85% and correct no-SEP predictions higher than 92%. Finally we discuss further work towards potential improvements and the applicability of our model in real life conditions.

Download Full-text

A Comparative Study of Computational Intelligence Algorithms for Sensor Localization

International Journal of Sensors Wireless Communications and Control ◽

10.2174/2210327909666181206103304 ◽

2019 ◽

Vol 9 (2) ◽

pp. 224-236

Author(s):

Vaishali R. Kulkarni ◽

Veena Desai ◽

Raghavendra Kulkarni

Keyword(s):

Firefly Algorithm ◽

Heuristic Algorithms ◽

Computing Time ◽

Sensor Nodes ◽

Convergence Time ◽

Localization Error ◽

Location Awareness ◽

Cultural Algorithm ◽

Localization Accuracy ◽

Multi Stage

Background & Objective: Location of sensors is an important information in wireless sensor networks for monitoring, tracking and surveillance applications. The accurate and quick estimation of the location of sensor nodes plays an important role. Localization refers to creating location awareness for as many sensor nodes as possible. Multi-stage localization of sensor nodes using bio-inspired, heuristic algorithms is the central theme of this paper. Methodology: Biologically inspired heuristic algorithms offer the advantages of simplicity, resourceefficiency and speed. Four such algorithms have been evaluated in this paper for distributed localization of sensor nodes. Two evolutionary computation-based algorithms, namely cultural algorithm and the genetic algorithm, have been presented to optimize the localization process for minimizing the localization error. The results of these algorithms have been compared with those of swarm intelligence- based optimization algorithms, namely the firefly algorithm and the bee algorithm. Simulation results and analysis of stage-wise localization in terms of number of localized nodes, computing time and accuracy have been presented. The tradeoff between localization accuracy and speed has been investigated. Results: The comparative analysis shows that the firefly algorithm performs the localization in the most accurate manner but takes longest convergence time. Conclusion: Further, the cultural algorithm performs the localization in a very quick time; but, results in high localization error.

Download Full-text

Tax Fraud Detection through Neural Networks: An Application Using a Sample of Personal Income Taxpayers

Future Internet ◽

10.3390/fi11040086 ◽

2019 ◽

Vol 11 (4) ◽

pp. 86 ◽

Cited By ~ 2

Author(s):

César Pérez López ◽

María Delgado Rodríguez ◽

Sonia de Lucas Santos

Keyword(s):

Neural Network ◽

Neural Networks ◽

Income Tax ◽

Fraud Detection ◽

Personal Income ◽

Personal Income Tax ◽

Tax Returns ◽

The Neural Networks ◽

Tax Fraud ◽

Efficiency Rate

The goal of the present research is to contribute to the detection of tax fraud concerning personal income tax returns (IRPF, in Spanish) filed in Spain, through the use of Machine Learning advanced predictive tools, by applying Multilayer Perceptron neural network (MLP) models. The possibilities springing from these techniques have been applied to a broad range of personal income return data supplied by the Institute of Fiscal Studies (IEF). The use of the neural networks enabled taxpayer segmentation as well as calculation of the probability concerning an individual taxpayer’s propensity to attempt to evade taxes. The results showed that the selected model has an efficiency rate of 84.3%, implying an improvement in relation to other models utilized in tax fraud detection. The proposal can be generalized to quantify an individual’s propensity to commit fraud with regards to other kinds of taxes. These models will support tax offices to help them arrive at the best decisions regarding action plans to combat tax fraud.

Download Full-text

Evaluation of Mixed Deep Neural Networks for Reverberant Speech Enhancement

Biomimetics ◽

10.3390/biomimetics5010001 ◽

2019 ◽

Vol 5 (1) ◽

pp. 1 ◽

Cited By ~ 1

Author(s):

Michelle Gutiérrez-Muñoz ◽

Astryd González-Salazar ◽

Marvin Coto-Jiménez

Keyword(s):

Neural Networks ◽

Short Term Memory ◽

Computational Cost ◽

Real Life ◽

Fixed Number ◽

Training Procedure ◽

Statistical Validation ◽

Significant Drop ◽

Training Time ◽

Important Solution

Speech signals are degraded in real-life environments, as a product of background noise or other factors. The processing of such signals for voice recognition and voice analysis systems presents important challenges. One of the conditions that make adverse quality difficult to handle in those systems is reverberation, produced by sound wave reflections that travel from the source to the microphone in multiple directions. To enhance signals in such adverse conditions, several deep learning-based methods have been proposed and proven to be effective. Recently, recurrent neural networks, especially those with long short-term memory (LSTM), have presented surprising results in tasks related to time-dependent processing of signals, such as speech. One of the most challenging aspects of LSTM networks is the high computational cost of the training procedure, which has limited extended experimentation in several cases. In this work, we present a proposal to evaluate the hybrid models of neural networks to learn different reverberation conditions without any previous information. The results show that some combinations of LSTM and perceptron layers produce good results in comparison to those from pure LSTM networks, given a fixed number of layers. The evaluation was made based on quality measurements of the signal’s spectrum, the training time of the networks, and statistical validation of results. In total, 120 artificial neural networks of eight different types were trained and compared. The results help to affirm the fact that hybrid networks represent an important solution for speech signal enhancement, given that reduction in training time is on the order of 30%, in processes that can normally take several days or weeks, depending on the amount of data. The results also present advantages in efficiency, but without a significant drop in quality.

Download Full-text

Automatic Detection of Arrhythmia Based on Multi-Resolution Representation of ECG Signal

Sensors ◽

10.3390/s20061579 ◽

2020 ◽

Vol 20 (6) ◽

pp. 1579

Author(s):

Dongqi Wang ◽

Qinghua Meng ◽

Dongming Chen ◽

Hupo Zhang ◽

Lisheng Xu

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Deep Neural Networks ◽

Channel Model ◽

Expert Knowledge ◽

Automatic Detection ◽

Data Representation ◽

Learning Technology ◽

Arrhythmia Detection ◽

Automatic Feature Extraction

Automatic detection of arrhythmia is of great significance for early prevention and diagnosis of cardiovascular disease. Traditional feature engineering methods based on expert knowledge lack multidimensional and multi-view information abstraction and data representation ability, so the traditional research on pattern recognition of arrhythmia detection cannot achieve satisfactory results. Recently, with the increase of deep learning technology, automatic feature extraction of ECG data based on deep neural networks has been widely discussed. In order to utilize the complementary strength between different schemes, in this paper, we propose an arrhythmia detection method based on the multi-resolution representation (MRR) of ECG signals. This method utilizes four different up to date deep neural networks as four channel models for ECG vector representations learning. The deep learning based representations, together with hand-crafted features of ECG, forms the MRR, which is the input of the downstream classification strategy. The experimental results of big ECG dataset multi-label classification confirm that the F1 score of the proposed method is 0.9238, which is 1.31%, 0.62%, 1.18% and 0.6% higher than that of each channel model. From the perspective of architecture, this proposed method is highly scalable and can be employed as an example for arrhythmia recognition.

Download Full-text

Location- and Person-Independent Activity Recognition with WiFi, Deep Neural Networks, and Reinforcement Learning

ACM Transactions on Internet of Things ◽

10.1145/3424739 ◽

2021 ◽

Vol 2 (1) ◽

pp. 1-25

Author(s):

Yongsen Ma ◽

Sheheryar Arshad ◽

Swetha Muniraju ◽

Eric Torkildson ◽

Enrico Rantala ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Reinforcement Learning ◽

Activity Recognition ◽

Deep Neural Networks ◽

State Machine ◽

Recognition Algorithm ◽

The State ◽

Neural Architecture ◽

Learning Agent

In recent years, Channel State Information (CSI) measured by WiFi is widely used for human activity recognition. In this article, we propose a deep learning design for location- and person-independent activity recognition with WiFi. The proposed design consists of three Deep Neural Networks (DNNs): a 2D Convolutional Neural Network (CNN) as the recognition algorithm, a 1D CNN as the state machine, and a reinforcement learning agent for neural architecture search. The recognition algorithm learns location- and person-independent features from different perspectives of CSI data. The state machine learns temporal dependency information from history classification results. The reinforcement learning agent optimizes the neural architecture of the recognition algorithm using a Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM). The proposed design is evaluated in a lab environment with different WiFi device locations, antenna orientations, sitting/standing/walking locations/orientations, and multiple persons. The proposed design has 97% average accuracy when testing devices and persons are not seen during training. The proposed design is also evaluated by two public datasets with accuracy of 80% and 83%. The proposed design needs very little human efforts for ground truth labeling, feature engineering, signal processing, and tuning of learning parameters and hyperparameters.

Download Full-text