Smart SDN Management of Fog Services

10.36227/techrxiv.11640162 ◽

2020 ◽

Author(s):

Piotr Frohlich ◽

Erol Gelenbe ◽

Mateusz P. Nowak

Keyword(s):

Machine Learning ◽

Reinforcement Learning ◽

Measurement Data ◽

Experimental Results ◽

Time Varying ◽

Test Bed ◽

Iot Devices ◽

Smart Service ◽

Service Manager ◽

Sdn Controller

We present a smart Service Manager whose role is to direct user requests (such as those coming from IoT devices) at the edge towards appropriate servers where the services they request can be satisfied, when services can be housed at different Fog locations, and the system is subject to variations in workload. The approach we propose is based on using an SDN controller as a decision element, and to incorporate measurement data based machine learning that uses Reinforcement Learning to make the best choices. The system we have developed is illustrated with experimental results on a test-bed in the presence of time-varying loads at the servers. The experiments confirm the ability of the system to adapt to significant changes in system load so as to preserve the QoS perceived by end users.

Download Full-text

Application of deep reinforcement learning for battery design

10.32469/10355/86552 ◽

2020 ◽

Author(s):

◽

Dongpeng Liu

Keyword(s):

Machine Learning ◽

Reinforcement Learning ◽

Prediction Model ◽

Measurement Data ◽

Material Design ◽

Training Data ◽

Machine Learning Techniques ◽

Material Research ◽

Neural Network Models ◽

Policy Gradient

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] The conventional material research and development are mainly driven by human intuition, labor, and manual decision. It is ineffective and inefficient. Due to the complexity of material design and the magnitude of experimental and computational work, the discovery of materials with conventional methods usually takes very long development cycles (10-20 years) with enormous labor and costs. To address this challenge, we proposed a machine-learning framework called Material Artificial Intelligence Robotics-driven System (MARS), aiming to reduce the costs with the help of machine learning techniques. We applied advanced deep-learning networks to better predict conductivity. We explored neural network models and tree-based models such as LightGBM. In particular, we made the models more interpretable and identified the relationships between the electrolyte's composition and the ionic conductivity. To search for the optimal conductivity, we developed a sophisticated deep reinforcement learning (RL) model called DDPG (Deep Deterministic Policy Gradient) to explore novel recipes to reach much higher conductivity. DDPG begins the RL process by entering new states through actions, where each action at a specific state (with a one-hot vector, representing selections of electrolyte components) would yield a reward Q, trained by the predictor developed in the previous step. After the optimal compositions have been found for the maximum conductivity, voltage stability and modulus, new measurements would be conducted to confirm these compositions. The new measurement data were then fed back to improve the prediction model. In this way, the prediction model is constantly being updated by each RL prediction. Once a successful update has been made to the prediction model, the whole process iterates. Finally, a well-trained DDPG model combines the benefits of both Q-learning and Policy Gradient method. DDPG is faster, simpler, more robust, and able to achieve much higher conductivity than conventional search methods. Finally, the model could provide compositions that lead to higher conductivities than the highest conductivity in the training data. Then, we generated more training data according to these compositions to retrain the prediction model. The generated recipes have been attested both by machine learning metrics and wet lab experiments. The generated best conductivity (2:51e[superscript -3]) has meet our expectations of battery recipes.

Download Full-text

Smart SDN Management of Fog Services to Optimize QoS and Energy

Sensors ◽

10.3390/s21093105 ◽

2021 ◽

Vol 21 (9) ◽

pp. 3105

Author(s):

Piotr Fröhlich ◽

Erol Gelenbe ◽

Jerzy Fiołka ◽

Jacek Chęciński ◽

Mateusz Nowak ◽

...

Keyword(s):

Power Consumption ◽

Response Times ◽

Goal Function ◽

Time Varying ◽

Test Bed ◽

Average Response Time ◽

Client Server ◽

Control Scheme ◽

Iot Devices ◽

Network Power

The short latency required by IoT devices that need to access specific services have led to the development of Fog architectures that can serve as a useful intermediary between IoT systems and the Cloud. However, the massive numbers of IoT devices that are being deployed raise concerns about the power consumption of such systems as the number of IoT devices and Fog servers increase. Thus, in this paper, we describe a software-defined network (SDN)-based control scheme for client–server interaction that constantly measures ongoing client–server response times and estimates network power consumption, in order to select connection paths that minimize a composite goal function, including both QoS and power consumption. The approach using reinforcement learning with neural networks has been implemented in a test-bed and is detailed in this paper. Experiments are presented that show the effectiveness of our proposed system in the presence of a time-varying workload of client-to-service requests, resulting in a reduction of power consumption of approximately 15% for an average response time increase of under 2%.

Download Full-text

Learning and control

10.1093/oso/9780199674923.003.0026 ◽

2018 ◽

Author(s):

Ivan Herreros

Keyword(s):

Machine Learning ◽

Reinforcement Learning ◽

Brain Function ◽

Control Strategies ◽

Learning Problems ◽

Animal Learning ◽

Feed Forward Control ◽

Machine Learning Applications ◽

And Control

This chapter discusses basic concepts from control theory and machine learning to facilitate a formal understanding of animal learning and motor control. It first distinguishes between feedback and feed-forward control strategies, and later introduces the classification of machine learning applications into supervised, unsupervised, and reinforcement learning problems. Next, it links these concepts with their counterparts in the domain of the psychology of animal learning, highlighting the analogies between supervised learning and classical conditioning, reinforcement learning and operant conditioning, and between unsupervised and perceptual learning. Additionally, it interprets innate and acquired actions from the standpoint of feedback vs anticipatory and adaptive control. Finally, it argues how this framework of translating knowledge between formal and biological disciplines can serve us to not only structure and advance our understanding of brain function but also enrich engineering solutions at the level of robot learning and control with insights coming from biology.

Download Full-text

Hybrid features prediction model of movie quality using Multi-machine learning techniques for effective business resource planning

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-201844 ◽

2021 ◽

Vol 40 (5) ◽

pp. 9361-9382 ◽

Cited By ~ 1

Author(s):

Naeem Iqbal ◽

Rashid Ahmad ◽

Faisal Jamil ◽

Do-Hyeun Kim

Keyword(s):

Machine Learning ◽

Social Media ◽

Resource Planning ◽

Experimental Results ◽

Quality Prediction ◽

Classification Models ◽

Hybrid Features ◽

Social Media Data ◽

Media Data

Quality prediction plays an essential role in the business outcome of the product. Due to the business interest of the concept, it has extensively been studied in the last few years. Advancement in machine learning (ML) techniques and with the advent of robust and sophisticated ML algorithms, it is required to analyze the factors influencing the success of the movies. This paper presents a hybrid features prediction model based on pre-released and social media data features using multiple ML techniques to predict the quality of the pre-released movies for effective business resource planning. This study aims to integrate pre-released and social media data features to form a hybrid features-based movie quality prediction (MQP) model. The proposed model comprises of two different experimental models; (i) predict movies quality using the original set of features and (ii) develop a subset of features based on principle component analysis technique to predict movies success class. This work employ and implement different ML-based classification models, such as Decision Tree (DT), Support Vector Machines with the linear and quadratic kernel (L-SVM and Q-SVM), Logistic Regression (LR), Bagged Tree (BT) and Boosted Tree (BOT), to predict the quality of the movies. Different performance measures are utilized to evaluate the performance of the proposed ML-based classification models, such as Accuracy (AC), Precision (PR), Recall (RE), and F-Measure (FM). The experimental results reveal that BT and BOT classifiers performed accurately and produced high accuracy compared to other classifiers, such as DT, LR, LSVM, and Q-SVM. The BT and BOT classifiers achieved an accuracy of 90.1% and 89.7%, which shows an efficiency of the proposed MQP model compared to other state-of-art- techniques. The proposed work is also compared with existing prediction models, and experimental results indicate that the proposed MQP model performed slightly better compared to other models. The experimental results will help the movies industry to formulate business resources effectively, such as investment, number of screens, and release date planning, etc.

Download Full-text

Quantum Reinforcement Learning with Quantum Photonics

Photonics ◽

10.3390/photonics8020033 ◽

2021 ◽

Vol 8 (2) ◽

pp. 33

Author(s):

Lucas Lamata

Keyword(s):

Machine Learning ◽

Reinforcement Learning ◽

Quantum Computation ◽

Exchange Information ◽

Quantum Machine Learning ◽

Quantum Technology ◽

Quantum Photonics

Quantum machine learning has emerged as a promising paradigm that could accelerate machine learning calculations. Inside this field, quantum reinforcement learning aims at designing and building quantum agents that may exchange information with their environment and adapt to it, with the aim of achieving some goal. Different quantum platforms have been considered for quantum machine learning and specifically for quantum reinforcement learning. Here, we review the field of quantum reinforcement learning and its implementation with quantum photonics. This quantum technology may enhance quantum computation and communication, as well as machine learning, via the fruitful marriage between these previously unrelated fields.

Download Full-text

Adaptive dynamic programming and deep reinforcement learning for the control of an unmanned surface vehicle: Experimental results

Control Engineering Practice ◽

10.1016/j.conengprac.2021.104807 ◽

2021 ◽

Vol 111 ◽

pp. 104807

Author(s):

Alejandro Gonzalez-Garcia ◽

David Barragan-Alcantar ◽

Ivana Collado-Gonzalez ◽

Leonardo Garrido

Keyword(s):

Dynamic Programming ◽

Reinforcement Learning ◽

Adaptive Dynamic Programming ◽

Experimental Results ◽

Unmanned Surface Vehicle ◽

Adaptive Dynamic

Download Full-text

Machine Learning Empowered Trust Evaluation Method for IoT Devices

IEEE Access ◽

10.1109/access.2021.3076118 ◽

2021 ◽

Vol 9 ◽

pp. 65066-65077

Author(s):

Wei Ma ◽

Xing Wang ◽

Mingsheng Hu ◽

Qinglei Zhou

Keyword(s):

Machine Learning ◽

Evaluation Method ◽

Trust Evaluation ◽

Iot Devices

Download Full-text

Predictive Maintenance in Building Facilities: A Machine Learning-Based Approach

Sensors ◽

10.3390/s21041044 ◽

2021 ◽

Vol 21 (4) ◽

pp. 1044

Author(s):

Yassine Bouabdallaoui ◽

Zoubeir Lafhaj ◽

Pascal Yim ◽

Laure Ducoulombier ◽

Belkacem Bennadji

Keyword(s):

Machine Learning ◽

Model Development ◽

Predictive Maintenance ◽

Data Availability ◽

Machine Learning Techniques ◽

Automation System ◽

Building Maintenance ◽

Collection Data ◽

Iot Devices

The operation and maintenance of buildings has seen several advances in recent years. Multiple information and communication technology (ICT) solutions have been introduced to better manage building maintenance. However, maintenance practices in buildings remain less efficient and lead to significant energy waste. In this paper, a predictive maintenance framework based on machine learning techniques is proposed. This framework aims to provide guidelines to implement predictive maintenance for building installations. The framework is organised into five steps: data collection, data processing, model development, fault notification and model improvement. A sport facility was selected as a case study in this work to demonstrate the framework. Data were collected from different heating ventilation and air conditioning (HVAC) installations using Internet of Things (IoT) devices and a building automation system (BAS). Then, a deep learning model was used to predict failures. The case study showed the potential of this framework to predict failures. However, multiple obstacles and barriers were observed related to data availability and feedback collection. The overall results of this paper can help to provide guidelines for scientists and practitioners to implement predictive maintenance approaches in buildings.

Download Full-text

Application of Deep Reinforcement Learning to Predict Shaft Deformation Considering Hull Deformation of Medium-Sized Oil/Chemical Tanker

Journal of Marine Science and Engineering ◽

10.3390/jmse9070767 ◽

2021 ◽

Vol 9 (7) ◽

pp. 767

Author(s):

Shin-Pyo Choi ◽

Jae-Ung Lee ◽

Jun-Bum Park

Keyword(s):

Reinforcement Learning ◽

Reaction Force ◽

Measurement Data ◽

System Stability ◽

Main Bearing ◽

Shaft System ◽

Novel Approach ◽

Design Changes ◽

Prediction Techniques ◽

Shaft Deformation

The enlargement of ships has increased the relative hull deformation owing to draft changes. Moreover, design changes such as an increased propeller diameter and pitch changes have occurred to compensate for the reduction in the engine revolution and consequent ship speed. In terms of propulsion shaft alignment, as the load of the stern tube support bearing increases, an uneven load distribution occurs between the shaft support bearings, leading to stern accidents. To prevent such accidents and to ensure shaft system stability, a shaft system design technique is required in which the shaft deformation resulting from the hull deformation is considered. Based on the measurement data of a medium-sized oil/chemical tanker, this study presents a novel approach to predicting the shaft deformation following stern hull deformation through inverse analysis using deep reinforcement learning, as opposed to traditional prediction techniques. The main bearing reaction force, which was difficult to reflect in previous studies, was predicted with high accuracy by comparing it with the measured value, and reasonable shaft deformation could be derived according to the hull deformation. The deep reinforcement learning technique in this study is expected to be expandable for predicting the dynamic behavior of the shaft of an operating vessel.

Download Full-text