scholarly journals Multi-Level Elasticity for Wide-Area Data Streaming Systems: A Reinforcement Learning Approach

Algorithms ◽  
2018 ◽  
Vol 11 (9) ◽  
pp. 134 ◽  
Author(s):  
Gabriele Russo Russo ◽  
Matteo Nardelli ◽  
Valeria Cardellini ◽  
Francesco Lo Presti
Keyword(s):  
Reinforcement Learning ◽  
High Volume ◽  
Wide Area ◽  
Data Streaming ◽  
Application Performance ◽  
Ubiquitous Sensing ◽  
Self Adaptation ◽  
Adaptation Policies ◽  
Multi Level ◽  
Dsp Applications

The capability of efficiently processing the data streams emitted by nowadays ubiquitous sensing devices enables the development of new intelligent services. Data Stream Processing (DSP) applications allow for processing huge volumes of data in near real-time. To keep up with the high volume and velocity of data, these applications can elastically scale their execution on multiple computing resources to process the incoming data flow in parallel. Being that data sources and consumers are usually located at the network edges, nowadays the presence of geo-distributed computing resources represents an attractive environment for DSP. However, controlling the applications and the processing infrastructure in such wide-area environments represents a significant challenge. In this paper, we present a hierarchical solution for the autonomous control of elastic DSP applications and infrastructures. It consists of a two-layered hierarchical solution, where centralized components coordinate subordinated distributed managers, which, in turn, locally control the elastic adaptation of the application components and deployment regions. Exploiting this framework, we design several self-adaptation policies, including reinforcement learning based solutions. We show the benefits of the presented self-adaptation policies with respect to static provisioning solutions, and discuss the strengths of reinforcement learning based approaches, which learn from experience how to optimize the application performance and resource allocation.

Towards Cooperative Caching for Vehicular Networks with Multi-level Federated Reinforcement Learning

2021 ◽  
Author(s):  
Lei Zhao ◽  
Yongyi Ran ◽  
Hao Wang ◽  
Junxia Wang ◽  
Jiangtao Luo
Keyword(s):  
Reinforcement Learning ◽  
Vehicular Networks ◽  
Cooperative Caching ◽  
Multi Level

A Two-Layer Approach to Developing Self-Adaptive Multi-Agent Systems in Open Environment

2014 ◽  
Vol 6 (1) ◽  
pp. 65-85 ◽  
Author(s):  
Xinjun Mao ◽  
Menggao Dong ◽  
Haibin Zhu
Keyword(s):  
Reinforcement Learning ◽  
Adaptive Systems ◽  
Multi Agent Systems ◽  
Uncertain Environments ◽  
Implementation Framework ◽  
Fine Grain ◽  
Adaptation Mechanisms ◽  
Multi Agent ◽  
Self Adaptation ◽  
Self Adaptive

Development of self-adaptive systems situated in open and uncertain environments is a great challenge in the community of software engineering due to the unpredictability of environment changes and the variety of self-adaptation manners. Explicit specification of expected changes and various self-adaptations at design-time, an approach often adopted by developers, seems ineffective. This paper presents an agent-based approach that combines two-layer self-adaptation mechanisms and reinforcement learning together to support the development and running of self-adaptive systems. The approach takes self-adaptive systems as multi-agent organizations and enables the agent itself to make decisions on self-adaptation by learning at run-time and at different levels. The proposed self-adaptation mechanisms that are based on organization metaphors enable self-adaptation at two layers: fine-grain behavior level and coarse-grain organization level. Corresponding reinforcement learning algorithms on self-adaptation are designed and integrated with the two-layer self-adaptation mechanisms. This paper further details developmental technologies, based on the above approach, in establishing self-adaptive systems, including extended software architecture for self-adaptation, an implementation framework, and a development process. A case study and experiment evaluations are conducted to illustrate the effectiveness of the proposed approach.

scholarly journals Learning an Efficient Gait Cycle of a Biped Robot Based on Reinforcement Learning and Artificial Neural Networks

2019 ◽  
Vol 9 (3) ◽  
pp. 502 ◽  
Author(s):  
Cristyan Gil ◽  
Hiram Calvo ◽  
Humberto Sossa
Keyword(s):  
Reinforcement Learning ◽  
Gait Cycle ◽  
Biped Robot ◽  
Q Learning ◽  
Nao Robot ◽  
Simulated Environment ◽  
Proposed Model ◽  
Normal Speed ◽  
Multi Level ◽  
Efficient Gait

Programming robots for performing different activities requires calculating sequences of values of their joints by taking into account many factors, such as stability and efficiency, at the same time. Particularly for walking, state of the art techniques to approximate these sequences are based on reinforcement learning (RL). In this work we propose a multi-level system, where the same RL method is used first to learn the configuration of robot joints (poses) that allow it to stand with stability, and then in the second level, we find the sequence of poses that let it reach the furthest distance in the shortest time, while avoiding falling down and keeping a straight path. In order to evaluate this, we focus on measuring the time it takes for the robot to travel a certain distance. To our knowledge, this is the first work focusing both on speed and precision of the trajectory at the same time. We implement our model in a simulated environment using q-learning. We compare with the built-in walking modes of an NAO robot by improving normal-speed and enhancing robustness in fast-speed. The proposed model can be extended to other tasks and is independent of a particular robot model.

scholarly journals Sparse Wide-Area Control of Power Systems using Data-driven Reinforcement Learning

2019 ◽  
Author(s):  
Amirhassan Fallah Dizche ◽  
Aranya Chakrabortty ◽  
Alexandra Duel-Hallen
Keyword(s):  
Reinforcement Learning ◽  
Power Systems ◽  
Data Driven ◽  
Wide Area ◽  
Using Data ◽  
Wide Area Control

A Multi-Level Self-Adaptation Approach For Microservice Systems

2019 ◽  
Author(s):  
Shuai Zhang ◽  
Mingjiang Zhang ◽  
Lin Ni ◽  
Peini Liu
Keyword(s):  
Self Adaptation ◽  
Multi Level

ReLAccS: A Multi-level Approach to Accelerator Design for Reinforcement Learning on FPGA-based Systems

2020 ◽  
pp. 1-1
Author(s):  
Akhil Raj Baranwal ◽  
Salim Ullah ◽  
Siva Satyendra Sahoo ◽  
Akash Kumar
Keyword(s):  
Reinforcement Learning ◽  
Multi Level ◽  
Accelerator Design

A self-managing wide-area data streaming service

2007 ◽  
Vol 10 (4) ◽  
pp. 365-383 ◽  
Author(s):  
Viraj Bhat ◽  
Manish Parashar ◽  
Hua Liu ◽  
Nagarajan Kandasamy ◽  
Mohit Khandekar ◽  
...  
Keyword(s):  
Wide Area ◽  
Data Streaming ◽  
Streaming Service ◽  
Area Data

scholarly journals Boundary spanning for governance of climate change adaptation in cities: Insights from a Dutch urban region

2017 ◽  
Vol 36 (5) ◽  
pp. 837-855 ◽  
Author(s):  
Marcin Dąbrowski
Keyword(s):  
Climate Change ◽  
Urban Climate ◽  
Boundary Spanning ◽  
Urban Setting ◽  
Urban Region ◽  
Track Record ◽  
Adaptation Policies ◽  
Multi Level ◽  
Negative Impacts ◽  
Urban Adaptation

Adapting to climate change in the urban setting requires cooperation across scales, levels of government, organisational boundaries and policy sectors. The study presented in the paper explores governance of urban adaptation policies through the conceptual lens of multi-level governance and boundary spanning. It focuses on the South Wing of the Randstad in The Netherlands, an urban region that is heavily exposed to the negative impacts of climate change, particularly to flooding, due to its location in the Rhine-Meuse delta and concentration of population and economic activity. Yet, it is also a region with strong traditions of cooperation and a track record of pioneering urban climate change measures. The study investigates how the features of the wider institutional context, in which this urban region operates shape the governance of urban adaptation policies and how the contextual factors constrain the scope for spanning horizontal, vertical and temporal boundaries needed for delivering those policies and making the cities of that region more climate-proof.

Sign in / Sign up

Export Citation Format

Share Document