Resilient Team Formation with Stabilisability of Agent Networks for Task Allocation

Team formation (TF) faces the problem of defining teams of agents able to accomplish a set of tasks. Resilience on TF problems aims to provide robustness and adaptability to unforeseen events involving agent deletion. However, agents are unaware of the inherent social welfare in these teams. This article tackles the problem of how teams can minimise their effort in terms of organisation and communication considering these dynamics. Our main contribution is twofold: first, we introduce the Stabilisable Team Formation (STF) as a generalisation of current resilient TF model, where a team is stabilisable if it possesses and preserves its inter-agent organisation from a graph-based perspective. Second, our experiments show that stabilisability is able to reduce the exponential execution time in several units of magnitude with the most restrictive configurations, proving that communication effort in subsequent task allocation problems are relaxed compared with current resilient teams. To do so, we developed SBB-ST, a branch-and-bound algorithm based on Distributed Constrained Optimisation Problems (DCOP) to compute teams. Results evidence that STF improves their predecessors, extends the resilience to subsequent task allocation problems represented as DCOP, and evidence how Stabilisability contributes to resilient TF problems by anticipating decisions for saving resources and minimising the effort on team organisation in dynamic scenarios.

Optimizing the Performance of Containerized Cloud Software Systems Using Adaptive PID Controllers

Control theory has proven to be a practical approach for the design and implementation of controllers, which does not inherit the problems of non-control theoretic controllers due to its strong mathematical background. State-of-the-art auto-scaling controllers suffer from one or more of the following limitations: (1) lack of a reliable performance model, (2) using a performance model with low scalability, tractability, or fidelity, (3) being application- or architecture-specific leading to low extendability, and (4) no guarantee on their efficiency. Consequently, in this article, we strive to mitigate these problems by leveraging an adaptive controller, which is composed of a neural network as the performance model and a Proportional-Integral-Derivative (PID) controller as the scaling engine. More specifically, we design, implement, and analyze different flavours of these adaptive and non-adaptive controllers, and we compare and contrast them against each other to find the most suitable one for managing containerized cloud software systems at runtime. The controller’s objective is to maintain the response time of the controlled software system in a pre-defined range, and meeting the Service-level Agreements, while leading to efficient resource provisioning.

Applying Machine Learning in Self-adaptive Systems

Recently, we have been witnessing a rapid increase in the use of machine learning techniques in self-adaptive systems. Machine learning has been used for a variety of reasons, ranging from learning a model of the environment of a system during operation to filtering large sets of possible configurations before analyzing them. While a body of work on the use of machine learning in self-adaptive systems exists, there is currently no systematic overview of this area. Such an overview is important for researchers to understand the state of the art and direct future research efforts. This article reports the results of a systematic literature review that aims at providing such an overview. We focus on self-adaptive systems that are based on a traditional Monitor-Analyze-Plan-Execute (MAPE)-based feedback loop. The research questions are centered on the problems that motivate the use of machine learning in self-adaptive systems, the key engineering aspects of learning in self-adaptation, and open challenges in this area. The search resulted in 6,709 papers, of which 109 were retained for data collection. Analysis of the collected data shows that machine learning is mostly used for updating adaptation rules and policies to improve system qualities, and managing resources to better balance qualities and resources. These problems are primarily solved using supervised and interactive learning with classification, regression, and reinforcement learning as the dominant methods. Surprisingly, unsupervised learning that naturally fits automation is only applied in a small number of studies. Key open challenges in this area include the performance of learning, managing the effects of learning, and dealing with more complex types of goals. From the insights derived from this systematic literature review, we outline an initial design process for applying machine learning in self-adaptive systems that are based on MAPE feedback loops.

SARDE

Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE , a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.

Enki

Data-driven Learning-enabled Systems are limited by the quality of available training data, particularly when trained offline. For systems that must operate in real-world environments, the space of possible conditions that can occur is vast and difficult to comprehensively predict at design time. Environmental uncertainty arises when run-time conditions diverge from design-time training conditions. To address this problem, automated methods can generate synthetic data to fill in gaps for training and test data coverage. We propose an evolution-based technique to assist developers with uncovering limitations in existing data when previously unseen environmental phenomena are introduced. This technique explores unique contexts for a given environmental condition, with an emphasis on diversity. Synthetic data generated by this technique may be used for two purposes: (1) to assess the robustness of a system to uncertain environmental factors and (2) to improve the system’s robustness. This technique is demonstrated to outperform random and greedy methods for multiple adverse environmental conditions applied to image-processing Deep Neural Networks.

A Q-values Sharing Framework for Multi-agent Reinforcement Learning under Budget Constraint

In a teacher-student framework, a more experienced agent (teacher) helps accelerate the learning of another agent (student) by suggesting actions to take in certain states. In cooperative multi-agent reinforcement learning (MARL), where agents must cooperate with one another, a student could fail to cooperate effectively with others even by following a teacher’s suggested actions, as the policies of all agents can change before convergence. When the number of times that agents communicate with one another is limited (i.e., there are budget constraints), an advising strategy that uses actions as advice could be less effective. We propose a partaker-sharer advising framework (PSAF) for cooperative MARL agents learning with budget constraints. In PSAF, each Q-learner can decide when to ask for and share its Q-values. We perform experiments in three typical multi-agent learning problems. The evaluation results indicate that the proposed PSAF approach outperforms existing advising methods under both constrained and unconstrained budgets. Moreover, we analyse the influence of advising actions and sharing Q-values on agent learning.

SecRET

Node localization is a fundamental requirement in underwater sensor networks (UWSNs) due to the ineptness of GPS and other terrestrial localization techniques in the underwater environment. In any UWSN monitoring application, the sensed information produces a better result when it is tagged with location information. However, the deployed nodes in UWSNs are vulnerable to many attacks, and hence, can be compromised by interested parties to generate incorrect location information. Consequently, using the existing localization schemes, the deployed nodes are unable to autonomously estimate the precise location information. In this regard, similar existing schemes for terrestrial wireless sensor networks are not applicable to UWSNs due to its inherent mobility, limited bandwidth availability, strict energy constraints, and high bit-error rates. In this article, we propose SecRET , a <underline>Sec</underline>ure <underline>R</underline>ange-based localization scheme empowered by <underline>E</underline>vidence <underline>T</underline>heory for UWSNs. With trust-based computations, the proposed scheme, SecRET , enables the unlocalized nodes to select the most reliable set of anchors with low resource consumption. Thus, the proposed scheme is adaptive to many attacks in UWSN environment. NS-3 based performance evaluation indicates that SecRET maintains energy-efficiency of the deployed nodes while ensuring efficient and secure localization, despite the presence of compromised nodes under various attacks.

Information Reuse and Stochastic Search

Many software systems operate in environments of change and uncertainty. Techniques for self-adaptation allow these systems to automatically respond to environmental changes, yet they do not handle changes to the adaptive system itself, such as the addition or removal of adaptation tactics. Instead, changes in a self-adaptive system often require a human planner to redo an expensive planning process to allow the system to continue satisfying its quality requirements under different conditions; automated techniques must replan from scratch. We propose to address this problem by reusing prior planning knowledge to adapt to unexpected situations. We present a planner based on genetic programming that reuses existing plans and evaluate this planner on two case-study systems: a cloud-based web server and a team of autonomous aircraft. While reusing material in genetic algorithms has been recently applied successfully in the area of automated program repair, we find that naively reusing existing plans for self- * planning can actually result in a utility loss. Furthermore, we propose a series of techniques to lower the costs of reuse, allowing genetic techniques to leverage existing information to improve utility when replanning for unexpected changes, and we find that coarsely shaped search-spaces present profitable opportunities for reuse.

REACT-ION: A Model-based Runtime Environment for Situation-aware Adaptations

Trends such as the Internet of Things lead to a growing number of networked devices and to a variety of communication systems. Adding self-adaptive capabilities to these communication systems is one approach to reducing administrative effort and coping with changing execution contexts. Existing frameworks can help reducing development effort but are neither tailored toward the use in communication systems nor easily usable without knowledge in self-adaptive systems development. Accordingly, in previous work, we proposed REACT, a reusable, model-based runtime environment to complement communication systems with adaptive behavior. REACT addresses heterogeneity and distribution aspects of such systems and reduces development effort. In this article, we propose REACT-ION—an extension of REACT for situation awareness. REACT-ION offers a context management module that is able to acquire, store, disseminate, and reason on context data. The context management module is the basis for (i) proactive adaptation with REACT-ION and (ii) self-improvement of the underlying feedback loop. REACT-ION can be used to optimize adaptation decisions at runtime based on the current situation. Therefore, it can cope with uncertainty and situations that were not foreseeable at design time. We show and evaluate in two case studies how REACT-ION’s situation awareness enables proactive adaptation and self-improvement.

AT-DIFC + : Toward Adaptive and Trust-Aware Decentralized Information Flow Control

Modern software systems and their corresponding architectures are increasingly decentralized, distributed, and dynamic. As a consequence, decentralized mechanisms are required to ensure security in such architectures. Decentralized Information Flow Control (DIFC) is a mechanism to control information flow in distributed systems. This article presents and discusses several improvements to an adaptive decentralized information flow approach that incorporates trust for decentralized systems to provide security. Adaptive Trust-Aware Decentralized Information Flow (AT-DIFC + ) combines decentralized information flow control mechanisms, trust-based methods, and decentralized control architectures to control and enforce information flow in an open, decentralized system. We strengthen our approach against newly discovered attacks and provide additional information about its reconfiguration, decentralized control architectures, and reference implementation. We evaluate the effectiveness and performance of AT-DIFC + on two case studies and perform additional experiments and to gauge the mitigations’ effectiveness against the identified attacks.