scholarly journals Resource-Driven Mission-Phasing Techniques for Constrained Agents in Stochastic Environments

2010 ◽  
Vol 38 ◽  
pp. 415-473 ◽  
Author(s):  
J. Wu ◽  
E. H. Durfee
Keyword(s):  
Optimization Problem ◽  
Resource Constraints ◽  
Single Agent ◽  
Sequential Optimization ◽  
Mixed Integer ◽  
Stochastic Nature ◽  
Stochastic Environments ◽  
Optimal Allocations ◽  
Multi Agent ◽  
Optimal Series

Because an agent's resources dictate what actions it can possibly take, it should plan which resources it holds over time carefully, considering its inherent limitations (such as power or payload restrictions), the competing needs of other agents for the same resources, and the stochastic nature of the environment. Such agents can, in general, achieve more of their objectives if they can use --- and even create --- opportunities to change which resources they hold at various times. Driven by resource constraints, the agents could break their overall missions into an optimal series of phases, optimally reconfiguring their resources at each phase, and optimally using their assigned resources in each phase, given their knowledge of the stochastic environment. In this paper, we formally define and analyze this constrained, sequential optimization problem in both the single-agent and multi-agent contexts. We present a family of mixed integer linear programming (MILP) formulations of this problem that can optimally create phases (when phases are not predefined) accounting for costs and limitations in phase creation. Because our formulations multaneously also find the optimal allocations of resources at each phase and the optimal policies for using the allocated resources at each phase, they exploit structure across these coupled problems. This allows them to find solutions significantly faster(orders of magnitude faster in larger problems) than alternative solution techniques, as we demonstrate empirically.

scholarly journals Multi-Agent Reinforcement Learning: A Review of Challenges and Applications

2021 ◽  
Vol 11 (11) ◽  
pp. 4948
Author(s):  
Lorenzo Canese ◽  
Gian Carlo Cardarilli ◽  
Luca Di Di Nunzio ◽  
Rocco Fazzolari ◽  
Daniele Giardino ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Mathematical Models ◽  
Learning Algorithms ◽  
Single Agent ◽  
Critical Issues ◽  
Multi Agent ◽  
Pros And Cons ◽  
Application Fields

In this review, we present an analysis of the most used multi-agent reinforcement learning algorithms. Starting with the single-agent reinforcement learning algorithms, we focus on the most critical issues that must be taken into account in their extension to multi-agent scenarios. The analyzed algorithms were grouped according to their features. We present a detailed taxonomy of the main multi-agent approaches proposed in the literature, focusing on their related mathematical models. For each algorithm, we describe the possible application fields, while pointing out its pros and cons. The described multi-agent algorithms are compared in terms of the most important characteristics for multi-agent reinforcement learning applications—namely, nonstationarity, scalability, and observability. We also describe the most common benchmark environments used to evaluate the performances of the considered methods.

scholarly journals Optimal Scheduling for Laboratory Automation of Life Science Experiments with Time Constraints

2021 ◽  
pp. 247263032110217
Author(s):  
Takeshi D. Itoh ◽  
Takaaki Horinouchi ◽  
Hiroki Uchida ◽  
Koichi Takahashi ◽  
Haruka Ozaki
Keyword(s):  
Scheduling Algorithms ◽  
Mixed Integer ◽  
Time Constraints ◽  
Live Cells ◽  
Laboratory Automation ◽  
Simulation Based Design ◽  
Simulation Based ◽  
Optimal Allocations ◽  
Scheduling Method ◽  
Time Required

In automated laboratories consisting of multiple different types of instruments, scheduling algorithms are useful for determining the optimal allocations of instruments to minimize the time required to complete experimental procedures. However, previous studies on scheduling algorithms for laboratory automation have not emphasized the time constraints by mutual boundaries (TCMBs) among operations, which is important in procedures involving live cells or unstable biomolecules. Here, we define the “scheduling for laboratory automation in biology” (S-LAB) problem as a scheduling problem for automated laboratories in which operations with TCMBs are performed by multiple different instruments. We formulate an S-LAB problem as a mixed-integer programming (MIP) problem and propose a scheduling method using the branch-and-bound algorithm. Simulations show that our method can find the optimal schedules of S-LAB problems that minimize overall execution time while satisfying the TCMBs. Furthermore, we propose the use of our scheduling method for the simulation-based design of job definitions and laboratory configurations.

scholarly journals Joint optimization of computing ratio and access points’ density for mixed mobile edge/cloud computing

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Tianqi Jing ◽  
Shiwen He ◽  
Fei Yu ◽  
Yongming Huang ◽  
Luxi Yang ◽  
...  
Keyword(s):  
Cloud Computing ◽  
Coverage Probability ◽  
Optimization Problem ◽  
Mobile Cloud Computing ◽  
Low Complexity ◽  
Mixed Integer ◽  
Joint Optimization ◽  
Mobile Cloud ◽  
Convex Problem

AbstractCooperation between the mobile edge computing (MEC) and the mobile cloud computing (MCC) in offloading computing could improve quality of service (QoS) of user equipments (UEs) with computation-intensive tasks. In this paper, in order to minimize the expect charge, we focus on the problem of how to offload the computation-intensive task from the resource-scarce UE to access point’s (AP) and the cloud, and the density allocation of APs’ at mobile edge. We consider three offloading computing modes and focus on the coverage probability of each mode and corresponding ergodic rates. The resulting optimization problem is a mixed-integer and non-convex problem in the objective function and constraints. We propose a low-complexity suboptimal algorithm called Iteration of Convex Optimization and Nonlinear Programming (ICONP) to solve it. Numerical results verify the better performance of our proposed algorithm. Optimal computing ratios and APs’ density allocation contribute to the charge saving.

scholarly journals On-Demand Channel Bonding in Heterogeneous WLANs: A Multi-Agent Deep Reinforcement Learning Approach

Sensors ◽  
2020 ◽  
Vol 20 (10) ◽  
pp. 2789 ◽  
Author(s):  
Hang Qi ◽  
Hao Huang ◽  
Zhiqun Hu ◽  
Xiangming Wen ◽  
Zhaoming Lu
Keyword(s):  
Reinforcement Learning ◽  
Transmission Rate ◽  
Single Agent ◽  
Time Of Day ◽  
Action Space ◽  
Traffic Load ◽  
Traffic Demand ◽  
Channel Bonding ◽  
On Demand ◽  
Multi Agent

In order to meet the ever-increasing traffic demand of Wireless Local Area Networks (WLANs), channel bonding is introduced in IEEE 802.11 standards. Although channel bonding effectively increases the transmission rate, the wider channel reduces the number of non-overlapping channels and is more susceptible to interference. Meanwhile, the traffic load differs from one access point (AP) to another and changes significantly depending on the time of day. Therefore, the primary channel and channel bonding bandwidth should be carefully selected to meet traffic demand and guarantee the performance gain. In this paper, we proposed an On-Demand Channel Bonding (O-DCB) algorithm based on Deep Reinforcement Learning (DRL) for heterogeneous WLANs to reduce transmission delay, where the APs have different channel bonding capabilities. In this problem, the state space is continuous and the action space is discrete. However, the size of action space increases exponentially with the number of APs by using single-agent DRL, which severely affects the learning rate. To accelerate learning, Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is used to train O-DCB. Real traffic traces collected from a campus WLAN are used to train and test O-DCB. Simulation results reveal that the proposed algorithm has good convergence and lower delay than other algorithms.

ONE IMPROVED AGENT GENETIC ALGORITHM — RING-LIKE AGENT GENETIC ALGORITHM FOR GLOBAL NUMERICAL OPTIMIZATION

2009 ◽  
Vol 26 (04) ◽  
pp. 479-502 ◽  
Author(s):  
BIN LIU ◽  
TEQI DUAN ◽  
YONGMING LI
Keyword(s):  
Genetic Algorithm ◽  
Numerical Optimization ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Genetic Operators ◽  
Global Numerical Optimization ◽  
Candidate Solution ◽  
Low Dimension ◽  
Multi Agent ◽  
Agent Structure

In this paper, a novel genetic algorithm — dynamic ring-like agent genetic algorithm (RAGA) is proposed for solving global numerical optimization problem. The RAGA combines the ring-like agent structure and dynamic neighboring genetic operators together to get better optimization capability. An agent in ring-like agent structure represents a candidate solution to the optimization problem. Any agent interacts with neighboring agents to evolve. With dynamic neighboring genetic operators, they compete and cooperate with their neighbors, and they can also use knowledge to increase energies. Global numerical optimization problems are the most important ones to verify the performance of evolutionary algorithm, especially of genetic algorithm and are mostly of interest to the corresponding researchers. In the corresponding experiments, several complex benchmark functions were used for optimization, several popular GAs were used for comparison. In order to better compare two agents GAs (MAGA: multi-agent genetic algorithm and RAGA), the several dimensional experiments (from low dimension to high dimension) were done. These experimental results show that RAGA not only is suitable for optimization problems, but also has more precise and more stable optimization results.

scholarly journals Multi-mobile agent itinerary planning algorithms for data gathering in wireless sensor networks: A review paper

2017 ◽  
Vol 13 (1) ◽  
pp. 155014771668484 ◽  
Author(s):  
Huthiafa Q Qadori ◽  
Zuriati A Zulkarnain ◽  
Zurina Mohd Hanapi ◽  
Shamala Subramaniam
Keyword(s):  
Wireless Sensor Networks ◽  
Sensor Networks ◽  
Mobile Agent ◽  
Mobile Agents ◽  
Single Agent ◽  
Data Gathering ◽  
Optimal Number ◽  
Wireless Sensor ◽  
Itinerary Planning ◽  
Multi Agent

Recently, wireless sensor networks have employed the concept of mobile agent to reduce energy consumption and obtain effective data gathering. Typically, in data gathering based on mobile agent, it is an important and essential step to find out the optimal itinerary planning for the mobile agent. However, single-agent itinerary planning suffers from two primary disadvantages: task delay and large size of mobile agent as the scale of the network is expanded. Thus, using multi-agent itinerary planning overcomes the drawbacks of single-agent itinerary planning. Despite the advantages of multi-agent itinerary planning, finding the optimal number of distributed mobile agents, source nodes grouping, and optimal itinerary of each mobile agent for simultaneous data gathering are still regarded as critical issues in wireless sensor network. Therefore, in this article, the existing algorithms that have been identified in the literature to address the above issues are reviewed. The review shows that most of the algorithms used one parameter to find the optimal number of mobile agents in multi-agent itinerary planning without utilizing other parameters. More importantly, the review showed that theses algorithms did not take into account the security of the data gathered by the mobile agent. Accordingly, we indicated the limitations of each proposed algorithm and new directions are provided for future research.

Optimization design of drive system for industrial robots based on dynamic performance

2017 ◽  
Vol 44 (6) ◽  
pp. 765-775 ◽  
Author(s):  
LianZheng Ge ◽  
Jian Chen ◽  
Ruifeng Li ◽  
Peidong Liang
Keyword(s):  
Optimization Model ◽  
Optimization Problem ◽  
Industrial Robot ◽  
Dynamic Performance ◽  
System Optimization ◽  
Drive System ◽  
Industrial Robots ◽  
Mixed Integer ◽  
Light Weight ◽  
Content Type

Purpose The global performance of industrial robots partly depends on the properties of drive system consisting of motor inertia, gearbox inertia, etc. This paper aims to deal with the problem of optimization of global dynamic performance for robotic drive system selected from available components. Design/methodology/approach Considering the performance specifications of drive system, an optimization model whose objective function is composed of working efficiency and natural frequency of robots is proposed. Meanwhile, constraints including the rated and peak torque of motor, lifetime of gearbox and light-weight were taken into account. Furthermore, the mapping relationship between discrete optimal design variables and component properties of drive system were presented. The optimization problem with mixed integer variables was solved by a mixed integer-laplace crossover power mutation algorithm. Findings The optimization results show that our optimization model and methods are applicable, and the performances are also greatly promoted without sacrificing any constraints of drive system. Besides, the model fits the overall performance well with respect to light-weight ratio, safety, cost reduction and others. Practical implications The proposed drive system optimization method has been used for a 4-DOF palletizing robot, which has been largely manufactured in a factory. Originality/value This paper focuses on how the simulation-based optimization can be used for the purpose of generating trade-offs between cost, performance and lifetime when designing robotic drive system. An applicable optimization model and method are proposed to handle the dynamic performance optimization problem of a drive system for industrial robot.

scholarly journals An Efficient Multi-Vehicle Routing Strategy for Goods Delivery Services

2021 ◽  
Author(s):  
Linh Nguyen
Keyword(s):  
Autonomous Vehicles ◽  
Optimization Problem ◽  
Transportation Cost ◽  
Mixed Integer ◽  
Optimal Routing ◽  
Ground Vehicles ◽  
Routing Optimization ◽  
Routing Strategy ◽  
Delivery Services ◽  
Delivery Vehicles

<pre>The paper addresses the problem of efficiently planning routes for multiple ground vehicles used in goods delivery services. Given popularity of today's e-commerce, particularly under the COVID-19 pandemic conditions, goods delivery services have been booming than ever, dominated by small-scaled (electric) bikes and promised by autonomous vehicles. However, finding optimal routing paths for multiple delivery vehicles operating simultaneously in order to minimize transportation cost is a fundamental but challenging problem. In this paper, it is first proposed to exploit the mixed integer programming paradigm to model the delivery routing optimization problem (DROP) for multiple simultaneously-operating vehicles given their energy constraints. The routing optimization problem is then solved by the multi-chromosome genetic algorithm, where the number of delivery vehicles can be optimized. The proposed approach was evaluated in a real-world experiment in which goods were expected to be delivered from a depot to 26 suburb locations in Canberra, Australia. The obtained results demonstrate effectiveness of the proposed algorithm.</pre>

scholarly journals Adaptive behaviors in multi-agent source localization using passive sensing

2016 ◽  
Vol 24 (6) ◽  
pp. 446-463 ◽  
Author(s):  
Mansoor Shaukat ◽  
Mandar Chitre
Keyword(s):  
Source Localization ◽  
Single Agent ◽  
Analytical Models ◽  
Localization Algorithm ◽  
Two Phase ◽  
Temporal Sampling ◽  
Passive Sensing ◽  
Source Signal ◽  
Localization Strategy ◽  
Multi Agent

In this paper, the role of adaptive group cohesion in a cooperative multi-agent source localization problem is investigated. A distributed source localization algorithm is presented for a homogeneous team of simple agents. An agent uses a single sensor to sense the gradient and two sensors to sense its neighbors. The algorithm is a set of individualistic and social behaviors where the individualistic behavior is as simple as an agent keeping its previous heading and is not self-sufficient in localizing the source. Source localization is achieved as an emergent property through agent’s adaptive interactions with the neighbors and the environment. Given a single agent is incapable of localizing the source, maintaining team connectivity at all times is crucial. Two simple temporal sampling behaviors, intensity-based-adaptation and connectivity-based-adaptation, ensure an efficient localization strategy with minimal agent breakaways. The agent behaviors are simultaneously optimized using a two phase evolutionary optimization process. The optimized behaviors are estimated with analytical models and the resulting collective behavior is validated against the agent’s sensor and actuator noise, strong multi-path interference due to environment variability, initialization distance sensitivity and loss of source signal.

Sign in / Sign up

Export Citation Format

Share Document