Autonomous and cooperative control of UAV cluster with multi-agent reinforcement learning

In this paper, we expolore Multi-Agent Reinforcement Learning (MARL) methods for unmanned aerial vehicle (UAV) cluster. Considering that the current UAV cluster is still in the program control stage, the fully autonomous and intelligent cooperative combat has not been realised. In order to realise the autonomous planning of the UAV cluster according to the changing environment and cooperate with each other to complete the combat goal, we propose a new MARL framework. It adopts the policy of centralised training with decentralised execution, and uses Actor-Critic network to select the execution action and then to make the corresponding evaluation. The new algorithm makes three key improvements on the basis of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. The first is to improve learning framework; it makes the calculated Q value more accurate. The second is to add collision avoidance setting, which can increase the operational safety factor. And the third is to adjust reward mechanism; it can effectively improve the cluster’s cooperative ability. Then the improved MADDPG algorithm is tested by performing two conventional combat missions. The simulation results show that the learning efficiency is obviously improved, and the operational safety factor is further increased compared with the previous algorithm.

Enabling modular autonomous feedback-loops in materials science through hierarchical experimental laboratory automation and orchestration

Materials acceleration platforms (MAPs) operate on the paradigm of integrating combinatorial synthesis, high-throughput characterization, automatic analysis, and machine learning. Within these MAPs, one or multiple autonomous feedback loops may aim to optimize materials for certain functional properties or generate new insights. The scope of a given experiment campaign is defined by the range of experiment and analysis actions that are integrated into the experiment framework. Herein we present a method for integrating many actions within a hierarchical experimental laboratory automation and orchestration (HELAO) framework. We demonstrate the capability of orchestrating distributed research instruments that can incorporate data from experiments, simulations, and databases. HELAO interfaces laboratory hardware and software that are distributed across several computers and operating systems for executing experiments, data analysis, provenance tracking, and autonomous planning. Parallelization is an effective approach for accelerating knowledge generation provided that multiple instruments can be effectively coordinated, which we demonstrate with parallel electrochemistry experiments orchestrated by HELAO. Efficient implementation of autonomous research strategies requires device sharing, asynchronous multithreading, and full integration of data management in experiment orchestration, which to the best of our knowledge, is demonstrated for the first time herein.

Autonomous Trajectory Planning and Control of Anti-Radiation Loitering Munitions under Uncertain Conditions

As an autonomous system, an anti-radiation loitering munition (LM) experiences uncertainty in both a priori and sensed information during loitering because it is difficult to accurately know target radar information in advance, and the sensing performance of the seeker is affected by disturbance and errors. If, as it does in the state of the art, uncertainties are ignored and the LM travels its planned route, its battle effectiveness will be severely restricted. To tackle this problem, this paper studies the method of autonomous planning and control of loitering routes using limited a priori information of target radar and real-time sensing results. We establish a motion and sensing model based on the characteristics of anti-radiation LMs and use particle filtering to iteratively infer the target radar information. Based on model predictive control, we select a loitering path to minimize the uncertainty of the target information, so as to achieve trajectory planning control that is conducive to the acquisition of target radar information. Simulation results show that the proposed method can effectively complete the autonomous trajectory planning and control of anti-radiation LMs under uncertain conditions.

Integrating Predictive Maintenance in Adaptive Process Scheduling for a Safe and Efficient Industrial Process

Predictive maintenance (PM) algorithms are widely applied for detecting operational anomalies on industrial processes to schedule for a maintenance intervention before a possible breakdown; however, much less focus has been devoted to the use of such prognostics in process scheduling. The existing solutions mostly integrate preventive approaches to protect the machines, usually causing downtimes. The premise of this study is to develop a process scheduling mechanism that selects an acceptable operating condition for an industrial process to adapt to the predicted anomalies. As PM is largely a data-driven approach (hence, it relies on the setup), we first compare different PM approaches and identify a one-class support vector machine (OCSVM) as the best performing option for the anomaly detection on our setup. Then, we propose a novel pipeline to integrate maintenance predictions into a real-time, adaptive process scheduling mechanism. According to the abnormal readings, it schedules for the most suitable operation, i.e., optimizing for machine health and process efficiency, toward preventing breakdowns while maintaining its availability and operational state, thereby reducing downtimes. To demonstrate the pipeline on the action, we implement our approach on a small-scale conveyor belt, utilizing our Internet of Things (IoT) framework. The results show that our PM-based adaptive process control retains an efficient process under abnormal conditions with less or no downtime. We also conclude that a PM approach does not provide sufficient efficiency without its integration into an autonomous planning process.

Comparing mental contrasting with implementation intentions against solution-focused and autonomous planning

Research suggests that mental contrasting with implementation intentions (MCII) enhances commitment and goal attainment. However, most studies have used limited comparison conditions. The present study compared MCII against two other potentially effective approaches: autonomous planning (AP), and solution-focused planning (SFP). It was thought that condition would have an indirect effect on goal progress by affecting commitment. However, goal attainment expectancy was hypothesised to be a moderator such that MCII has positive effects when expectancy is high but negative effects when expectancy is low. Ninety-eight female students were randomly assigned to one of three conditions: 1) MCII, 2) AP, or 3) SFP. All students initially set themselves a goal for the coming week regarding personal projects. Mean commitment and goal progress were marginally higher in the MCII condition than in the AP and SFP conditions but the differences were not statistically significant and (as predicted) much smaller than in previous research. Expectancy did not appear to have a moderating effect. The apparent benefits of MCII were larger relative to AP than to SFP. Results suggest that MCII may sometimes be no more effective than other approaches to goal-setting and planning, particularly if they are evidence-based and carefully-designed. Implications for schools are addressed.