Autonomous and cooperative control of UAV cluster with multi-agent reinforcement learning

In this paper, we expolore Multi-Agent Reinforcement Learning (MARL) methods for unmanned aerial vehicle (UAV) cluster. Considering that the current UAV cluster is still in the program control stage, the fully autonomous and intelligent cooperative combat has not been realised. In order to realise the autonomous planning of the UAV cluster according to the changing environment and cooperate with each other to complete the combat goal, we propose a new MARL framework. It adopts the policy of centralised training with decentralised execution, and uses Actor-Critic network to select the execution action and then to make the corresponding evaluation. The new algorithm makes three key improvements on the basis of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. The first is to improve learning framework; it makes the calculated Q value more accurate. The second is to add collision avoidance setting, which can increase the operational safety factor. And the third is to adjust reward mechanism; it can effectively improve the cluster’s cooperative ability. Then the improved MADDPG algorithm is tested by performing two conventional combat missions. The simulation results show that the learning efficiency is obviously improved, and the operational safety factor is further increased compared with the previous algorithm.

Overall fuzzy logic control strategy of direct driven PMSG wind turbine connected to grid

The fuzzy logic strategies reported in the literature about the control of direct drive permanent magnet synchronous generator (PMSG) connected to grid are limited in terms of inclusiveness and efficiency. So an overall control based on fuzzy logic and anti-windup compensation is proposed in this paper. Aiming at the inadequate of hill climb search (HCS) MPPT with fixed step size, the fuzzy logic is introduced in the stage of "generating rotor speed reference" to overcome the oscillations and slowness in traditional method. PI controllers are replaced by anti-windup fuzzy logic controllers in the "machine side control" stage and in "grid side control" stage to pertinently regulate the reference parameters. Then comparison tests with classical methods are implemented under varying climatic conditions. The results obtained demonstrate that the developed control is superior to other methods in response time (less than 4.528E-04 s), precision (an overshoot about 0.41%) and quality of produced energy (efficiency is 91%). The study verifying the feasibility and effectiveness of this algorithm in PMSG wind turbine connected to grid.

Global Optimal Stabilization of MT-HVDC Systems: Inverse Optimal Control Approach

The stabilization problem of multi-terminal high-voltage direct current (MT-HVDC) systems feeding constant power loads is addressed in this paper using an inverse optimal control (IOC). A hierarchical control structure using a convex optimization model in the secondary control stage and the IOC in the primary control stage is proposed to determine the set of references that allows the stabilization of the network under load variations. The main advantage of the IOC is that this control method ensures the closed-loop stability of the whole MT-HVDC system using a control Lyapunov function to determine the optimal control law. Numerical results in a reduced version of the CIGRE MT-HVDC system show the effectiveness of the IOC to stabilize the system under large disturbance scenarios, such as short-circuit events and topology changes. All the simulations are carried out in the MATLAB/Simulink environment.

An adaptive shock mitigation control method for helicopter seat system with magnetorheological energy absorbers

This research presents a minimal maximum deceleration (MMD) control method which can be used in the shock mitigation system with magnetorheological energy absorbers (MREAs). The proposed control method can make the payload stop at the end of the available MREA stroke with the lowest maximum deceleration, which does not exceed the deceleration threshold value and lead to the lowest occupant injury probability. The shock mitigation system controlled by MMD will experience constant deceleration control stage and maximum damping force control stage while making full use of the available MREA stroke. The comparative performance of the MMD control method with Bingham number (BN) control, constant deceleration (CD) control and minimum duration deceleration exposure (MDDE) control is shown. Then, the controllable drop velocity range and the required maximum MREA controllable damping force range of MMD control method is calculated. Subsequently, the optimal control method selection criterion among BN control method, CD control method and MMD control method is developed. Finally, the optimal selection criterion is applied to the drop induced shock mitigation system with varying payload velocity, payload mass (occupant type) and the maximum controllable damping force of MREA.

A self-testing kits distribution intervention improved HIV testing of men who have sex with men and their sexual partners: a 12-month stepped-wedge randomized controlled trial in China (Preprint)

BACKGROUND Men who have sex with men (MSM) have high HIV incidence and prevalence burdens but relatively low HIV testing rates. Literature showed insufficient evidence on the efficacy of Social media (WeChat) based HIV self-testing (HIVST) kits distribution approaches among MSM and their sexual partners. OBJECTIVE This study evaluated an Social media (WeChat) based HIVST distribution intervention in increasing HIV testing uptake among MSM and their sexual partners in China. METHODS The study used a 12-month stepped-wedge randomized controlled trial design and an online survey mode. MSM who were HIV-negative or with unknown HIV status were recruited through social media marketing and outreaches by community opinion leaders between August and December 2018 in Shenyang, Beijing, Chongqing, and Shenzhen. Participants were randomly allocated to four groups, which were intervened by sequence. The intervention group received free HIVST kits on top of the control condition that participants received standard education and care. Participants were followed up from January to December 2019. Generalized linear models were used to assess HIV testing coverage and frequency difference between the intervention and the control stage. RESULTS Each group enrolled 140 eligible MSM with a total of 560 participants. Twelve participants were diagnosed with HIV infections, and 4 of them were in the follow-up period. Participants in the intervention stage were nine times likely to receive an HIV test than in the control stage (85.6% vs. 39.2%, risk ratio [RR]=9.21, 95%CI 5.92~14.33). The intervention also increased participants’ HIV testing frequency (1.48 vs. 0.59 times, risk difference [RD]=0.89, 95%CI 0.82~0.96). Moreover, the intervention increased HIV testing proportion (49.0% vs. 30.1%, RR=2.23, 95%CI 1.46~3.40) and mean HIV testing frequency (0.69 vs. 0.35 times, RD=0.26, 95%CI 0.15~0.38) among participants sexual partners who received HIVST through secondary distribution. CONCLUSIONS An Social media (WeChat) based HIVST distribution intervention effectively increased HIV testing coverage and frequency in MSM and their sexual partners. Future programs could apply this approach in HIV education and testing efforts to reduce HIV incidence. CLINICALTRIAL This study was registered at the Chinese Clinical Trials website (http://www.chictr.org.cn/index.aspx) with the registration tracking number of ChiCTR1800019453 on November 12, 2018. INTERNATIONAL REGISTERED REPORT RR2-10.2196/17788

A Semianalytical Production Prediction Model and Dynamics Performance Analysis for Shale Gas Wells

The shale gas productivity model based on shale gas nonlinear seepage mechanism is an effective way to reasonably predict productivity. The incomplete gas nonlinear effects considered in the current production prediction models can lead to inaccurate production prediction. Based on the conventional five-zone compound flow model, comprehensive gas nonlinearities were considered in the improved compound linear flow model proposed in the paper and a semianalytical solution for productivity was obtained. The reliability of the productivity model was verified by the field data, and then, the 20-year production performance analysis of the gas well was studied. Ultimately, the key influencing factors of the fracture control stage and matrix control stage have been analyzed. Research indicated the following: (1) the EUR predicted by the productivity model is higher than the EUR that the comprehensive nonlinear effects are not considered, which demonstrated that the various nonlinear effects cannot be neglected during the production prediction to ensure the greater calculation accuracy; (2) during the early production stage of shale reservoir, the adsorbed gas is basically not recovered, and the cumulative adsorption contribution rate does not exceed 10%. The final adsorption gas contribution rate is 23.28%, and the annual adsorption rate can exceed 50% in the 20th year, showing that free gas and adsorbed gas are, respectively, important sources of the early stage of production and long-term stable production; (3) the widely ranged three-dimensional fracturing reformation of shale reservoirs and reasonable bottom hole pressure in the later matrix development process should be implemented to increase the effective early production of the reservoir and ensure the earlier gas production process of the matrix development. The findings of this study can help for better ensuring the prediction accuracy of the estimated ultimate recovery and understanding the main influencing factors of the dynamic performance of gas wells so as to provide a theoretical reference for production optimization and development plan formulation of the shale gas reservoirs.

Experimental Study of the Quantification of Indocyanine Green Fluorescence in Ischemic and Non-Ischemic Anastomoses, Using the Sergreen Software Program

Background: Tissue ischemia is a key risk factor for anastomotic leakage (AL). Indocyanine green (ICG) is widely used in colorectal surgery to define the segments with the best vascularization. In an experimental model, we present a new system for quantifying ICG saturation, SERGREEN software.Methods: This was a controlled experimental study with eight pigs. In the initial control stage, ICG saturation was analyzed at the level of two anastomoses in the right and left colon. Control images of the two segments were taken after ICG administration. The images were processed with the SERGREEN program. Then, in the experimental ischemia stage, the inferior mesenteric artery was sectioned at the level of the anastomosis of the left colon. Fifteen minutes after the section, sequential images of the two anastomoses were taken every 30’ for the following 2 h.Results: At the control stage, the mean scores were 134.2 (95% CI: 116.3-152.2) for the right colon and 147 (95% CI: 134.7-159.3) for the left colon (p = 0.174). The right colon remained stable throughout the experiment. In the left colon, saturation fell by 47.9 points with respect to the preischemia value (p <0.01). After the first postischemia determination, the values of the ischemic left colon remained stable throughout the experiment. The relative decrease in ICG saturation of the ischemic left colon was 32.6%.Conclusions: The SERGREEN program quantifies ICG saturation in normal and ischemic situations and detects differences between them. A reduction in ICG saturation of 32.6% or more was correlated with complete tissue ischemia.

THE EFFECTIVENESS OF THE SYSTEM FOR ASSESSING THE FUNCTIONAL LITERACY OF STUDENTS IN COMPUTER SCIENCE BASED ON THE CRITERION APPROACH

In the article, based on the criterion approach, the effectiveness of the system for assessing the functional literacy of students in computer science is proved by conducting a pedagogical experiment and conclusions are drawn about the results of practical work. The teaching experience consists of three stages. At the ascertaining stage, it was revealed to what extent the assessment tasks in computer science were compiled taking into account the directions of functional literacy of students. At the applicant stage, tasks and tasks were developed to assess the functional literacy of students based on a criteria-based approach in computer science. At the educational and control stage, the influence of the functional literacy assessment system, developed on the basis of a criterion approach in computer science, on the quality of knowledge and the methodology of teaching students in computer science was revealed. In addition, during the practice, the educational and methodological manual "tasks for assessing the functional literacy of students in computer science" was used. During the training in the control groups, the assessment of knowledge was carried out according to the traditional method, and in the experimental groups on tasks to assess the functional literacy of students based on the criteria approach.

A Study of the Influence of New Generation Granulated Sorbents on the Processes Regulating the Aggregate State of the Blood with the Use of Piezoelectric Thromboelastography

Background: Gastroduodenal bleeding (GDB) and the improvement of endoscopic hemostasis (EH) remain a priority in emergency surgery. This article presents the results of an experimental study of the effects of granular sorbents (Aseptisorb, Aseptisorb-A, Aseptisorb-D) on the system regulating the aggregate state of the blood using modern capabilities of piezoelectric thromboelastography (TEG). Methods and Results: The study involved 12 healthy volunteers (9/75% men and 3/25% women) aged between 18 and 58 years, with the average age of 34.0(26.0;44.0) years. For the study, the blood of healthy volunteers with normal indicators of the system regulating the aggregate state of the blood was used. In vitro experiments: Several tests were performed with the blood of each volunteer. In the first experiment (the control stage), the blood cuvette did not contain the test material. At the second stage of the experiment, the hemostatic properties of new generation granulated sorbents (Aseptisorb, Aseptisorb-A, and Aseptisorb-D) were studied. Experimental studies have shown that the use of granular sorbents Aseptisorb, Aseptisorb-A, and Aseptisorb-D in varying degrees affects the links of platelet and coagulation hemostasis, providing acceleration of thrombosis processes while increasing the maximum density of the clot. These effects determine the effectiveness of the clinical use of these sorbents to stop various types of bleeding. Conclusion: Experimental studies of the effect of granular sorbents on the system regulating the aggregate state of the blood using piezoelectric TEG have shown that the use of Aseptisorb, Aseptisorb-A, and Aseptisorb-D can significantly reduce the time of blood clotting and increase the maximum clot density, which determines the possibility of the use of these sorbents in endoscopic hemostatic treatment for GDB.