Deep reinforcement learning based control for Autonomous Vehicles in CARLA

Nowadays, Artificial Intelligence (AI) is growing by leaps and bounds in almost all fields of technology, and Autonomous Vehicles (AV) research is one more of them. This paper proposes the using of algorithms based on Deep Learning (DL) in the control layer of an autonomous vehicle. More specifically, Deep Reinforcement Learning (DRL) algorithms such as Deep Q-Network (DQN) and Deep Deterministic Policy Gradient (DDPG) are implemented in order to compare results between them. The aim of this work is to obtain a trained model, applying a DRL algorithm, able of sending control commands to the vehicle to navigate properly and efficiently following a determined route. In addition, for each of the algorithms, several agents are presented as a solution, so that each of these agents uses different data sources to achieve the vehicle control commands. For this purpose, an open-source simulator such as CARLA is used, providing to the system with the ability to perform a multitude of tests without any risk into an hyper-realistic urban simulation environment, something that is unthinkable in the real world. The results obtained show that both DQN and DDPG reach the goal, but DDPG obtains a better performance. DDPG perfoms trajectories very similar to classic controller as LQR. In both cases RMSE is lower than 0.1m following trajectories with a range 180-700m. To conclude, some conclusions and future works are commented.

A Random Traffic Assignment Model for Networks Based on Discrete Dynamic Bayesian Algorithms

In this paper, a stochastic traffic assignment model for networks is proposed for the study of discrete dynamic Bayesian algorithms. In this paper, we study a feasible method and theoretical system for implementing traffic engineering in networks based on Bayesian algorithm theory. We study the implementation of traffic assignment engineering in conjunction with the network stochastic model: first, we study the Bayesian algorithm theoretical model of control layer stripping in the network based on the discrete dynamic Bayesian algorithm theory and analyze the resource-sharing mechanism in different queuing rules; second, we study the extraction and evaluation theory of traffic assignment for the global view obtained by the control layer of the network and establish the Bayesian algorithm analysis model based on the traffic assignment; subsequently, the routing of bandwidth guarantee and delay guarantee in the network is studied based on Bayesian algorithm model and Bayesian algorithm network random traffic allocation theory. In this paper, a Bayesian algorithm estimation model based on Bayesian algorithm theory is constructed based on network random observed traffic assignment as input data. The model assumes that the roadway traffic distribution follows the network random principle, and based on this assumption, the likelihood function of the roadway online traffic under the network random condition is derived; the prior distribution of the roadway traffic is derived based on the maximum entropy principle; the posterior distribution of the roadway traffic is solved by combining the likelihood function and the prior distribution. The corresponding algorithm is designed for the model with roadway traffic as input, and the reliability of the algorithm is verified in the arithmetic example.

A Hierarchical Structure Control Strategy Based on MPC for a Six-DOF Flexible Joint Manipulator

The six-degree-of-freedom flexible joint manipulator is a complex system that suffers from the problem that the trajectory planning results are inconsistent with the control results. To keep the planned trajectory within the control range of the manipulator, a hierarchical structure control strategy is designed, which consists of a trajectory planning layer, a model predictive control layer, and a bottom control layer. Specifically, first, the target joint angles are obtained by a time-optimal trajectory planning algorithm based on a genetic algorithm in the trajectory planning layer. Second, in the model predictive control layer, considering the system physical constraints, the model predictive controller is adopted to provide the set points for the Proportion-Differentiation (PD) controllers. Finally, in the bottom control layer, the manipulator moves along the target trajectory under the PD controllers with the feedback control law. The simulation results show that, compared with the PD control strategy, the hierarchical structure control strategy can achieve better control performance and reduce the tracking error of the terminal trajectory by 33.70%.

Education Exchange Storage Protocol: Transformation Into Decentralized Learning Platform

The current micro-teaching process is readily online, and it is functional anywhere and anytime ubiquitously. All or most teaching and learning activities are accessible in centralized storage. However, centralized storage has inherent problems, such as a single point of failure with many possible data breaches, much duplication of data stored repeatedly in one location, and the lack of trust in third parties for data stored in centralized storage. Further issues include the high cost and low performance of the online systems that hinder the quality of the education process. In this paper, we propose a new framework Education Exchange Storage Protocol (EESP). EESP aims to improve the efficiency of the decentralized storage ecosystem in micro-teaching, coupled with blockchain technology acting as a control layer. Blockchain empowers the decentralized system by bringing together the most incompatible unstructured entities and integrate them. The decentralized storage system is armed with a blockchain smart contract that acts as a control layer, featuring impregnable security, immutability, trace-ability, and transparency. The EESP framework aims to elevate teaching and learning through blockchain decentralized storage systems in a transformational way, including but not limited to things like micro-credential, massive open online courses, and gamification, all in a single immersive learning platform. Finally, we tested and evaluated this framework using the truffle simulator, and the results demonstrate that the EESP model significantly improves performance.

The Application of Manufacturing Execution System in Virtual Manufacturing

Compaired with mart manufacturing and digital manufacturing, virtual manufacturing is a more advanced mode, which is more flexible, more inexpensive and more suitable for modern competitive society. No matter what type of manufacturing, Manufacturing Execution System (MES) is necessary and plays a key role. So this paper focuses on the MES in virtual manufacturing. MES serves as a bridge to connect the upper planning layer and the control layer of the factory. It has at least 8 functions, including data collection, production process management, human resource management, workpieces tracking, production planning and scheduling, quality control, documentation system and maintenance management. As a typical virtual manufacturing enterprise, the company A is chosen to be introduced, including the background, composition of MES and implementation of MES.

A Zigbee Based Framework for Low Power Voice Communication System

IEEE standard 802.15.4 defines two layers for low rate WPANs, which are physical layer and media access control layer, which restrict the data rate to 250 kbps. This alliance took low level PHY and MAC layers as the base for the development of the network protocol, security and application for the Zigbee. To solve the Problem of reduce cost and power consumption to make this technology easily available for common crowd. This system shows the total working details of the network, i.e., wireless voice communication. This work has been done to reduce the cost of entire production of such devices that contributes in home automation system and similar projects. The main objective of the research was to transfer voice over low-power micro-controller such as 8-bit micro-controller by the implementation of Zigbee.

An Enhanced Distributed Voltage Regulation Scheme for Radial Feeder in Islanded Microgrid

Even the simplest version of the distribution networks face challenges such as maintaining load voltage and system frequency stability and at the same time minimizing the circulating reactive power in grid-forming nodes. As the consumers at the far end of the radial distribution network face serious voltage fluctuations and deviations once the load varies. Therefore, this paper presents an enhanced distributed control strategy to restore the load voltage magnitude and to realize power-sharing proportionally in islanded microgrids. This proposed study considers the voltage regulation at the load node as opposed to the inverter terminal. At the same time, a supervisory control layer is put on to observe and correct the load voltage and system frequency deviations. This presented method is aimed at replacing paralleled inverter control methods hitherto used. Stability analysis using system-wide methodical small-signal models, the MATLAB/Simulink, and experimental results obtained with conventional and proposed control schemes verify the effectiveness of the proposed methodology.