Path Following Control for UAV Using Deep Reinforcement Learning Approach

Unmanned aerial vehicles (UAVs) have been extensively used in civil and industrial applications due to the rapid development of the guidance, navigation and control (GNC) technologies. Especially, using deep reinforcement learning methods for motion control acquires a major progress recently, since deep [Formula: see text]-learning algorithm has been successfully applied to the continuous action domain problem. This paper proposes an improved deep deterministic policy gradient (DDPG) algorithm for path following control problem of UAV. A specific reward function is designed for minimizing the cross-track error of the path following problem. In the training phase, a double experience replay buffer (DERB) is used to increase the learning efficiency and accelerate the convergence speed. First, the model of UAV path following problem has been established. After that, the framework of DDPG algorithm is constructed. Then the state space, action space and reward function of the UAV path following algorithm are designed. DERB is proposed to accelerate the training phase. Finally, simulation results are carried out to show the effectiveness of the proposed DERB–DDPG method.

Path Planning of Coastal Ships Based on Optimized DQN Reward Function

Path planning is a key issue in the field of coastal ships, and it is also the core foundation of ship intelligent development. In order to better realize the ship path planning in the process of navigation, this paper proposes a coastal ship path planning model based on the optimized deep Q network (DQN) algorithm. The model is mainly composed of environment status information and the DQN algorithm. The environment status information provides training space for the DQN algorithm and is quantified according to the actual navigation environment and international rules for collision avoidance at sea. The DQN algorithm mainly includes four components which are ship state space, action space, action exploration strategy and reward function. The traditional reward function of DQN may lead to the low learning efficiency and convergence speed of the model. This paper optimizes the traditional reward function from three aspects: (a) the potential energy reward of the target point to the ship is set; (b) the reward area is added near the target point; and (c) the danger area is added near the obstacle. Through the above optimized method, the ship can avoid obstacles to reach the target point faster, and the convergence speed of the model is accelerated. The traditional DQN algorithm, A* algorithm, BUG2 algorithm and artificial potential field (APF) algorithm are selected for experimental comparison, and the experimental data are analyzed from the path length, planning time, number of path corners. The experimental results show that the optimized DQN algorithm has better stability and convergence, and greatly reduces the calculation time. It can plan the optimal path in line with the actual navigation rules, and improve the safety, economy and autonomous decision-making ability of ship navigation.

Shaping physical, social and imaginary spaces in healthcare design labs

The use of service design to support healthcare innovation has increased over the past decade. Recently, a growing number of design labs have been established to facilitate service design processes inside healthcare organizations. There is a growing need to gain a deeper understanding of how to set up and work within these spaces so that they live up to their promise of healthcare innovation and do not become a hype that fades out over time. Despite a growing body of literature on design labs, little attention has been given to the role of the lab space and how space may be ‘made use of’ to support healthcare service design. To examine the practice of making use of space, action research was conducted by embedding a design lab inside a hospital. Through empirical observations, we unpack three spatial dimensions that are made use of inside healthcare service design labs: (1) physical spaces supporting sensemaking and promoting innovation as culture; (2) social spaces facilitating and encouraging interactions among stakeholders; and (3) imaginary spaces challenging mental models and shaping propositions collaboratively. This extended understanding of lab space challenges existing research priorities, suggesting practical implications for using space more purposefully within design labs.

The World Is Not Enough: A Political Dark Ecology

Timothy Morton’s dark ecology is positioned as an aesthetic and ethical study which is far removed from political theory. Although Morton touches upon actual political crises connected with global warming and on climate change skepticism and also deploys such fundamental concepts of political philosophy as territory, space, action and solidarity, he describes his approach as ontological rather than political. The author finds that dark ecology’s own foundations have been inherited from political theory. However, that does not make it inconsistent; on the contrary, under the right conditions it enables a different apprehension of both ecology and political philosophy. The author asks how politics would proceed in a world of uncertainty and proposes viewing Morton’s theory as a treatise on a political theory that addresses the problem of collective action. This is the main concept of any political philosophy out of which its description of the social order is constructed. Dark ecology denies any possibility of action by emphasizing uncertainty and the impossibility of predicting an action’s consequences, and this opens up new possibilities for conceptualizing action. It is precisely uncertainty that permits segregating action from the guilt that motivated Morton to divorce dark ecology from political philosophy. This is a narrative about the transformation of Morton the emancipator into a law-giver, about how political theory has evolved in parallel with the onset of the Anthropocene, about what will happen to the Leviathan in the age of global warming, and how to change the perception of the political from ontological categories to ethical ones.

Space, Action, and Images

The worlds of ancient Greece and Rome are characterized by a high degree of visuality of their social spaces. A theoretical fundament is laid out by the distinction between experienced and conceptual space, in interrelation with human actions. Fundamental is cultural practice, religious rituals, and political activities in interrelation with public architecture and urban spaces. Social life, as it developed in visually marked spaces, is exemplified in a concentric sequence: civic sanctuaries and agorai/fora, city areas and territories, empires, and liminal zones, comparing different concepts between Greece and Rome.

Dynamic Spectrum Allocation Using Q-Learning in Cognitive Radio Systems

In this paper we present an improved dynamic spectrum allocation algorithm based on the intelligence of Q-learning. The state space, action space and reward function of the algorithm are built, and, the agents are guided to perform actions through designing the reward function. Numerical simulation results show that the proposed algorithm can improve system throughput efficiently compared to other algorithms. Facing the status of spectrum resources is tension and spectrum utilization is low, it can also boost the spectrum using condition in the future.