Policy gradient optimization of controllers for natural dynamic mono-pedal gait

2020 ◽  
Vol 15 (3) ◽  
pp. 036010
Author(s):  
Israel Schallheim ◽  
Miriam Zacksenhouse
2020 ◽  
Vol 167 ◽  
pp. 107329 ◽  
Author(s):  
Shiyang Yan ◽  
Yuan Xie ◽  
Fangyu Wu ◽  
Jeremy S. Smith ◽  
Wenjin Lu ◽  
...  

2010 ◽  
Vol 2010 ◽  
pp. 1-20 ◽  
Author(s):  
Liang Tang ◽  
Hong-sheng Xi ◽  
Jin Zhu ◽  
Bao-qun Yin

A mathematical model forM/G/1-type queueing networks with multiple user applications and limited resources is established. The goal is to develop a dynamic distributed algorithm for this model, which supports all data traffic as efficiently as possible and makes optimally fair decisions about how to minimize the network performance cost. An online policy gradient optimization algorithm based on a single sample path is provided to avoid suffering from a “curse of dimensionality”. The asymptotic convergence properties of this algorithm are proved. Numerical examples provide valuable insights for bridging mathematical theory with engineering practice.


2019 ◽  
Vol 21 (2) ◽  
pp. 745-754
Author(s):  
Otávio Augusto de Oliveira Lima Barra ◽  
Fábio Perdigão Vasconcelos ◽  
Danilo Vieira dos Santos ◽  
Adely Pereira Silveira

O Brasil é um país com uma extensa linha de costa, são cerca de 7.367 km de extensão do seu litoral, com um potencial natural para a geração de energia eólica. O estado do Ceará é um dos maiores produtores de energia eólica para o país, obtendo notoriedade e a necessidade de manutenção dos seus parques eólicos, especialmente se instalados em zonas de costa, onde há uma grande dinâmica natural. O presente trabalho, busca o acompanhamento das dinâmicas morfológicas na praia de Volta do Rio, localizada em Acaraú/CE, que fica a cerca de 238 km de Fortaleza/CE. Os dados coletados em idas à campo, constataram que há um forte processo erosivo atuante na praia de Volta do Rio, o que alerta para a contenção do avanço marinho sob o parque eólico presente no local. A erosão é um fenômeno natural que trabalha na modelação de demasiadas formas terrestres. No litoral, isso não é diferente, por ser um ambiente altamente dinâmico onde há a interação entre continente, atmosfera e oceano, sendo possível encontrar diversos atuantes que podem intensificar os processos erosivos, sejam eles o vento, maré, ou por intervenções humanas, como construções e ocupações indevidas ao longo da linha de costa.Palavras Chave: Volta do Rio; Energia Eólica; Erosão. ABSTRACTBrazil is a country with an extensive coastline, about 7,367 km of coastline, with a natural potential for wind power generation. The state of Ceará is one of the largest producers of wind energy for the country, obtaining notoriety and required maintenance of its wind farms, especially if located in coastal areas, where there is a great natural dynamic. The present work seeks the movement of morphological dynamics in the beach of Volta do Rio, located in Acaraú/CE, which is about 238 km from Fortaleza/CE. The data collected in the field found that there is a strong erosive process on the Beach of Volta do Rio, which warns about the expansion of advanced marine on the wind farm present on site. Erosion is a natural phenomenon that works in the modeling of many hearth forms. On the coast, this is not different, considering a highly dynamic environment in which there is an interaction between continent, atmosphere and ocean, being possible to find many factors that can intensify the erosive processes, such as wind, tide, or human intervention, as constructions and improper occupations along the coast line.Key words: Volta do Rio; Wind Energy; Erosion. RESUMENBrasil es un país con una extensa costa, cerca de 7.367 km de costa, con un potencial natural para la generación de energía eólica. El estado del Ceará es uno de los mayores productores de energía eólica del país, ganando notoriedad y la necesidad de mantener sus parques eólicos, especialmente si está instalado en zonas costeras, donde existe una gran dinámica natural. La presente investigación tiene como objetivo monitorear la dinámica morfológica en la playa de Vuelta del Rio, ubicada en Acaraú / CE, que está a unos 238 km de Fortaleza / CE. Los datos recopilados en los viajes de campo, encontraron que hay un fuerte proceso erosivo en la playa de Vuelta del Rio, que advierte sobre la contención del avance marino bajo el parque eólico presente en el sitio. La erosión es un fenómeno natural que funciona en el modelado de muchas formas terrestres. En la costa, esto no es diferente, ya que es un entorno altamente dinámico donde existe la interacción entre el continente, la atmósfera y el océano, permitiendo encontrar varios actores que pueden intensificar los procesos erosivos, ya sea viento, marea o intervenciones humanas, como edificios y ocupaciones inadecuadas a lo largo de la costa.Palabras clave: Vuelta del Río; Energía Eólica; Erosión.


2021 ◽  
Vol 9 (3) ◽  
pp. 252
Author(s):  
Yushan Sun ◽  
Xiaokun Luo ◽  
Xiangrui Ran ◽  
Guocheng Zhang

This research aims to solve the safe navigation problem of autonomous underwater vehicles (AUVs) in deep ocean, which is a complex and changeable environment with various mountains. When an AUV reaches the deep sea navigation, it encounters many underwater canyons, and the hard valley walls threaten its safety seriously. To solve the problem on the safe driving of AUV in underwater canyons and address the potential of AUV autonomous obstacle avoidance in uncertain environments, an improved AUV path planning algorithm based on the deep deterministic policy gradient (DDPG) algorithm is proposed in this work. This method refers to an end-to-end path planning algorithm that optimizes the strategy directly. It takes sensor information as input and driving speed and yaw angle as outputs. The path planning algorithm can reach the predetermined target point while avoiding large-scale static obstacles, such as valley walls in the simulated underwater canyon environment, as well as sudden small-scale dynamic obstacles, such as marine life and other vehicles. In addition, this research aims at the multi-objective structure of the obstacle avoidance of path planning, modularized reward function design, and combined artificial potential field method to set continuous rewards. This research also proposes a new algorithm called deep SumTree-deterministic policy gradient algorithm (SumTree-DDPG), which improves the random storage and extraction strategy of DDPG algorithm experience samples. According to the importance of the experience samples, the samples are classified and stored in combination with the SumTree structure, high-quality samples are extracted continuously, and SumTree-DDPG algorithm finally improves the speed of the convergence model. Finally, this research uses Python language to write an underwater canyon simulation environment and builds a deep reinforcement learning simulation platform on a high-performance computer to conduct simulation learning training for AUV. Data simulation verified that the proposed path planning method can guide the under-actuated underwater robot to navigate to the target without colliding with any obstacles. In comparison with the DDPG algorithm, the stability, training’s total reward, and robustness of the improved Sumtree-DDPG algorithm planner in this study are better.


Sign in / Sign up

Export Citation Format

Share Document