Quadrotor Motion Control Using Deep Reinforcement Learning

We present a deep neural net-based controller trained by a model-free reinforcement learning (RL) algorithm to achieve hover stabilization for a quadrotor unmanned aerial vehicle (UAV). With RL, two neural nets are trained. One neural net is used as a stochastic controller which gives the distribution of control inputs. The other maps the UAV state to a scalar which estimates the reward of the controller. A proximal policy optimization (PPO) method, which is an actor-critic policy gradient approach, is used to train the neural nets. Simulation results show that the trained controller achieves a comparable level of performance to a manually-tuned PID controller, despite not depending on any model information. The paper considers different choices of reward function and their influence on controller performance.

An approximate derivate-based controller for regulating gene expression

Inside individual cells, protein population counts are subject to molecular noise due to low copy numbers and the inherent probabilistic nature of biochemical processes. Such random fluctuations in the level of a protein critically impact functioning of intracellular biological networks, and not surprisingly, cells encode diverse regulatory mechanisms to buffer noise. We investigate the effectiveness of proportional and derivative-based feedback controllers to suppress protein count fluctuations originating from two noise sources: bursty expression of the protein, and external disturbance in protein synthesis. Designs of biochemical reactions that function as proportional and derivative controllers are discussed, and the corresponding closed-loop system is analyzed for stochastic controller realizations. Our results show that proportional controllers are effective in buffering protein copy number fluctuations from both noise sources, but this noise suppression comes at the cost of reduced static sensitivity of the output to the input signal. Next, we discuss the design of a coupled feedforward-feedback biochemical circuit that approximately functions as a derivate controller. Analysis using both analytical methods and Monte Carlo simulations reveals that this derivative controller effectively buffers output fluctuations from bursty stochastic expression, while maintaining the static input-output sensitivity of the open-loop system. As expected, the derivative controller performs poorly in terms of rejecting external disturbances. In summary, this study provides a systematic stochastic analysis of biochemical controllers, and paves the way for their synthetic design and implementation to minimize deleterious fluctuations in gene product levels.