Deep Reinforcement Learning Enabled Network Routing Optimization Approach with an Enhanced DDPG Algorithm

Routing technology is one of the main supporting technologies of wireless sensor networks. Only by using routing algorithm reasonably or finding better routing optimization algorithm, can the function of wireless sensor networks be maximized. Therefore, the research of routing technology for wireless sensor networks has important theoretical and practical significance. Based on the analysis of existing routing protocols in wireless sensor networks, this paper focuses on LEACH protocol.

Download Full-text

Prioritized Sweeping Reinforcement Learning Based Routing for MANETs

Indonesian Journal of Electrical Engineering and Computer Science ◽

10.11591/ijeecs.v5.i2.pp383-390 ◽

2017 ◽

Vol 5 (2) ◽

pp. 383 ◽

Cited By ~ 1

Author(s):

Rahul M Desai ◽

B P Patil

Keyword(s):

Reinforcement Learning ◽

Shortest Path ◽

Ad Hoc ◽

Mobile Ad Hoc Network ◽

Network Routing ◽

Delivery Ratio ◽

Learning Method ◽

Shortest Path Routing ◽

Mobile Ad Hoc ◽

Prioritized Sweeping

<p class="Default">In this paper, prioritized sweeping confidence based dual reinforcement learning based adaptive network routing is investigated. Shortest Path routing is always not suitable for any wireless mobile network as in high traffic conditions, shortest path will always select the shortest path which is in terms of number of hops, between source and destination thus generating more congestion. In prioritized sweeping reinforcement learning method, optimization is carried out over confidence based dual reinforcement routing on mobile ad hoc network and path is selected based on the actual traffic present on the network at real time. Thus they guarantee the least delivery time to reach the packets to the destination. Analysis is done on 50 Nodes Mobile ad hoc networks with random mobility. Various performance parameters such as Interval and number of nodes are used for judging the network. Packet delivery ratio, dropping ratio and delay shows optimum results using the prioritized sweeping reinforcement learning method.</p>

Download Full-text

Using Reinforcement Learning to Estimate Human Joint Moments via EMG Signals or Joint Kinematics: An Alternative Solution to Musculoskeletal-Based Biomechanics

Journal of Biomechanical Engineering ◽

10.1115/1.4049333 ◽

2020 ◽

Author(s):

Wen Wu ◽

Kate Saul ◽

He (Helen) Huang

Keyword(s):

Reinforcement Learning ◽

Correlation Coefficients ◽

Metacarpophalangeal Joint ◽

Joint Moment ◽

Biomechanical Analysis ◽

Model Parameters ◽

Optimization Approach ◽

Joint Moments ◽

Moment Estimation ◽

Policy Optimization

Abstract Reinforcement learning (RL) has potential to provide innovative solutions to existing challenges in estimating joint moments in motion analysis, such as kinematic or electromyography (EMG) noise and unknown model parameters. Here we explore feasibility of RL to assist joint moment estimation for biomechanical applications. Forearm and hand kinematics and forearm EMGs from 4 muscles during free finger and wrist movement were collected from six healthy subjects. Using the Proximal Policy Optimization approach, we trained and tested two types of RL agents that estimated joint moment based on measured kinematics or measured EMGs, respectively. To quantify the performance of RL agents, the estimated joint moment was used to drive a forward dynamic model for estimating kinematics, which were then compared with measured kinematics. The results demonstrated that both RL agents can accurately reproduce wrist and metacarpophalangeal joint motion. The correlation coefficients between estimated and measured kinematics, derived from the kinematics-driven agent and subject-specific EMG-driven agents, were 0.98±0.01 and 0.94±0.03 for the wrist, respectively, and were 0.95±0.02 and 0.84±0.06 for the metacarpophalangeal joint, respectively. In addition, a biomechanically reasonable joint moment-angle-EMG relationship (i.e. dependence of joint moment on joint angle and EMG) was predicted using only 15 seconds of collected data. In conclusion, this study serves as a proof of concept that an RL approach can assist in biomechanical analysis and human-machine interface applications by deriving joint moments from kinematic or EMG data.

Download Full-text

Communication Constellation Design Trimmed by Network Routing

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.433-440.4977 ◽

2012 ◽

Vol 433-440 ◽

pp. 4977-4982

Author(s):

Xiao Yan Chen ◽

Chao Han

Keyword(s):

Time Delays ◽

Design Method ◽

Network Routing ◽

Satellite Network ◽

Communication Service ◽

Routing Optimization ◽

Constellation Design ◽

Routing Performance ◽

Simulation Scenario ◽

Independent Design

Communication constellation design and satellite network routing are conventionally treated as two independent design processes; however, the routing results can actually give opinions to the constellation structure on its communication service quality. Based on this consideration, a new communication constellation design method is proposed in this paper which allows the constellation design to be trimmed by the routing in the way of introducing the routing result as a feedback, aiming to achieve a better constellation structure for the routing. The design method rationale is explained and a simulation scenario is given. With a basic Celestri configuration and opening the inclination to several options, the minimum link handovers and time delays offered by routing optimization on each configuration structure are compared. The simulation results show that by altering the Celestri inclination with a small amount, the constellation can achieve better routing performance, which validates the new design method.

Download Full-text