Reinforcement learning-based dynamic position control of mobile node for ocean sensor networks

2021 ◽  
pp. 014233122110430
Author(s):  
Weijun Wang ◽  
Huafeng Wu ◽  
Xianglun Kong ◽  
Yuanyuan Zhang ◽  
Yang Ye ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Sensor Networks ◽  
Position Control ◽  
Response Speed ◽  
Fast Response ◽  
Mobile Node ◽  
Position Estimation ◽  
Mobile Nodes ◽  
Control Scheme ◽  
Ocean Sensor Networks

In this paper, a novel dynamic position control (PC) approach for mobile nodes (MNs) is proposed for ocean sensor networks (OSNs) which directly utilizes a neural network to represent a PC strategy. The calculation of position estimation no longer needs to be carried out in the proposed scheme, so the localization error is eliminated. In addition, reinforcement learning is used to train the PC strategy, so that the MN can learn a more highly accurate and fast response control strategy. Moreover, to verify its applicability to the real-world environment, we conducted field experiment deployment in OSNs consisting of a MN designed by us and some fixed nodes. The experimental results demonstrate the effectiveness of our proposed control scheme with impressive improvements on PC accuracy by more than 53% and response speed by more than 15%.

scholarly journals Cooperative Search and Rescue with Artificial Fishes Based on Fish-Swarm Algorithm for Underwater Wireless Sensor Networks

2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Wei Zhao ◽  
Zhenmin Tang ◽  
Yuwang Yang ◽  
Lei Wang ◽  
Shaohua Lan
Keyword(s):  
Sensor Networks ◽  
Effective Properties ◽  
Limited Range ◽  
Mobile Node ◽  
Joint Detection ◽  
Mobile Nodes ◽  
Control Approach ◽  
Control Scheme ◽  
Detection Probabilities ◽  
Swarm Algorithm

This paper presents a searching control approach for cooperating mobile sensor networks. We use a density function to represent the frequency of distress signals issued by victims. The mobile nodes’ moving in mission space is similar to the behaviors of fish-swarm in water. So, we take the mobile node as artificial fish node and define its operations by a probabilistic model over a limited range. A fish-swarm based algorithm is designed requiring local information at each fish node and maximizing the joint detection probabilities of distress signals. Optimization of formation is also considered for the searching control approach and is optimized by fish-swarm algorithm. Simulation results include two schemes: preset route and random walks, and it is showed that the control scheme has adaptive and effective properties.

Improving Area Coverage With Mobile Nodes in Wireless Sensor Networks

2021 ◽  
Vol 13 (1) ◽  
pp. 36-48
Author(s):  
Dhruvi Patel ◽  
Arunita Jaekel
Keyword(s):  
Wireless Sensor Networks ◽  
Sensor Networks ◽  
Sensor Node ◽  
Mobile Node ◽  
Sensor Nodes ◽  
Wireless Sensor ◽  
Mobile Nodes ◽  
Area Of Interest ◽  
Adequate Coverage ◽  
Considerable Work

Wireless sensor networks (WSN) consist of sensor nodes that detect relevant events in their vicinity and relay this information for further analysis. Considerable work has been done in the area of sensor node placement to ensure adequate coverage of the area of interest. However, in many applications it may not be possible to accurately place individual sensor nodes. In such cases, imprecise placement can result in regions, referred to as coverage holes, that are not monitored by any sensor node. The use of mobile nodes that can ‘visit' such uncovered regions after deployment has been proposed in the literature as an effective way to maintain adequate coverage. In this paper, the authors propose a novel integer linear programming (ILP) formulation that determines the paths the mobile node(s) should take to realize the specified level of coverage in the shortest time. The authors also present a heuristic algorithm that can be used for larger networks.

scholarly journals Research on the Difficulty of Mobile Node Deployment’s Self-Play in Wireless Ad Hoc Networks Based on Deep Reinforcement Learning

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Huitao Wang ◽  
Ruopeng Yang ◽  
Changsheng Yin ◽  
Xiaofei Zou ◽  
Xuefeng Wang
Keyword(s):  
Reinforcement Learning ◽  
Ad Hoc Networks ◽  
Wireless Ad Hoc Networks ◽  
Ad Hoc ◽  
Mobile Node ◽  
Machine Learning Algorithms ◽  
Mobile Nodes ◽  
Sample Data ◽  
Hoc Networks ◽  
Dynamic Updating

Deep reinforcement learning is one kind of machine learning algorithms which uses the maximum cumulative reward to learn the optimal strategy. The difficulty is how to ensure the fast convergence of the model and generate a large number of sample data to promote the model optimization. Using the deep reinforcement learning framework of the AlphaZero algorithm, the deployment problem of wireless nodes in wireless ad hoc networks is equivalent to the game of Go. A deployment model of mobile nodes in wireless ad hoc networks based on the AlphaZero algorithm is designed. Because the application scenario of wireless ad hoc network does not have the characteristics of chessboard symmetry and invariability, it cannot expand the data sample set by rotating and changing the chessboard orientation. The strategy of dynamic updating learning rate and the method of selecting the latest model to generate sample data are used to solve the problem of fast model convergence.

scholarly journals Model-Free Attitude Control of Spacecraft Based on PID-Guide TD3 Algorithm

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
ZhiBin Zhang ◽  
XinHong Li ◽  
JiPing An ◽  
WanXin Man ◽  
GuoHui Zhang
Keyword(s):  
Reinforcement Learning ◽  
High Precision ◽  
Attitude Control ◽  
Response Speed ◽  
Fast Response ◽  
Fine Tuning ◽  
Training Time ◽  
Model Free ◽  
Attitude Tracking ◽  
Policy Gradient

This paper is devoted to model-free attitude control of rigid spacecraft in the presence of control torque saturation and external disturbances. Specifically, a model-free deep reinforcement learning (DRL) controller is proposed, which can learn continuously according to the feedback of the environment and realize the high-precision attitude control of spacecraft without repeatedly adjusting the controller parameters. Considering the continuity of state space and action space, the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm based on actor-critic architecture is adopted. Compared with the Deep Deterministic Policy Gradient (DDPG) algorithm, TD3 has better performance. TD3 obtains the optimal policy by interacting with the environment without using any prior knowledge, so the learning process is time-consuming. Aiming at this problem, the PID-Guide TD3 algorithm is proposed, which can speed up the training speed and improve the convergence precision of the TD3 algorithm. Aiming at the problem that reinforcement learning (RL) is difficult to deploy in the actual environment, the pretraining/fine-tuning method is proposed for deployment, which can not only save training time and computing resources but also achieve good results quickly. The experimental results show that DRL controller can realize high-precision attitude stabilization and attitude tracking control, with fast response speed and small overshoot. The proposed PID-Guide TD3 algorithm has faster training speed and higher stability than the TD3 algorithm.

RNST

2011 ◽  
pp. 230-257
Author(s):  
Guangjie Han ◽  
Wen Shen ◽  
Chuan Zhu ◽  
Lei Shu ◽  
Joel J.P.C. Rodrigues
Keyword(s):  
Sensor Networks ◽  
Real Time ◽  
Resource Limitation ◽  
Mobile Node ◽  
Sensor Nodes ◽  
Localization Error ◽  
Mobile Nodes ◽  
Two Phase ◽  
Position Information ◽  
The Real

The key problem of location service in indoor sensor networks is to quickly and precisely acquire the position information of mobile nodes. Due to resource limitation of the sensor nodes, some of the traditional positioning algorithms, such as Two-Phase Positioning (TPP) algorithm, are too complicated to be implemented and they can not provide the real-time localization of the mobile node. We analyze the localization error, which is produced when one tries to estimate the mobile node using trilateration method in the localization process. We draw the conclusion that the localization error is the least when three reference nodes form an equilateral triangle. Therefore, we improve the TPP algorithm and propose Reference Node Selection algorithm based on Trilateration (RNST), which can provide real-time localization service for the mobile nodes. Our proposed algorithm is verified by the simulation experiment. Based on the analysis of the acquired data and comparison with that of the TPP algorithm, we conclude that our algorithm can meet real-time localization requirement of the mobile nodes in an indoor environment, and make the localization error less than that of the traditional algorithm; therefore our proposed algorithm can effectively solve the real-time localization problem of the mobile nodes in indoor sensor networks.

scholarly journals Improved Position Control for an EGR Valve System with Low Control Frequency

Energies ◽  
2020 ◽  
Vol 13 (2) ◽  
pp. 457
Author(s):  
Hyeong-Jin Kim ◽  
Yung-Deug Son ◽  
Jang-Mok Kim
Keyword(s):  
Position Control ◽  
Feedforward Control ◽  
Fast Response ◽  
Control Frequency ◽  
Position Response ◽  
Control Scheme ◽  
Position Controller ◽  
Valve Position ◽  
Feedforward Controller ◽  
Gas Recirculation

An exhaust gas recirculation (EGR) valve position control system requires fast response without overshoot, but the low control frequency limits control bandwidth and results in poor position response. A novel EGR valve position control scheme is proposed to improve the position response at low control frequency. This is based on the feedforward controller, but the feedforward control loop is implemented without the position pattern generator or derivative. The proposed method estimates the acceleration command through the relationship between the position controller output, the speed command and the speed limiter, and compensates the cascaded proportional-proportional integral (P-PI) controller. In this method, many operations are not required and noise due to derivative is not generated. This method can improve the position response without much computation and derivative noise at the low control frequency. Experimental results are presented to verify the feasibility of the proposed position control algorithm.

scholarly journals A Hybrid Mobile Node Localization Algorithm Based on Adaptive MCB-PSO Approach in Wireless Sensor Networks

2020 ◽  
Vol 2020 ◽  
pp. 1-17 ◽  
Author(s):  
Hua Wu ◽  
Ju Liu ◽  
Zheng Dong ◽  
Yang Liu
Keyword(s):  
Wireless Sensor Networks ◽  
Sensor Networks ◽  
Three Dimensional ◽  
Mobile Node ◽  
Global Optimum ◽  
Wireless Sensor ◽  
Localization Algorithm ◽  
Mobile Nodes ◽  
Node Localization ◽  
Localization Accuracy

In this paper, a hybrid adaptive MCB-PSO node localization algorithm is proposed for three-dimensional mobile wireless sensor networks (MWSNs), which considers the random mobility of both anchor and unknown nodes. An improved particle swarm optimization (PSO) approach is presented with Monte Carlo localization boxed (MCB) to locate mobile nodes. It solves the particle degeneracy problem that appeared in traditional MCB. In the proposed algorithm, a random waypoint model is incorporated to describe random movements of anchor and unknown nodes based on different time units. An adaptive anchor selection operator is designed to improve the performance of standard PSO for each particle based on time units and generations, to maintain the searching ability in the last few time units and particle generations. The objective function of standard PSO is then reformed to make it obtain a better rate of convergence and more accurate cost value for the global optimum position. Furthermore, the moving scope of each particle is constrained in a specified space to improve the searching efficiency as well as to save calculation time. Experiments are made in MATLAB software, and it is compared with DV-Hop, Centroid, MCL, and MCB. Three evaluation indexes are introduced, namely, normalized average localization error, average localization time, and localization rate. The simulation results show that the proposed algorithm works well in every situation with the highest localization accuracy, least time consumptions, and highest localization rates.

scholarly journals An Efficient Localization Method for Mobile Nodes in Wireless Sensor Networks

2017 ◽  
Vol 13 (03) ◽  
pp. 160 ◽  
Author(s):  
Zhiyu Qiu ◽  
Lihong Wu ◽  
Peixin Zhang
Keyword(s):  
Wireless Sensor Networks ◽  
Monte Carlo ◽  
Wireless Sensor Network ◽  
Sensor Networks ◽  
Sensor Network ◽  
Mobile Node ◽  
Wireless Sensor ◽  
Mobile Nodes ◽  
Node Localization ◽  
Location Algorithm

<p style="margin: 1em 0px; -ms-layout-grid-mode: char;"><span style="font-family: Times New Roman; font-size: medium;">With the development of electronic technology and communication protocols, wireless sensor network technology is developing rapidly. In a sense, the traditional static wireless sensor network has been unable to meet the needs of new applications. However, the introduction of mobile nodes extends the application of wireless sensor networks, despite the technical challenges. Because of its flexibility, the mobile wireless sensor network has attracted great attention, and even small, self-controlled mobile sensor devices have appeared. At present, mobile node localization has become one of the hotspots in wireless sensor networks. As the storage energy of wireless sensor network nodes is limited, and the communication radius is small, many scientists have focused their research direction on the location algorithm of mobile nodes. According to the continuity principle of mobile node movement, in this paper we propose an improved mobile node localization algorithm based on the Monte Carlo Location (MCL) algorithm, and the method can reduce the sampling interval effectively. First of all, this paper introduces the structure and classification of wireless sensor localization technology. Secondly, the principle of the Monte Carlo Location algorithm is described in detail. Thirdly, we propose an efficient method for mobile node localization based on the MCL algorithm. Finally, the effectiveness and accuracy of the new algorithm are verified by comparative analysis.</span></p>

scholarly journals High-Accuracy and Fast-Response Flywheel Torque Control

2014 ◽  
Vol 2014 ◽  
pp. 1-10
Author(s):  
Xinxiu Zhou ◽  
Meng Li ◽  
Ran Zhang
Keyword(s):  
Degrees Of Freedom ◽  
Current Mode ◽  
Response Speed ◽  
Fast Response ◽  
Torque Control ◽  
Recursive Least Squares ◽  
Control Scheme ◽  
Feedforward Controller ◽  
Speed Mode ◽  
Motor Drive System

Compared with current mode flywheel torque controller, speed mode torque controller has superior disturbance rejection capability. However, the speed loop delay reduces system dynamic response speed. To solve this problem, a two-degrees-of-freedom controller (2DOFC) which consists of a feedback controller (FBC) and a command feedforward controller (FFC) is proposed. The transfer function of FFC is found based on the inverse model of motor drive system, whose parameters are identified by recursive least squares (RLS) algorithm in real-time. Upon this, Kalman filter with softening factor is introduced for the improved parameters identification and torque control performances. Finally, the validity and the superiority of the proposed control scheme are verified through experiments with magnetically suspended flywheel (MSFW) motor.

Mobility Adaptive Energy Efficient and Low Latency MAC for Wireless Sensor Networks

2013 ◽  
Vol 4 (2) ◽  
pp. 40-54
Author(s):  
Bilal Muhammad Khan ◽  
Rabia Bilal
Keyword(s):  
Wireless Sensor Networks ◽  
Sensor Networks ◽  
Energy Efficient ◽  
Simulation Analysis ◽  
Mobile Node ◽  
Mac Protocols ◽  
Wireless Sensor ◽  
Low Latency ◽  
Sleep Mode ◽  
Mobile Nodes

In this paper a high throughput, low latency, mobility adaptive and energy efficient medium access protocol (MAC) called Mobility Adaptive (MA) for wireless sensor networks. MA-MAC ensures that transmissions incur no collisions, and allows nodes to undergo sleep mode whenever they are not transmitting or receiving. It uses delay allocation scheme based on traffic priority at each node and avoids allocating same backoff delay for more than one node unless they are in separate clusters. It also allows nodes to determine when they can switch to sleep mode during operation. MA-MAC for mobile nodes provides fast association between the mobile node and the cluster coordinator. The proposed MAC performs well in both static and mobile scenarios, which shows its significance over existing MAC protocols proposed for mobile applications. The performance of MA-MAC is evaluated through extensive simulation, analysis and comparison with other mobility aware MAC protocols. The results show that MA-MAC outperforms significantly the existing CSMA/CA, Sensor Mac (S-MAC), Mobile MAC (MOB-MAC), Mobility aware Delay sensitive MAC (MD-MAC) and Dynamic Sensor MAC (DS-MAC) protocols including throughput, latency and energy consumption.

Sign in / Sign up

Export Citation Format

Share Document