scholarly journals A better-performing Q-learning game-theoretic distributed routing for underwater wireless sensor networks

2018 ◽  
Vol 14 (1) ◽  
pp. 155014771875472 ◽  
Author(s):  
Sungwook Kim
Keyword(s):  
Sensor Networks ◽  
Simulation Analysis ◽  
Dynamic Structure ◽  
Propagation Delay ◽  
Underwater Sensor Networks ◽  
Performance Simulation ◽  
Q Learning ◽  
Transmission Problems ◽  
Learning Game ◽  
Game Theoretic

Underwater sensor networks have recently emerged as a promising networking technique for various underwater applications. However, the acoustic routing of underwater sensor networks in the aquatic environment presents challenges in terms of dynamic structure, high rates of energy consumption, long propagation delay, and narrow bandwidth. Therefore, it is difficult to adapt traditional routing protocols, which are known to be reliable in terrestrial wireless networks. In this study, we focus on the development of novel routing algorithms to tackle acoustic transmission problems in underwater sensor networks. The proposed scheme is based on reinforcement learning and game theory and is designed as a routing game model to provide an effective packet-forwarding mechanism. In particular, our Q-learning game paradigm captures the dynamics of the underwater sensor networks system in a decentralized, distributed manner. The results of a performance simulation analysis show that the proposed scheme can outperform existing schemes while displaying balanced system performance in terms of energy efficiency and underwater sensor networks throughput.

scholarly journals Packet Flow Based Reinforcement Learning MAC Protocol for Underwater Acoustic Sensor Networks

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2284
Author(s):  
Ibrahim B. Alhassan ◽  
Paul D. Mitchell
Keyword(s):  
Sensor Networks ◽  
Mac Protocol ◽  
Propagation Delay ◽  
Basic Function ◽  
Acoustic Sensor ◽  
Underwater Acoustic Sensor Networks ◽  
Medium Access ◽  
Underwater Acoustic ◽  
Q Learning ◽  
Acoustic Sensor Networks

Medium access control (MAC) is one of the key requirements in underwater acoustic sensor networks (UASNs). For a MAC protocol to provide its basic function of efficient sharing of channel access, the highly dynamic underwater environment demands MAC protocols to be adaptive as well. Q-learning is one of the promising techniques employed in intelligent MAC protocol solutions, however, due to the long propagation delay, the performance of this approach is severely limited by reliance on an explicit reward signal to function. In this paper, we propose a restructured and a modified two stage Q-learning process to extract an implicit reward signal for a novel MAC protocol: Packet flow ALOHA with Q-learning (ALOHA-QUPAF). Based on a simulated pipeline monitoring chain network, results show that the protocol outperforms both ALOHA-Q and framed ALOHA by at least 13% and 148% in all simulated scenarios, respectively.

scholarly journals A Performance Simulation Tool for the Analysis of Data Gathering in Both Terrestrial and Underwater Sensor Networks

IEEE Access ◽  
2017 ◽  
Vol 5 ◽  
pp. 4190-4208 ◽  
Author(s):  
Mukhtar Ghaleb ◽  
Emad Felemban ◽  
Shamala Subramaniam ◽  
Adil A. Sheikh ◽  
Saad Bin Qaisar
Keyword(s):  
Sensor Networks ◽  
Data Gathering ◽  
Underwater Sensor Networks ◽  
Simulation Tool ◽  
Performance Simulation ◽  
A Performance

scholarly journals Receiver-Initiated Handshaking MAC Based on Traffic Estimation for Underwater Sensor Networks ‡

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 3895 ◽  
Author(s):  
Yuan Dong ◽  
Lina Pu ◽  
Yu Luo ◽  
Zheng Peng ◽  
Haining Mo ◽  
...  
Keyword(s):  
Energy Efficiency ◽  
Sensor Networks ◽  
Propagation Delay ◽  
Mac Protocols ◽  
Underwater Sensor Networks ◽  
Medium Access ◽  
The Status ◽  
Efficient Data ◽  
Traffic Estimation ◽  
Delay Tolerant

In underwater sensor networks (UWSNs), the unique characteristics of acoustic channels have posed great challenges for the design of medium access control (MAC) protocols. The long propagation delay problem has been widely explored in recent literature. However, the long preamble problem with acoustic modems revealed in real experiments brings new challenges to underwater MAC design. The overhead of control messages in handshaking-based protocols becomes significant due to the long preamble in underwater acoustic modems. To address this problem, we advocate the receiver-initiated handshaking method with parallel reservation to improve the handshaking efficiency. Despite some existing works along this direction, the data polling problem is still an open issue. Without knowing the status of senders, the receiver faces two challenges for efficient data polling: when to poll data from the sender and how much data to request. In this paper, we propose a traffic estimation-based receiver-initiated MAC (TERI-MAC) to solve this problem with an adaptive approach. Data polling in TERI-MAC depends on an online approximation of traffic distribution. It estimates the energy efficiency and network latency and starts the data request only when the preferred performance can be achieved. TERI-MAC can achieve a stable energy efficiency with arbitrary network traffic patterns. For traffic estimation, we employ a resampling technique to keep a small computation and memory overhead. The performance of TERI-MAC in terms of energy efficiency, channel utilization, and communication latency is verified in simulations. Our results show that, compared with existing receiver-initiated-based underwater MAC protocols, TERI-MAC can achieve higher energy efficiency at the price of a delay penalty. This confirms the strength of TERI-MAC for delay-tolerant applications.

scholarly journals A MAC Protocol of Concurrent Scheduling Based on Spatial-Temporal Uncertainty for Underwater Sensor Networks

2021 ◽  
Vol 2021 ◽  
pp. 1-15
Author(s):  
Xin Liu ◽  
Xiujuan Du ◽  
Meiju Li ◽  
Lijuan Wang ◽  
Chong Li
Keyword(s):  
Energy Consumption ◽  
Sensor Networks ◽  
Mac Protocol ◽  
Average Energy ◽  
Propagation Delay ◽  
Spatial Uncertainty ◽  
Underwater Sensor Networks ◽  
Temporal Uncertainty ◽  
Concurrent Scheduling ◽  
Time Division

Underwater sensor networks (UWSNs) are characterized by large energy consumption, limited power supply, low bit rate, and long propagation delay, as well as spatial-temporal uncertainty, which present both challenges and opportunities for media access control (MAC) protocol design. The time-division transmissions can effectively avoid collisions since different nodes transmit packets at different period of time. Nevertheless, in UWSNs with long propagation delay, in order to avoid collisions, the period of time is subject to be long enough, which results in poor channel utilization and low throughput. In view of the long and different propagation delay between a receiving node and multiple sending nodes in UWSNs, as long as there is no collision at the receiving node, multiple sending nodes can transmit packets simultaneously. Therefore, in this paper, we propose a MAC protocol of concurrent scheduling based on spatial-temporal uncertainty called CSSTU-MAC (concurrent scheduling based on spatial-temporal uncertainty MAC) for UWSNs. The CSSTU-MAC protocol utilizes the characteristics of temporal-spatial uncertainty as well as long propagation delay in UWSNs to achieve concurrent transmission and collision avoidance. Simulation results show that the CSSTU-MAC protocol outperforms the existing MAC protocol with time-division transmissions in terms of average energy consumption and network throughput.

scholarly journals A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks

Sensors ◽  
2017 ◽  
Vol 17 (7) ◽  
pp. 1660 ◽  
Author(s):  
Zhigang Jin ◽  
Yingying Ma ◽  
Yishan Su ◽  
Shuo Li ◽  
Xiaomei Fu
Keyword(s):  
Sensor Networks ◽  
Routing Algorithm ◽  
Underwater Sensor Networks ◽  
Q Learning

scholarly journals Distributed Learning for Dynamic Channel Access in Underwater Sensor Networks

Entropy ◽  
2020 ◽  
Vol 22 (9) ◽  
pp. 992 ◽  
Author(s):  
Huicheol Shin ◽  
Yongjae Kim ◽  
Seungjae Baek ◽  
Yujae Song
Keyword(s):  
Sensor Networks ◽  
Network Throughput ◽  
Underwater Sensor Networks ◽  
Channel Access ◽  
Q Learning ◽  
Acoustic Sensor Networks ◽  
Access Problem ◽  
Markov Decision ◽  
Dynamic Channel ◽  
Physical Features

In this study, the problem of dynamic channel access in distributed underwater acoustic sensor networks (UASNs) is considered. First, we formulate the dynamic channel access problem in UASNs as a multi-agent Markov decision process, wherein each underwater sensor is considered an agent whose objective is to maximize the total network throughput without coordinating with or exchanging messages among different underwater sensors. We then propose a distributed deep Q-learning-based algorithm that enables each underwater sensor to learn not only the behaviors (i.e., actions) of other sensors, but also the physical features (e.g., channel error probability) of its available acoustic channels, in order to maximize the network throughput. We conduct extensive numerical evaluations and verify that the performance of the proposed algorithm is similar to or even better than the performance of baseline algorithms, even when implemented in a distributed manner.

Sign in / Sign up

Export Citation Format

Share Document