A Q-learning based energy threshold optimization algorithm in LAA networks

In order to improve an efficiency of energy detection for a spectrum sensing in cognitive radio (CR), this paper proposes a dynamic threshold optimization algorithm. The traditional energy detection algorithm uses a fixed threshold, and can't guarantee always the optimal sensing performance in any environment. The improvement for sensing performance need to minimize the undetected probability and the probability of false alarm, and it is dissimilar for different CR users to accept these two errors. We improve the traditional energy detection algorithm, and firstly introduce a preference factor to characterize CR users’ different requirements for these two errors, then, propose a dynamic threshold optimization algorithm by minimizing integrated detection error for different signal-to-noise ratio (SNR). The simulation results show that the proposed algorithm effectively reduces the integrated spectrum sensing error, and increases the probability of detection, especially in low SNR.

Download Full-text

A Q-learning-based swarm optimization algorithm for economic dispatch problem

Neural Computing and Applications ◽

10.1007/s00521-015-2070-1 ◽

2015 ◽

Vol 27 (8) ◽

pp. 2333-2350 ◽

Cited By ~ 10

Author(s):

Yi-Zeng Hsieh ◽

Mu-Chun Su

Keyword(s):

Optimization Algorithm ◽

Economic Dispatch ◽

Swarm Optimization ◽

Q Learning ◽

Economic Dispatch Problem

Download Full-text

Indoor Emergency Path Planning Based on the Q-Learning Optimization Algorithm

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi11010066 ◽

2022 ◽

Vol 11 (1) ◽

pp. 66

Author(s):

Shenghua Xu ◽

Yang Gu ◽

Xiaoyan Li ◽

Cai Chen ◽

Yingyi Hu ◽

...

Keyword(s):

Path Planning ◽

Shortest Path ◽

Optimization Algorithm ◽

Indoor Environment ◽

Large Scale ◽

Learning Algorithm ◽

Simulated Data ◽

Convergence Speed ◽

Grid Environment ◽

Q Learning

The internal structure of buildings is becoming increasingly complex. Providing a scientific and reasonable evacuation route for trapped persons in a complex indoor environment is important for reducing casualties and property losses. In emergency and disaster relief environments, indoor path planning has great uncertainty and higher safety requirements. Q-learning is a value-based reinforcement learning algorithm that can complete path planning tasks through autonomous learning without establishing mathematical models and environmental maps. Therefore, we propose an indoor emergency path planning method based on the Q-learning optimization algorithm. First, a grid environment model is established. The discount rate of the exploration factor is used to optimize the Q-learning algorithm, and the exploration factor in the ε-greedy strategy is dynamically adjusted before selecting random actions to accelerate the convergence of the Q-learning algorithm in a large-scale grid environment. An indoor emergency path planning experiment based on the Q-learning optimization algorithm was carried out using simulated data and real indoor environment data. The proposed Q-learning optimization algorithm basically converges after 500 iterative learning rounds, which is nearly 2000 rounds higher than the convergence rate of the Q-learning algorithm. The SASRA algorithm has no obvious convergence trend in 5000 iterations of learning. The results show that the proposed Q-learning optimization algorithm is superior to the SARSA algorithm and the classic Q-learning algorithm in terms of solving time and convergence speed when planning the shortest path in a grid environment. The convergence speed of the proposed Q- learning optimization algorithm is approximately five times faster than that of the classic Q- learning algorithm. The proposed Q-learning optimization algorithm in the grid environment can successfully plan the shortest path to avoid obstacle areas in a short time.

Download Full-text

Q-Learning and MADMM Optimization Algorithm Based Interference Aware Channel Assignment Strategy for Load Balancing in WMNS

International Journal of Intelligent Engineering and Systems ◽

10.22266/ijies2021.0228.04 ◽

2021 ◽

Vol 14 (1) ◽

pp. 32-41

Author(s):

Karunya Rathan ◽

◽

Susai Roslin ◽

Keyword(s):

Optimization Algorithm ◽

Channel Assignment ◽

Learning Automata ◽

Multicast Tree ◽

Delivery Ratio ◽

Q Learning ◽

Packet Delivery ◽

Wireless Mesh ◽

End To End Delay ◽

Assignment Strategy

Wireless Mesh Networks (WMNs) have been considered one of the main technologies for configuring wireless machines since they appeared. In a WMN, wireless routers provide multi-hop wireless connectivity between hosts on the network and allow access to the internet through the gateway routers. These wireless routers are normally equipped with the multiple radios in the wireless mesh network that operate on multiple channels with the multiple interference, which is caused to reduce the network performance and end-to-end delay. In this paper, we proposed an efficient optimization algorithm to solve the channel assignment problem which cause due to the multichannel multiradios in WMN’s. The main objective of our paper is to minimize the channel interference among networked devices. So, initially we construct a multicast tree with minimum interference by using Q-Learning algorithm, which is helps to minimize the end-to-end delay of packet delivery. From the constructed multicast tree, we intend to develop a channel assignment strategy with the minimum interference by using Modified version of Alternative Direction Method of Multipliers (MADMM) optimization algorithm, which is helps to increase the network throughput and packet delivery ratio. The proposed strategy was implemented by using NS-2 (Network Simulator-2) and the experimental result show that the performance of the proposed method is very high compared to the other method and the performance was calculated by using the feature metrics such as average throughput, packet delivery ratio, end-toend delay and total cost, which is compared with the other existing channel assignment strategies such as Learning Automata and Genetic Algorithm (LA-GA), GA-based approach, link-channel selection and rate-allocation (LCR) and learning automata based multicast routing (LAMR) channel assignment methods.

Download Full-text