An Improved Algorithm of Spectrum Access in OSA

2012 ◽  
Vol 236-237 ◽  
pp. 1133-1138
Author(s):  
Sheng Liang Li ◽  
Xiao Wang ◽  
Shi Zeng Guo
Keyword(s):  
Spectrum Sensing ◽  
Transition Probability ◽  
Random Access ◽  
Spectrum Access ◽  
Distributed Spectrum Sensing ◽  
Markov Decision ◽  
The Difference ◽  
Partially Observable ◽  
Channel Transition ◽  
Random Access Algorithm

We present a distributed spectrum sensing and access algorithm applied in opportunistic spectrum access (OSA) networks. It is proposed in the condition of cognitive radio (CR) users’ correct spectrum sensing. In each slot, a CR user will decide whether to sense, which channel to sense, and whether to access. However, due to the hardware and energy constraints, the CR users may not be able to sense or monitor all the channels, so we design this access algorithm derived from the theory of Partially Observable Markov Decision Process (POMDP). To simplify the analysis, we assume that the CR users will sense and access only one channel. Simulation results show that the proposed access algorithm can greatly improve the throughput of CR users than the random access algorithm. It has good access performance, and can quickly achieve stable access throughput as well. It is shown that with greater channel transition probability difference the CR users will obtain higher access throughput, and when the difference equals to zero this access algorithm becomes random access algorithm.

“Threshold-Deciding” Policy in Distributed Multiuser Opportunistic Spectrum Access

2014 ◽  
Vol 926-930 ◽  
pp. 2867-2870
Author(s):  
Yu Meng Wang ◽  
Liang Shen ◽  
Xiang Gao ◽  
Cheng Long Xu ◽  
Xiao Ya Li ◽  
...  
Keyword(s):  
Opportunistic Spectrum Access ◽  
Spectrum Access ◽  
Secondary Users ◽  
Adaptive Policy ◽  
Markov Decision ◽  
Single User ◽  
Simulation Results ◽  
Partially Observable ◽  
Time And Energy ◽  
The Given

This paper studies the problem of distributed multiuser Opportunistic Spectrum Access based on Partially Observable Markov Decision Process (POMDP). Due to the similarity of spectrum environment, secondary users may choose the same channel adopting their own single user approach, which leads to collision. Referring to the previous works, we propose a more flexible and adaptive policy named “threshold-deciding”. Firstly, the SU gets a channel by adopting the random policy. Secondly, the SU decides whether to sense the channel by comparing the available probability with the given threshold. The policy not only decreases the collisions among SUs but also reduces the consumption of time and energy. The simulation results shows that the upgrade of performance is up to 100% compared with the existing random policy, which demonstrate the advantage of the proposed policy.

Dynamic Spectrum Access Considering Channel Fading in Cognitive Networks

2014 ◽  
Vol 543-547 ◽  
pp. 2013-2016
Author(s):  
Ye Bin Tao ◽  
Shi Ding Zhu
Keyword(s):  
Dynamic Spectrum Access ◽  
Cognitive Networks ◽  
Dynamic Spectrum ◽  
Time Varying ◽  
Channel Fading ◽  
Spectrum Access ◽  
Throughput Performance ◽  
Markov Decision ◽  
Simulation Results ◽  
Partially Observable

This paper investigated the method of Dynamic Spectrum Access (DSA) in cognitive networks, considering the PU channels both time-varying and fading. We used the Partially Observable Markov Decision Process (POMDP) framework to model this problem and designed a greedy strategy. The simulation results shows that the proposed strategy obtained better throughput performance than the existing works.

scholarly journals Dynamic spectrum access with deep Q-learning in densely occupied and partially observable environments

2021 ◽  
Vol 13 (1) ◽  
pp. 1-6
Author(s):  
Slavica Tomović ◽  
Igor Radusinović
Keyword(s):  
Dynamic Spectrum Access ◽  
Simulation Analysis ◽  
Dynamic Spectrum ◽  
Spectrum Access ◽  
Channel State ◽  
Q Learning ◽  
Communication Environments ◽  
Markov Decision ◽  
History Of ◽  
Partially Observable

In this paper, we propose a new Dynamic Spectrum Access (DSA) method for multi-channel wireless networks. We assume that DSA nodes do not have prior knowledge of the system dynamics, and have only partial observability of the channels. Thus, the problem is formulated as a Partially Observable Markov Decision Process (POMDP) with exponential time complexity. We have developed a novel Deep Reinforcement Learning (DRL) based DSA method which combines a double deep Q-learning architecture with a recurrent neural network and takes advantage of a prioritized experience buffer. The simulation analysis shows that the proposed method accurately predicts a channel state based on the fixed-length history of partial observations. Compared with other DRL methods for DSA, the proposed solution can find a near-optimal policy in a smaller number of iterations and suits a wider range of communication environments, including dynamic ones, where channel occupancy pattern changes over time. The performance improvement increases with the number of channels and with a channel state transition uncertainty. To boost the performance of the algorithm in densely occupied environments, multiple DRL exploration strategies are examined and evaluation results are presented in the paper.

scholarly journals Optimality of Multichannel Myopic Sensing in the Presence of Sensing Error for Opportunistic Spectrum Access

2013 ◽  
Vol 2013 ◽  
pp. 1-12 ◽  
Author(s):  
Xiaofeng Jiang ◽  
Hongsheng Xi
Keyword(s):  
Optimization Problem ◽  
Infinite Horizon ◽  
Practical Interest ◽  
Opportunistic Spectrum Access ◽  
Spectrum Access ◽  
Opportunistic Access ◽  
Markov Decision ◽  
Myopic Policy ◽  
Limited Sensing ◽  
Partially Observable

The optimization problem for the performance of opportunistic spectrum access is considered in this study. A user, with the limited sensing capacity, has opportunistic access to a communication system with multiple channels. The user can only choose several channels to sense and decides whether to access these channels based on the sensing information in each time slot. Meanwhile, the presence of sensing error is considered. A reward is obtained when the user accesses a channel. The objective is to maximize the expected (discounted or average) reward accrued over an infinite horizon. This problem can be formulated as a partially observable Markov decision process. This study shows the optimality of the simple and robust myopic policy which focuses on maximizing the immediate reward. The results show that the myopic policy is optimal in the case of practical interest.

scholarly journals Robust Policy Synthesis for Uncertain POMDPs via Convex Optimization

2020 ◽  
Author(s):  
Marnix Suilen ◽  
Nils Jansen ◽  
Murat Cubuktepe ◽  
Ufuk Topcu
Keyword(s):  
Optimization Problem ◽  
Probability Distributions ◽  
Transition Probability ◽  
Convex Polytopes ◽  
Quadratic Program ◽  
Markov Decision ◽  
Uncertainty Set ◽  
Imperfect Knowledge ◽  
Partially Observable ◽  
Quadratically Constrained

We study the problem of policy synthesis for uncertain partially observable Markov decision processes (uPOMDPs). The transition probability function of uPOMDPs is only known to belong to a so-called uncertainty set, for instance in the form of probability intervals. Such a model arises when, for example, an agent operates under information limitation due to imperfect knowledge about the accuracy of its sensors. The goal is to compute a policy for the agent that is robust against all possible probability distributions within the uncertainty set. In particular, we are interested in a policy that robustly ensures the satisfaction of temporal logic and expected reward specifications. We state the underlying optimization problem as a semi-infinite quadratically-constrained quadratic program (QCQP), which has finitely many variables and infinitely many constraints. Since QCQPs are non-convex in general and practically infeasible to solve, we resort to the so-called convex-concave procedure to convexify the QCQP. Even though convex, the resulting optimization problem still has infinitely many constraints and is NP-hard. For uncertainty sets that form convex polytopes, we provide a transformation of the problem to a convex QCQP with finitely many constraints. We demonstrate the feasibility of our approach by means of several case studies that highlight typical bottlenecks for our problem. In particular, we show that we are able to solve benchmarks with hundreds of thousands of states, hundreds of different observations, and we investigate the effect of different levels of uncertainty in the models.

Optimal adaptive inspection and maintenance for redundant systems

2021 ◽  
pp. 1748006X2110201
Author(s):  
Chaochao Lin ◽  
Matteo Pozzi
Keyword(s):  
Engineering Systems ◽  
Discounted Cost ◽  
Markov Decision ◽  
Inspection And Maintenance ◽  
And Performance ◽  
Partially Observable ◽  
Series Systems ◽  
Selection Of ◽  
Redundant Systems

Optimal exploration of engineering systems can be guided by the principle of Value of Information (VoI), which accounts for the topological important of components, their reliability and the management costs. For series systems, in most cases higher inspection priority should be given to unreliable components. For redundant systems such as parallel systems, analysis of one-shot decision problems shows that higher inspection priority should be given to more reliable components. This paper investigates the optimal exploration of redundant systems in long-term decision making with sequential inspection and repairing. When the expected, cumulated, discounted cost is considered, it may become more efficient to give higher inspection priority to less reliable components, in order to preserve system redundancy. To investigate this problem, we develop a Partially Observable Markov Decision Process (POMDP) framework for sequential inspection and maintenance of redundant systems, where the VoI analysis is embedded in the optimal selection of exploratory actions. We investigate the use of alternative approximate POMDP solvers for parallel and more general systems, compare their computation complexities and performance, and show how the inspection priorities depend on the economic discount factor, the degradation rate, the inspection precision, and the repair cost.

Sign in / Sign up

Export Citation Format

Share Document