scholarly journals Fast Two-Stage Computation of an Index Policy for Multi-Armed Bandits with Setup Delays

Mathematics ◽  
2020 ◽  
Vol 9 (1) ◽  
pp. 52
Author(s):  
José Niño-Mora
Keyword(s):  
Numerical Study ◽  
Arithmetic Operation ◽  
Bandit Problem ◽  
Index Policy ◽  
Two Stage ◽  
Second Stage ◽  
Whittle Index ◽  
Set Up ◽  
Computing Method ◽  
Special Case

We consider the multi-armed bandit problem with penalties for switching that include setup delays and costs, extending the former results of the author for the special case with no switching delays. A priority index for projects with setup delays that characterizes, in part, optimal policies was introduced by Asawa and Teneketzis in 1996, yet without giving a means of computing it. We present a fast two-stage index computing method, which computes the continuation index (which applies when the project has been set up) in a first stage and certain extra quantities with cubic (arithmetic-operation) complexity in the number of project states and then computes the switching index (which applies when the project is not set up), in a second stage, with quadratic complexity. The approach is based on new methodological advances on restless bandit indexation, which are introduced and deployed herein, being motivated by the limitations of previous results, exploiting the fact that the aforementioned index is the Whittle index of the project in its restless reformulation. A numerical study demonstrates substantial runtime speed-ups of the new two-stage index algorithm versus a general one-stage Whittle index algorithm. The study further gives evidence that, in a multi-project setting, the index policy is consistently nearly optimal.

scholarly journals Index policies for discounted bandit problems with availability constraints

2008 ◽  
Vol 40 (02) ◽  
pp. 377-400 ◽  
Author(s):  
Savas Dayanik ◽  
Warren Powell ◽  
Kazutoshi Yamazaki
Keyword(s):  
Bandit Problem ◽  
Bandit Problems ◽  
Index Policy ◽  
State Action ◽  
Index Policies ◽  
Availability Constraints ◽  
Whittle Index ◽  
Multiarmed Bandit

A multiarmed bandit problem is studied when the arms are not always available. The arms are first assumed to be intermittently available with some state/action-dependent probabilities. It is proven that no index policy can attain the maximum expected total discounted reward in every instance of that problem. The Whittle index policy is derived, and its properties are studied. Then it is assumed that the arms may break down, but repair is an option at some cost, and the new Whittle index policy is derived. Both problems are indexable. The proposed index policies cannot be dominated by any other index policy over all multiarmed bandit problems considered here. Whittle indices are evaluated for Bernoulli arms with unknown success probabilities.

scholarly journals Numerical Study of the Axial Gap and Hot Streak Effects on Thermal and Flow Characteristics in Two-Stage High Pressure Gas Turbine

Energies ◽  
2018 ◽  
Vol 11 (10) ◽  
pp. 2654 ◽  
Author(s):  
Myung Gon Choi ◽  
Jaiyoung Ryu
Keyword(s):  
Gas Turbine ◽  
Gas Turbines ◽  
Power Plants ◽  
Numerical Study ◽  
Flow Characteristics ◽  
Suction Side ◽  
Combined Cycle ◽  
Maximum Temperature ◽  
Two Stage ◽  
Second Stage

Combined cycle power plants (CCPPs) are becoming more important as the global demand for electrical power increases. The power and efficiency of CCPPs are directly affected by the performance and thermal efficiency of the gas turbines. This study is the first unsteady numerical study that comprehensively considers axial gap (AG) in the first-stage stator and first-stage rotor (R1) and hot streaks in the combustor outlet throughout an entire two-stage turbine, as these factors affect the aerodynamic performance of the turbine. To resolve the three-dimensional unsteady-state compressible flow, an unsteady Reynolds-averaged Navier–Stokes (RANS) equation was used to calculate a k-ω SST γ turbulence model. The AG distance d was set as 80% (case 1) and 120% (case 3) for the design value case 2 (13 mm or d/Cs1 = 0.307) in a GE-E3 gas turbine model. Changes in the AG affect the overall flow field characteristics and efficiency. If AG decreases, the time-averaged maximum temperature and pressure of R1 exhibit differences of approximately 3 K and 400 Pa, respectively. In addition, the low-temperature zone around the hub and tip regions of R1 and second-stage rotor (R2) on the suction side becomes smaller owing to a secondary flow and the area-averaged surface temperature increases. The area-averaged heat flux of the blade surface increases by a maximum of 10.6% at the second-stage stator and 2.8% at R2 as the AG decreases. The total-to-total efficiencies of the overall turbine increase by 0.306% and 0.295% when the AG decreases.

scholarly journals Index policies for discounted bandit problems with availability constraints

2008 ◽  
Vol 40 (2) ◽  
pp. 377-400 ◽  
Author(s):  
Savas Dayanik ◽  
Warren Powell ◽  
Kazutoshi Yamazaki
Keyword(s):  
Bandit Problem ◽  
Bandit Problems ◽  
Index Policy ◽  
State Action ◽  
Index Policies ◽  
Availability Constraints ◽  
Whittle Index ◽  
Multiarmed Bandit

A multiarmed bandit problem is studied when the arms are not always available. The arms are first assumed to be intermittently available with some state/action-dependent probabilities. It is proven that no index policy can attain the maximum expected total discounted reward in every instance of that problem. The Whittle index policy is derived, and its properties are studied. Then it is assumed that the arms may break down, but repair is an option at some cost, and the new Whittle index policy is derived. Both problems are indexable. The proposed index policies cannot be dominated by any other index policy over all multiarmed bandit problems considered here. Whittle indices are evaluated for Bernoulli arms with unknown success probabilities.

scholarly journals A Fast-Pivoting Algorithm for Whittle’s Restless Bandit Index

Mathematics ◽  
2020 ◽  
Vol 8 (12) ◽  
pp. 2226 ◽  
Author(s):  
José Niño-Mora
Keyword(s):  
Numerical Study ◽  
Index Policy ◽  
State Spaces ◽  
Restless Bandit ◽  
Restless Bandits ◽  
Pivoting Algorithm ◽  
Markov Decision ◽  
Whittle Index ◽  
Decision Epoch ◽  
Change State

The Whittle index for restless bandits (two-action semi-Markov decision processes) provides an intuitively appealing optimal policy for controlling a single generic project that can be active (engaged) or passive (rested) at each decision epoch, and which can change state while passive. It further provides a practical heuristic priority-index policy for the computationally intractable multi-armed restless bandit problem, which has been widely applied over the last three decades in multifarious settings, yet mostly restricted to project models with a one-dimensional state. This is due in part to the difficulty of establishing indexability (existence of the index) and of computing the index for projects with large state spaces. This paper draws on the author’s prior results on sufficient indexability conditions and an adaptive-greedy algorithmic scheme for restless bandits to obtain a new fast-pivoting algorithm that computes the n Whittle index values of an n-state restless bandit by performing, after an initialization stage, n steps that entail (2/3)n3+O(n2) arithmetic operations. This algorithm also draws on the parametric simplex method, and is based on elucidating the pattern of parametric simplex tableaux, which allows to exploit special structure to substantially simplify and reduce the complexity of simplex pivoting steps. A numerical study demonstrates substantial runtime speed-ups versus alternative algorithms.

An Enhanced Two Stage MMSE Equalizer for Coded FBMC/OQAM Systems

2019 ◽  
Vol 10 (1) ◽  
pp. 69-82 ◽  
Author(s):  
Mohammad Rizk Assaf ◽  
Abdel-Nasser Assimi
Keyword(s):  
Error Propagation ◽  
Channel Model ◽  
Error Correcting Codes ◽  
Two Stage ◽  
Probability Of Error ◽  
Second Stage ◽  
Different Types ◽  
Mmse Equalizer ◽  
And Performance ◽  
Ici Cancellation

In this article, the authors investigate the enhanced two stage MMSE (TS-MMSE) equalizer in bit-interleaved coded FBMC/OQAM system which gives a tradeoff between complexity and performance, since error correcting codes limits error propagation, so this allows the equalizer to remove not only ICI but also ISI in the second stage. The proposed equalizer has shown less design complexity compared to the other MMSE equalizers. The obtained results show that the probability of error is improved where SNR gain reaches 2 dB measured at BER compared with ICI cancellation for different types of modulation schemes and ITU Vehicular B channel model. Some simulation results are provided to illustrate the effectiveness of the proposed equalizer.

Personalised attraction recommendation for enhancing topic diversity and accuracy

2021 ◽  
pp. 016555152199980
Author(s):  
Yuanyuan Lin ◽  
Chao Huang ◽  
Wei Yao ◽  
Yifei Shao
Keyword(s):  
Real World ◽  
Information Overload ◽  
Two Stage ◽  
Misclassification Cost ◽  
Second Stage ◽  
Low Visibility ◽  
Recommendation Diversity ◽  
Optimisation Model ◽  
Definition Of ◽  
Improved Methods

Attraction recommendation plays an important role in tourism, such as solving information overload problems and recommending proper attractions to users. Currently, most recommendation methods are dedicated to improving the accuracy of recommendations. However, recommendation methods only focusing on accuracy tend to recommend popular items that are often purchased by users, which results in a lack of diversity and low visibility of non-popular items. Hence, many studies have suggested the importance of recommendation diversity and proposed improved methods, but there is room for improvement. First, the definition of diversity for different items requires consideration for domain characteristics. Second, the existing algorithms for improving diversity sacrifice the accuracy of recommendations. Therefore, the article utilises the topic ‘features of attractions’ to define the calculation method of recommendation diversity. We developed a two-stage optimisation model to enhance recommendation diversity while maintaining the accuracy of recommendations. In the first stage, an optimisation model considering topic diversity is proposed to increase recommendation diversity and generate candidate attractions. In the second stage, we propose a minimisation misclassification cost optimisation model to balance recommendation diversity and accuracy. To assess the performance of the proposed method, experiments are conducted with real-world travel data. The results indicate that the proposed two-stage optimisation model can significantly improve the diversity and accuracy of recommendations.

scholarly journals Two-stage multi-tasking transform framework for large-scale many-objective optimization problems

2021 ◽  
Author(s):  
Lu Chen ◽  
Handing Wang ◽  
Wenping Ma
Keyword(s):  
Large Scale ◽  
Original Problem ◽  
Optimization Problems ◽  
Reference Points ◽  
Optimization Strategy ◽  
Two Stage ◽  
Decision Space ◽  
Second Stage ◽  
Decision Variables ◽  
Objective Space

AbstractReal-world optimization applications in complex systems always contain multiple factors to be optimized, which can be formulated as multi-objective optimization problems. These problems have been solved by many evolutionary algorithms like MOEA/D, NSGA-III, and KnEA. However, when the numbers of decision variables and objectives increase, the computation costs of those mentioned algorithms will be unaffordable. To reduce such high computation cost on large-scale many-objective optimization problems, we proposed a two-stage framework. The first stage of the proposed algorithm combines with a multi-tasking optimization strategy and a bi-directional search strategy, where the original problem is reformulated as a multi-tasking optimization problem in the decision space to enhance the convergence. To improve the diversity, in the second stage, the proposed algorithm applies multi-tasking optimization to a number of sub-problems based on reference points in the objective space. In this paper, to show the effectiveness of the proposed algorithm, we test the algorithm on the DTLZ and LSMOP problems and compare it with existing algorithms, and it outperforms other compared algorithms in most cases and shows disadvantage on both convergence and diversity.

scholarly journals A Two-Stage Mono- and Multi-Objective Method for the Optimization of General UPS Parallel Manipulators

Mathematics ◽  
2021 ◽  
Vol 9 (5) ◽  
pp. 543
Author(s):  
Alejandra Ríos ◽  
Eusebio E. Hernández ◽  
S. Ivvan Valdez
Keyword(s):  
Genetic Algorithm ◽  
Performance Metrics ◽  
Tracking Error ◽  
Parallel Manipulators ◽  
Optimization Method ◽  
Statistical Hypothesis ◽  
Two Stage ◽  
Multi Objective ◽  
Second Stage ◽  
Conflicting Objectives

This paper introduces a two-stage method based on bio-inspired algorithms for the design optimization of a class of general Stewart platforms. The first stage performs a mono-objective optimization in order to reach, with sufficient dexterity, a regular target workspace while minimizing the elements’ lengths. For this optimization problem, we compare three bio-inspired algorithms: the Genetic Algorithm (GA), the Particle Swarm Optimization (PSO), and the Boltzman Univariate Marginal Distribution Algorithm (BUMDA). The second stage looks for the most suitable gains of a Proportional Integral Derivative (PID) control via the minimization of two conflicting objectives: one based on energy consumption and the tracking error of a target trajectory. To this effect, we compare two multi-objective algorithms: the Multiobjective Evolutionary Algorithm based on Decomposition (MOEA/D) and Non-dominated Sorting Genetic Algorithm-III (NSGA-III). The main contributions lie in the optimization model, the proposal of a two-stage optimization method, and the findings of the performance of different bio-inspired algorithms for each stage. Furthermore, we show optimized designs delivered by the proposed method and provide directions for the best-performing algorithms through performance metrics and statistical hypothesis tests.

scholarly journals Multiple Sclerosis: Assessment of Disease Progression and Effects of Treatment

1987 ◽  
Vol 14 (S3) ◽  
pp. 518-520 ◽  
Author(s):  
D.W. Paty
Keyword(s):  
Multiple Sclerosis ◽  
Disease Progression ◽  
Treatment Effects ◽  
Two Stage ◽  
Second Stage ◽  
Sequential Development ◽  
Stage Disease

ABSTRACT:MS could well be a two stage disease. The first stage involves the sequential development of multiple small lesions, mostly inflammatory, that accumulate at a given rate. The second stage could be that of consolidation and confluence of lesions that involves not only demyelination but gliosis. MRI now gives us an opportunity to watch the evolution of these processes and also to monitor treatment effects. It is only after the evolution of this process is understood that we can design rational therapies directed toward the prevention of spasticity in MS.

Sign in / Sign up

Export Citation Format

Share Document