Markov Decision Models for Optimal Decision Making in Bottlenecks Identification and Mitigation

2015 ◽  
Author(s):  
Yang Li ◽  
Qing Chang ◽  
Xiaoning Jin ◽  
Jun Ni
Keyword(s):  
Optimal Policy ◽  
Data Driven ◽  
Optimal Decision ◽  
Stochastic Methods ◽  
Bottleneck Analysis ◽  
Markov Decision Models ◽  
Markov Decision ◽  
Optimal Decision Making

Existing methods for bottleneck detection can be categorized into two: methods based on stochastic analysis and methods based on data-driven analysis. The stochastic methods are accurate in estimating bottlenecks in long term, ignoring the current improvement opportunities, while the data-driven methods tend to do the opposite. In this paper, we develop an optimal policy to integrate the two methods based on Markov decision theory. The characterization of the optimal policy is provided. In addition, to implement the policy, the optimal frequency for carrying out bottleneck analysis is investigated. Numerical experiment is performed to validate the effectiveness of the optimal policy and compare it to the existing methods.

scholarly journals Memory inception and preservation in slime moulds: the quest for a common mechanism

2019 ◽  
Vol 374 (1774) ◽  
pp. 20180368 ◽  
Author(s):  
A. Boussard ◽  
J. Delescluse ◽  
A. Pérez-Escudero ◽  
A. Dussutour
Keyword(s):  
Optimal Decision ◽  
Common Mechanism ◽  
Cognitive Architectures ◽  
Dormant Stage ◽  
Dormancy Period ◽  
Slime Moulds ◽  
Short And Long Term ◽  
Optimal Decision Making ◽  
Past Experiences

Learning and memory are indisputably key features of animal success. Using information about past experiences is critical for optimal decision-making in a fluctuating environment. Those abilities are usually believed to be limited to organisms with a nervous system, precluding their existence in non-neural organisms. However, recent studies showed that the slime mouldPhysarum polycephalum, despite being unicellular, displays habituation, a simple form of learning. In this paper, we studied the possible substrate of both short- and long-term habituation in slime moulds. We habituated slime moulds to sodium, a known repellent, using a 6 day training and turned them into a dormant state named sclerotia. Those slime moulds were then revived and tested for habituation. We showed that information acquired during the training was preserved through the dormant stage as slime moulds still showed habituation after a one-month dormancy period. Chemical analyses indicated a continuous uptake of sodium during the process of habituation and showed that sodium was retained throughout the dormant stage. Lastly, we showed that memory inception via constrained absorption of sodium for 2 h elicited habituation. Our results suggest that slime moulds absorbed the repellent and used it as a ‘circulating memory’.This article is part of the theme issue ‘Liquid brains, solid brains: How distributed cognitive architectures process information’.

scholarly journals Optimizing Age Penalty in Time-Varying Networks with Markovian and Error-Prone Channel State

Entropy ◽  
2021 ◽  
Vol 23 (1) ◽  
pp. 91
Author(s):  
Yuchao Chen ◽  
Haoyue Tang ◽  
Jintao Wang ◽  
Jian Song
Keyword(s):  
Optimal Policy ◽  
Infinite Horizon ◽  
Base Station ◽  
Optimal Decision ◽  
Time Varying ◽  
Sensitive Data ◽  
Multiple Sensors ◽  
Data Freshness ◽  
Markov Decision ◽  
Time Varying Networks

In this paper, we consider a scenario where the base station (BS) collects time-sensitive data from multiple sensors through time-varying and error-prone channels. We characterize the data freshness at the terminal end through a class of monotone increasing functions related to Age of information (AoI). Our goal is to design an optimal policy to minimize the average age penalty of all sensors in infinite horizon under bandwidth and power constraint. By formulating the scheduling problem into a constrained Markov decision process (CMDP), we reveal the threshold structure for the optimal policy and approximate the optimal decision by solving a truncated linear programming (LP). Finally, a bandwidth-truncated policy is proposed to satisfy both power and bandwidth constraint. Through theoretical analysis and numerical simulations, we prove the proposed policy is asymptotic optimal in the large sensor regime.

scholarly journals Data-Driven Power System Optimal Decision Making Strategy under Wildfire Events

2022 ◽  
Author(s):  
Wanshi Hong ◽  
Bin Wang ◽  
Mengqi Yao ◽  
Duncan Callaway ◽  
Larry Dale ◽  
...  
Keyword(s):  
Decision Making ◽  
Power System ◽  
Data Driven ◽  
Optimal Decision ◽  
System Optimal ◽  
Optimal Decision Making

scholarly journals Uncertainty and Demand for Insurance: A Theoretical Model of How Self-Control Manages the Optimal Decision-Making

2021 ◽  
Vol 12 ◽  
Author(s):  
Mattia Adamo ◽  
Andrea P. Malizia
Keyword(s):  
Decision Making ◽  
Theoretical Model ◽  
Self Control ◽  
Optimal Decision ◽  
Insurance Sector ◽  
Demand For Insurance ◽  
Choice Making ◽  
Optimal Decision Making ◽  
The Individual

With the present work, we aim to mark a beginning line on the study of decision-making of potential consumers in the insurance sector, with the long-term purpose of defining the optimal cognitive processes to be undertaken when deciding whether to purchase insurance or not. Decision-making in conditions of uncertainty is influenced by the dual-self model doers/planner integrated with the hot–cold states and prospect utility function. Thus, we present a theoretical model of choice-making to evaluate the level of optimal self-control necessary to be exerted if the individual is either in the hot or in the cold state depending on the arousal. This theoretical choice-making model lays the ground for the decision journey by following the long-term utility and avoiding gross mistakes that could lead the consumer not to insure, when the odds suggest doing it, or vice versa, in situations when it would not be necessary.

scholarly journals Data-Driven Performance Evaluation Framework for Multi-Modal Public Transport Systems

Sensors ◽  
2021 ◽  
Vol 22 (1) ◽  
pp. 17
Author(s):  
Ana Belén Rodríguez Rodríguez González ◽  
Juan José Vinagre Vinagre Díaz ◽  
Mark R. Wilby ◽  
Rubén Fernández Fernández Pozo
Keyword(s):  
Performance Evaluation ◽  
Public Transport ◽  
Evaluation Framework ◽  
Data Driven ◽  
Transport Systems ◽  
Optimal Decision ◽  
Mobility Patterns ◽  
Public Transport System ◽  
Decision Making Processes ◽  
Optimal Decision Making

Transport agencies require accurate and updated information about public transport systems for the optimal decision-making processes regarding design and operation. In addition to assessing topology and service components, users’ behaviors must be considered. To this end, a data-driven performance evaluation based on passengers’ actual routes is key. Automatic fare collection platforms provide meaningful smart card data (SCD), but these are incomplete when gathered by entry-only systems. To obtain origin–destination (OD) matrices, we must manage complete journeys. In this paper, we use an adapted trip chaining method to reconstruct incomplete multi-modal journeys by finding spatial similarities between the outbound and inbound routes of the same user. From this dataset, we develop a performance evaluation framework that provides novel metrics and visualization utilities. First, we generate a space-time characterization of the overall operation of transport networks. Second, we supply enhanced OD matrices showing mobility patterns between zones and average traversed distances, travel times, and operation speeds, which model the real efficacy of the public transport system. We applied this framework to the Comunidad de Madrid (Spain), using 4 months’ worth of real SCD, showing its potential to generate meaningful information about the performance of multi-modal public transport systems.

Sign in / Sign up

Export Citation Format

Share Document