A Rollout Control Algorithm for Discrete-Time Stochastic Systems

2010 ◽  
Author(s):  
Andreas A. Malikopoulos
Keyword(s):  
Decision Making ◽  
Markov Chain ◽  
Intelligent Systems ◽  
Control Algorithm ◽  
Stochastic Systems ◽  
Intelligent System ◽  
Control Policy ◽  
Suboptimal Control ◽  
Sequential Decision ◽  
Significant Research

The growing demand for making autonomous intelligent systems that can learn how to improve their performance while interacting with their environment has induced significant research on computational cognitive models. Computational intelligence, or rationality, can be achieved by modeling a system and the interaction with its environment through actions, perceptions, and associated costs. A widely adopted paradigm for modeling this interaction is the controlled Markov chain. In this context, the problem is formulated as a sequential decision-making process in which an intelligent system has to select those control actions in several time steps to achieve long-term goals. This paper presents a rollout control algorithm that aims to build an online decision-making mechanism for a controlled Markov chain. The algorithm yields a lookahead suboptimal control policy. Under certain conditions, a theoretical bound on its performance can be established.

Approximating Markov Chain Approach to Optimal Feedback Control of a Flexible Needle

2016 ◽  
Vol 138 (11) ◽  
Author(s):  
Javad Sovizi ◽  
Suren Kumar ◽  
Venkat Krovi
Keyword(s):  
Markov Chain ◽  
Feedback Control ◽  
Stochastic Systems ◽  
Control Policy ◽  
Nonlinear Problems ◽  
Optimal Feedback Control ◽  
Policy Space ◽  
Computationally Efficient ◽  
Insertion Point ◽  
Probability Of Success

Abstract We present a computationally efficient approach for the intra-operative update of the feedback control policy for the steerable needle in the presence of the motion uncertainty. The solution to dynamic programming (DP) equations, to obtain the optimal control policy, is difficult or intractable for nonlinear problems such as steering flexible needle in soft tissue. We use the method of approximating Markov chain to approximate the continuous (and controlled) process with its discrete and locally consistent counterpart. This provides the ground to examine the linear programming (LP) approach to solve the imposed DP problem that significantly reduces the computational demand. A concrete example of the two-dimensional (2D) needle steering is considered to investigate the effectiveness of the LP method for both deterministic and stochastic systems. We compare the performance of the LP-based policy with the results obtained through more computationally demanding algorithm, iterative policy space approximation. Finally, the reliability of the LP-based policy dealing with motion and parametric uncertainties as well as the effect of insertion point/angle on the probability of success is investigated.

A State-Space Representation Model and Learning Algorithm for Real-Time Decision-Making Under Uncertainty

2007 ◽  
Author(s):  
Andreas A. Malikopoulos ◽  
Panos Y. Papalambros ◽  
Dennis N. Assanis
Keyword(s):  
Decision Making ◽  
Optimal Control ◽  
Real Time ◽  
Learning Algorithm ◽  
Control Policy ◽  
Sequential Decision Making ◽  
Space Representation ◽  
Decision Making Under Uncertainty ◽  
Sequential Decision ◽  
State Space Representation

Modeling dynamic systems incurring stochastic disturbances for deriving a control policy is a ubiquitous task in engineering. However, in some instances obtaining a model of a system may be impractical or impossible. Alternative approaches have been developed using a simulation-based stochastic framework, in which the system interacts with its environment in real time and obtains information that can be processed to produce an optimal control policy. In this context, the problem of developing a policy for controlling the system’s behavior is formulated as a sequential decision-making problem under uncertainty. This paper considers real-time sequential decision-making under uncertainty modeled as a Markov Decision Process (MDP). A state-space representation model is constructed through a learning mechanism and is used to improve system performance over time. The model allows decision making based on gradually enhanced knowledge of system response as it transitions from one state to another, in conjunction with actions taken at each state. A learning algorithm is implemented realizing in real time the optimal control policy associated with the state transitions. The proposed method is demonstrated on the single cart-pole balancing problem and a vehicle cruise control problem.

scholarly journals Sistem Pendukung Keputusan Prediksi Kecepatan Studi Mahasiswa Menggunakan Metode ID3

2011 ◽  
Vol 2 (2) ◽  
Author(s):  
Ronny Ardi Giovani ◽  
Paulus Mudjihartono ◽  
Pranowo Pranowo
Keyword(s):  
Decision Making ◽  
Decision Support ◽  
Computer Science ◽  
Support System ◽  
Intelligent Systems ◽  
Intelligent System ◽  
Speed Prediction ◽  
Past Data ◽  
Tree Development

Abstract. Decision Support System of Students’ Study Speed Prediction Using ID3 Method. Speed can be a decisive period of study a student taking a degree in sajana. In this study would be built applications that serve to speed decision making predictions Students study Computer Science University of Atma Jaya Yogyakarta. Students will be expected sooner or later than the period of study by taking one course or thesis that will be undertaken after a certain semesters. There are many methods of classification of one method of ID3 (Induction Decision 3 'Tree'). Development system in this study made use of intelligent systems-based applications. The results achieved after the system is formed, among others, sophisticated and intelligent system capable of storing past data is used as a reference for decision making, where students with certain criteria can know the travel time of their studies, and can refer to the database so the system can be more detailed and rigorous in determining the choice. Keywords: study period speed, Decision Support Systems, ID3, Intelligent Systems Kecepatan masa studi dapat menjadi penentu seorang mahasiswa dalam menempuh gelar sajana. Dalam penelitian ini akan dibangun aplikasi yang berfungsi untuk pengambilan keputusan prediksi kecepatan studi Mahasiswa Teknik Informatika Universitas Atma Jaya Yogyakarta. Mahasiswa akan diprediksi  cepat atau lambatnya masa studi dalam menempuh mata kuliah maupun skripsi yang akan dijalani setelah semester tertentu. Ada banyak metode klasifikasi salah satunya metode ID3 (Induction Decision 3 ‘Tree’). Pembangunan sistem dalam penelitian ini dibuat menggunakan aplikasi berbasis sistem  cerdas. Hasil yang dicapai setelah sistem ini terbentuk antara lain sistem canggih dan cerdas yang mampu menyimpan data masa lalu yang digunakan sebagai acuan pengambilan keputusan, dimana mahasiswa dengan kriteria tertentu dapat diketahui masa tempuh studi mereka, serta dapat mengacu pada database sehingga sistem dapat lebih detail serta teliti dalam menentukan pilihan. Kata Kunci: kecepatan masa studi, Sistem Pendukung Keputusan, ID3, Sistem Cerdas

scholarly journals Research on the Framework of Intelligent Command Decision System for Flood Control

2019 ◽  
Vol 2 (1) ◽  
Author(s):  
Yao Zhang
Keyword(s):  
Decision Making ◽  
Intelligent Systems ◽  
Flood Control ◽  
Intelligent System ◽  
Decision Makers ◽  
Intelligent Decision Making ◽  
Related Information ◽  
Control Command ◽  
Decision Making System ◽  
Key Issues

Starting from the process of flood control and flood control decision-making, the shortcomings of the traditional flood control and flood control command and decision-making system are analyzed, and an intelligent decision-making system for flood control and flood control is proposed. The structure and functions of the system framework are elaborated in detail, and the key issues in the process of building intelligent systems are pointed out. Based on the real-time information monitoring system, this intelligent system can predict the next phase of hydrometeorology, flood and other related information, help decision makers to identify risks, and optimize the best flood control dispatching plan for decision makers to choose.

scholarly journals An Intelligent Decision-Making Support System for the Detection and Staging of Prostate Cancer in Developing Countries

2020 ◽  
Vol 2020 ◽  
pp. 1-18
Author(s):  
Jun Zhang ◽  
Zhigang Chen ◽  
Jia Wu ◽  
Kanghuai Liu
Keyword(s):  
Prostate Cancer ◽  
Decision Making ◽  
Developing Countries ◽  
Intelligent Systems ◽  
Intelligent System ◽  
Diagnostic System ◽  
Medical Personnel ◽  
Medical Systems ◽  
Intelligent Decision Making ◽  
Diagnosis Rate

Most developing countries face huge challenges in the medical field; scarce medical resources and inadequate medical personnel will affect the development and stability of the society. Therefore, for most developing countries, the development of intelligent medical systems can greatly alleviate the social contradictions arising from this problem. In this study, a new data decision-making intelligent system for prostate cancer based on perceptron neural network is proposed, which mainly makes decisions by associating some relevant disease indicators and combining them with medical images. Through data collection, analysis and integration of medical data, as well as the disease detection and decision-making process, patients are given an auxiliary diagnosis and treatment, so as to solve the problems and social contradictions faced by most developing countries. Through the study of hospitalization information of more than 8,000 prostate patients in three hospitals, about 2,156,528 data items were collected and compiled for experiment purposes. Experimental data shows that when the patient base increases from 200 to 8,000, the accuracy of the machine-assisted diagnostic system will increase from 61% to 87%, and the doctor’s diagnosis rate will be reduced to 81%. From the study, it is concluded that when the patient base reaches a certain number, the diagnostic accuracy of the machine-assisted diagnosis system will exceed the doctor’s expertise. Therefore, intelligent systems can help doctors and medical experts treat patients more effectively.

Convergence Properties of a Computational Learning Model for Unknown Markov Chains

2008 ◽  
Author(s):  
Andreas A. Malikopoulos
Keyword(s):  
Decision Making ◽  
Markov Chain ◽  
Intelligent Systems ◽  
Transition Probabilities ◽  
Optimal Decision ◽  
Underlying Structure ◽  
Computational Learning ◽  
Robust Model ◽  
Finite Time Horizon ◽  
Optimal Decision Making

The increasing complexity of engineering systems has motivated continuing research on computational learning methods towards making autonomous intelligent systems that can learn how to improve their performance over time while interacting with their environment. These systems need not only to be able to sense their environment, but should also integrate information from the environment into all decision making. The evolution of such systems is modeled as an unknown controlled Markov chain. In previous research, the predictive optimal decision-making (POD) model was developed that aims to learn in real time the unknown transition probabilities and associated costs over a varying finite time horizon. In this paper, the convergence of POD to the stationary distribution of a Markov chain is proven, thus establishing POD as a robust model for making autonomous intelligent systems. The paper provides the conditions that POD can be valid, and an interpretation of its underlying structure.

Mission Performance Aids

2015 ◽  
pp. 190-218
Keyword(s):  
Risk Factors ◽  
Decision Making ◽  
Intelligent Systems ◽  
Intelligent System ◽  
Cumulative Effect ◽  
Critical Events ◽  
Operational Conditions ◽  
Operational Decision Making ◽  
Mission Critical ◽  
Flight Deck

Flight deck displays that automatically adapt themselves to changing operational conditions are referred to as mission adaptive displays, or smart cockpits. A smart cockpit is an intelligent system possessing advanced reasoning capabilities. Mission Performance Aids are a particular manifestation of Mission Adaptive Displays. Mission Performance Aids fall into three categories: Precision Maneuver Guidance (PMG), Mission Performance Evaluator (MPE), and Operational Decision Making (ODM). The MPE alerts the crew to parameter exceedance. The ODM can calculate a cumulative effect with respect to two or more risk factors being encountered simultaneously. They discern all mission critical events, including escape maneuvers. Currently, this type of performance aid is not available. So this section should prove especially useful for designers of advanced intelligent systems.

A Real-Time Computational Learning Model for Sequential Decision-Making Problems Under Uncertainty

2009 ◽  
Vol 131 (4) ◽  
Author(s):  
Andreas A. Malikopoulos ◽  
Panos Y. Papalambros ◽  
Dennis N. Assanis
Keyword(s):  
Decision Making ◽  
Real Time ◽  
Control Policy ◽  
Learning Model ◽  
Sequential Decision Making ◽  
State Transitions ◽  
Space Representation ◽  
System Response ◽  
Sequential Decision ◽  
Cruise Control

Modeling dynamic systems incurring stochastic disturbances for deriving a control policy is a ubiquitous task in engineering. However, in some instances obtaining a model of a system may be impractical or impossible. Alternative approaches have been developed using a simulation-based stochastic framework, in which the system interacts with its environment in real time and obtains information that can be processed to produce an optimal control policy. In this context, the problem of developing a policy for controlling the system’s behavior is formulated as a sequential decision-making problem under uncertainty. This paper considers the problem of deriving a control policy for a dynamic system with unknown dynamics in real time, formulated as a sequential decision-making under uncertainty. The evolution of the system is modeled as a controlled Markov chain. A new state-space representation model and a learning mechanism are proposed that can be used to improve system performance over time. The major difference between the existing methods and the proposed learning model is that the latter utilizes an evaluation function, which considers the expected cost that can be achieved by state transitions forward in time. The model allows decision-making based on gradually enhanced knowledge of system response as it transitions from one state to another, in conjunction with actions taken at each state. The proposed model is demonstrated on the single cart-pole balancing problem and a vehicle cruise-control problem.

Intelligent evaluation of suppliers using extent fuzzy TOPSIS method

2018 ◽  
Vol 25 (1) ◽  
pp. 259-279 ◽  
Author(s):  
Vinod Yadav ◽  
Milind Kumar Sharma ◽  
Shailender Singh
Keyword(s):  
Decision Making ◽  
Intelligent Systems ◽  
Supplier Selection ◽  
Intelligent System ◽  
Fuzzy Topsis ◽  
Vital Role ◽  
Small Scale ◽  
Quality Cost ◽  
Content Type ◽  
Selection Decision

Purpose In a developing economy like India, the contribution of small- and medium-sized enterprises (SMEs) to the national gross domestic product is significant. This sector creates immense employment opportunities and produces economic products and services. To survive in the globalized marked condition, it is essential for SMEs to be competitive on several fronts such as quality, cost, delivery, lead time, flexibility, etc. Hence, it is imperative for them to have a sound supplier base. Therefore, supplier selection problem (SSP) has a vital role to play in supply chain management of SMEs. The paper aims to discuss these issues. Design/methodology/approach However, SSP has now become a significant challenge to address due to the complexity, vagueness and various criteria involved in it. Recently, fuzzy Technique for Order Performance by Similarity to Ideal Solution method has been widely used to tackle such problems. Findings The present paper aims at developing an intelligent system for SSP, which can consider the multiple criteria and the uncertainty aspects in the decision process. A case study of a small-scale manufacturing company has been presented. Practical implications This study provides a guideline for SME sector to implement intelligent systems for supplier selection decision-making problems. Case application concludes that this model improves firm’s decision making and suppliers’ performance. Originality/value The proposed intelligent model can provide the guidelines and directions for the decision makers to effectively choose suppliers in the current competitive environment. And it also provides an opportunity for supplier improvement.

Development of Cross-Platform Cognitive Tools Invariant to Problem Areas and their Integration into Intelligent Systems

2016 ◽  
Vol 683 ◽  
pp. 609-616 ◽  
Author(s):  
Anna Yankovskaya ◽  
Artyom Yamshanov
Keyword(s):  
Decision Making ◽  
Intelligent Systems ◽  
Intelligent System ◽  
Material Research ◽  
Cognitive Tools ◽  
New Approach ◽  
Knowledge Analysis ◽  
Cross Platform ◽  
Cognitive Graphics ◽  
Simplex Family

Nowadays application and development of cognitive graphic tools for the usage in intelligent system of data and knowledge analysis, decision-making and its justification for different problem areas including material research are urgency. Most significantly developed cognitive graphics tools based on n-simplex which are invariant to problem areas are presented. Specificity of program realization of cognitive graphics tools which is invariant to problem areas is described. Most significant results are given and discussed. Future investigations are connected with the usage of new approach to rendering, cross-platform realization, improving cognitive features and expanding n-simplex family

Sign in / Sign up

Export Citation Format

Share Document