scholarly journals Sliding-Window Thompson Sampling for Non-Stationary Settings

2020 ◽  
Vol 68 ◽  
pp. 311-364
Author(s):  
Francesco Trovo ◽  
Stefano Paladino ◽  
Marcello Restelli ◽  
Nicola Gatti
Keyword(s):  
Real World ◽  
State Of The Art ◽  
Sliding Window ◽  
Upper Bounds ◽  
Decision Problems ◽  
Sequential Decision ◽  
Thompson Sampling ◽  
The Past ◽  
Real World Applications ◽  
Window Approach

Multi-Armed Bandit (MAB) techniques have been successfully applied to many classes of sequential decision problems in the past decades. However, non-stationary settings -- very common in real-world applications -- received little attention so far, and theoretical guarantees on the regret are known only for some frequentist algorithms. In this paper, we propose an algorithm, namely Sliding-Window Thompson Sampling (SW-TS), for nonstationary stochastic MAB settings. Our algorithm is based on Thompson Sampling and exploits a sliding-window approach to tackle, in a unified fashion, two different forms of non-stationarity studied separately so far: abruptly changing and smoothly changing. In the former, the reward distributions are constant during sequences of rounds, and their change may be arbitrary and happen at unknown rounds, while, in the latter, the reward distributions smoothly evolve over rounds according to unknown dynamics. Under mild assumptions, we provide regret upper bounds on the dynamic pseudo-regret of SW-TS for the abruptly changing environment, for the smoothly changing one, and for the setting in which both the non-stationarity forms are present. Furthermore, we empirically show that SW-TS dramatically outperforms state-of-the-art algorithms even when the forms of non-stationarity are taken separately, as previously studied in the literature.

scholarly journals Non Stationary Multi-Armed Bandit: Empirical Evaluation of a New Concept Drift-Aware Algorithm

Entropy ◽  
2021 ◽  
Vol 23 (3) ◽  
pp. 380
Author(s):  
Emanuele Cavenaghi ◽  
Gabriele Sottocornola ◽  
Fabio Stella ◽  
Markus Zanker
Keyword(s):  
Real World ◽  
Concept Drift ◽  
Empirical Evaluation ◽  
Sliding Window ◽  
Discount Factor ◽  
Data Streaming ◽  
Sources Of Information ◽  
Sequential Decision ◽  
Time Step ◽  
Thompson Sampling

The Multi-Armed Bandit (MAB) problem has been extensively studied in order to address real-world challenges related to sequential decision making. In this setting, an agent selects the best action to be performed at time-step t, based on the past rewards received by the environment. This formulation implicitly assumes that the expected payoff for each action is kept stationary by the environment through time. Nevertheless, in many real-world applications this assumption does not hold and the agent has to face a non-stationary environment, that is, with a changing reward distribution. Thus, we present a new MAB algorithm, named f-Discounted-Sliding-Window Thompson Sampling (f-dsw TS), for non-stationary environments, that is, when the data streaming is affected by concept drift. The f-dsw TS algorithm is based on Thompson Sampling (TS) and exploits a discount factor on the reward history and an arm-related sliding window to contrast concept drift in non-stationary environments. We investigate how to combine these two sources of information, namely the discount factor and the sliding window, by means of an aggregation function f(.). In particular, we proposed a pessimistic (f=min), an optimistic (f=max), as well as an averaged (f=mean) version of the f-dsw TS algorithm. A rich set of numerical experiments is performed to evaluate the f-dsw TS algorithm compared to both stationary and non-stationary state-of-the-art TS baselines. We exploited synthetic environments (both randomly-generated and controlled) to test the MAB algorithms under different types of drift, that is, sudden/abrupt, incremental, gradual and increasing/decreasing drift. Furthermore, we adapt four real-world active learning tasks to our framework—a prediction task on crimes in the city of Baltimore, a classification task on insects species, a recommendation task on local web-news, and a time-series analysis on microbial organisms in the tropical air ecosystem. The f-dsw TS approach emerges as the best performing MAB algorithm. At least one of the versions of f-dsw TS performs better than the baselines in synthetic environments, proving the robustness of f-dsw TS under different concept drift types. Moreover, the pessimistic version (f=min) results as the most effective in all real-world tasks.

Has Dynamic Programming Improved Decision Making?

2019 ◽  
Vol 11 (1) ◽  
pp. 833-858 ◽  
Author(s):  
John Rust
Keyword(s):  
Decision Making ◽  
Dynamic Programming ◽  
Real World ◽  
Operations Research ◽  
Decision Rules ◽  
Decision Problems ◽  
Optimal Decision ◽  
Sequential Decision ◽  
Success Stories ◽  
Real World Applications

Dynamic programming (DP) is a powerful tool for solving a wide class of sequential decision-making problems under uncertainty. In principle, it enables us to compute optimal decision rules that specify the best possible decision in any situation. This article reviews developments in DP and contrasts its revolutionary impact on economics, operations research, engineering, and artificial intelligence with the comparative paucity of its real-world applications to improve the decision making of individuals and firms. The fuzziness of many real-world decision problems and the difficulty in mathematically modeling them are key obstacles to a wider application of DP in real-world settings. Nevertheless, I discuss several success stories, and I conclude that DP offers substantial promise for improving decision making if we let go of the empirically untenable assumption of unbounded rationality and confront the challenging decision problems faced every day by individuals and firms.

scholarly journals Co-GCN for Multi-View Semi-Supervised Learning

2020 ◽  
Vol 34 (04) ◽  
pp. 4691-4698
Author(s):  
Shu Li ◽  
Wen-Tao Li ◽  
Wei Wang
Keyword(s):  
Supervised Learning ◽  
Real World ◽  
State Of The Art ◽  
Data Sets ◽  
Real World Data ◽  
Convolutional Network ◽  
The Past ◽  
Real World Applications ◽  
Supervised Methods ◽  
Disjoint Sets

In many real-world applications, the data have several disjoint sets of features and each set is called as a view. Researchers have developed many multi-view learning methods in the past decade. In this paper, we bring Graph Convolutional Network (GCN) into multi-view learning and propose a novel multi-view semi-supervised learning method Co-GCN by adaptively exploiting the graph information from the multiple views with combined Laplacians. Experimental results on real-world data sets verify that Co-GCN can achieve better performance compared with state-of-the-art multi-view semi-supervised methods.

scholarly journals A Survey on Bias and Fairness in Machine Learning

2021 ◽  
Vol 54 (6) ◽  
pp. 1-35
Author(s):  
Ninareh Mehrabi ◽  
Fred Morstatter ◽  
Nripsuta Saxena ◽  
Kristina Lerman ◽  
Aram Galstyan
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Deep Learning ◽  
Real World ◽  
State Of The Art ◽  
Future Directions ◽  
Discriminatory Behavior ◽  
Real World Applications ◽  
Near Future ◽  
Different Sources

With the widespread use of artificial intelligence (AI) systems and applications in our everyday lives, accounting for fairness has gained significant importance in designing and engineering of such systems. AI systems can be used in many sensitive environments to make important and life-changing decisions; thus, it is crucial to ensure that these decisions do not reflect discriminatory behavior toward certain groups or populations. More recently some work has been developed in traditional machine learning and deep learning that address such challenges in different subdomains. With the commercialization of these systems, researchers are becoming more aware of the biases that these applications can contain and are attempting to address them. In this survey, we investigated different real-world applications that have shown biases in various ways, and we listed different sources of biases that can affect AI applications. We then created a taxonomy for fairness definitions that machine learning researchers have defined to avoid the existing bias in AI systems. In addition to that, we examined different domains and subdomains in AI showing what researchers have observed with regard to unfair outcomes in the state-of-the-art methods and ways they have tried to address them. There are still many future directions and solutions that can be taken to mitigate the problem of bias in AI systems. We are hoping that this survey will motivate researchers to tackle these issues in the near future by observing existing work in their respective fields.

Against generalisation: Data-driven decisions need context to be human-compatible

2021 ◽  
pp. 026638212110619
Author(s):  
Sharon Richardson
Keyword(s):  
Artificial Intelligence ◽  
Real World ◽  
Data Science ◽  
Machine Learning Algorithms ◽  
Data Driven ◽  
Theoretical Frameworks ◽  
The Past ◽  
Real World Applications ◽  
Contextual Framework ◽  
Data Driven Decisions

During the past two decades, there have been a number of breakthroughs in the fields of data science and artificial intelligence, made possible by advanced machine learning algorithms trained through access to massive volumes of data. However, their adoption and use in real-world applications remains a challenge. This paper posits that a key limitation in making AI applicable has been a failure to modernise the theoretical frameworks needed to evaluate and adopt outcomes. Such a need was anticipated with the arrival of the digital computer in the 1950s but has remained unrealised. This paper reviews how the field of data science emerged and led to rapid breakthroughs in algorithms underpinning research into artificial intelligence. It then discusses the contextual framework now needed to advance the use of AI in real-world decisions that impact human lives and livelihoods.

scholarly journals INDUSTRIAL APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS

2014 ◽  
pp. 8-20
Author(s):  
Kurosh Madani
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Real World ◽  
Industrial Applications ◽  
Large Field ◽  
Nonlinear Functions ◽  
The Past ◽  
Real World Applications ◽  
Ann Models ◽  
Artificial Neural

In a large number of real world dilemmas and related applications the modeling of complex behavior is the central point. Over the past decades, new approaches based on Artificial Neural Networks (ANN) have been proposed to solve problems related to optimization, modeling, decision making, classification, data mining or nonlinear functions (behavior) approximation. Inspired from biological nervous systems and brain structure, Artificial Neural Networks could be seen as information processing systems, which allow elaboration of many original techniques covering a large field of applications. Among their most appealing properties, one can quote their learning and generalization capabilities. The main goal of this paper is to present, through some of main ANN models and based techniques, their real application capability in real world industrial dilemmas. Several examples through industrial and real world applications have been presented and discussed.

An Efficient Pruning and Filtering Strategy to Mine Partial Periodic Patterns from a Sequence of Event Sets

2014 ◽  
Vol 10 (2) ◽  
pp. 18-38 ◽  
Author(s):  
Kung-Jiuan Yang ◽  
Tzung-Pei Hong ◽  
Yuh-Min Chen ◽  
Guo-Cheng Lan
Keyword(s):  
Time Series ◽  
Real World ◽  
Oil Price ◽  
Time Stamp ◽  
Single Event ◽  
Periodic Patterns ◽  
Event Time ◽  
The Past ◽  
Real World Applications ◽  
Speed Up

Partial periodic patterns are commonly seen in real-world applications. The major problem of mining partial periodic patterns is the efficiency problem due to a huge set of partial periodic candidates. Although some efficient algorithms have been developed to tackle the problem, the performance of the algorithms significantly drops when the mining parameters are set low. In the past, the authors have adopted the projection-based approach to discover the partial periodic patterns from single-event time series. In this paper, the authors extend it to mine partial periodic patterns from a sequence of event sets which multiple events concurrently occur at the same time stamp. Besides, an efficient pruning and filtering strategy is also proposed to speed up the mining process. Finally, the experimental results on a synthetic dataset and real oil price dataset show the good performance of the proposed approach.

scholarly journals Particle Swarm Contour Search Algorithm

Entropy ◽  
2020 ◽  
Vol 22 (4) ◽  
pp. 407 ◽  
Author(s):  
Dominik Weikert ◽  
Sebastian Mai ◽  
Sanaz Mostaghim
Keyword(s):  
Image Processing ◽  
Real World ◽  
State Of The Art ◽  
Search Algorithm ◽  
Particle Swarm ◽  
Search Space ◽  
Local Information ◽  
The State ◽  
Complete Knowledge ◽  
Real World Applications

In this article, we present a new algorithm called Particle Swarm Contour Search (PSCS)—a Particle Swarm Optimisation inspired algorithm to find object contours in 2D environments. Currently, most contour-finding algorithms are based on image processing and require a complete overview of the search space in which the contour is to be found. However, for real-world applications this would require a complete knowledge about the search space, which may not be always feasible or possible. The proposed algorithm removes this requirement and is only based on the local information of the particles to accurately identify a contour. Particles search for the contour of an object and then traverse alongside using their known information about positions in- and out-side of the object. Our experiments show that the proposed PSCS algorithm can deliver comparable results as the state-of-the-art.

scholarly journals Lifelong Spectral Clustering

2020 ◽  
Vol 34 (04) ◽  
pp. 5867-5874
Author(s):  
Gan Sun ◽  
Yang Cong ◽  
Qianqian Wang ◽  
Jun Li ◽  
Yun Fu
Keyword(s):  
Machine Learning ◽  
Real World ◽  
Spectral Clustering ◽  
State Of The Art ◽  
Clustering Algorithms ◽  
Orthogonal Basis ◽  
Learning Framework ◽  
The Past ◽  
Benchmark Datasets ◽  
Over Time

In the past decades, spectral clustering (SC) has become one of the most effective clustering algorithms. However, most previous studies focus on spectral clustering tasks with a fixed task set, which cannot incorporate with a new spectral clustering task without accessing to previously learned tasks. In this paper, we aim to explore the problem of spectral clustering in a lifelong machine learning framework, i.e., Lifelong Spectral Clustering (L2SC). Its goal is to efficiently learn a model for a new spectral clustering task by selectively transferring previously accumulated experience from knowledge library. Specifically, the knowledge library of L2SC contains two components: 1) orthogonal basis library: capturing latent cluster centers among the clusters in each pair of tasks; 2) feature embedding library: embedding the feature manifold information shared among multiple related tasks. As a new spectral clustering task arrives, L2SC firstly transfers knowledge from both basis library and feature library to obtain encoding matrix, and further redefines the library base over time to maximize performance across all the clustering tasks. Meanwhile, a general online update formulation is derived to alternatively update the basis library and feature library. Finally, the empirical experiments on several real-world benchmark datasets demonstrate that our L2SC model can effectively improve the clustering performance when comparing with other state-of-the-art spectral clustering algorithms.

PROVIDING TIMELY UPDATED SEQUENTIAL PATTERNS IN DECISION MAKING

2010 ◽  
Vol 09 (06) ◽  
pp. 873-888 ◽  
Author(s):  
TZUNG-PEI HONG ◽  
CHING-YAO WANG ◽  
CHUN-WEI LIN
Keyword(s):  
Decision Making ◽  
Real Time ◽  
Real World ◽  
Experimental Results ◽  
Sequential Patterns ◽  
The Past ◽  
Large Databases ◽  
Real World Applications ◽  
Critical Task ◽  
Maintenance Process

Mining knowledge from large databases has become a critical task for organizations. Managers commonly use the obtained sequential patterns to make decisions. In the past, databases were usually assumed to be static. In real-world applications, however, transactions may be updated. In this paper, a maintenance algorithm for rapidly updating sequential patterns for real-time decision making is proposed. The proposed algorithm utilizes previously discovered large sequences in the maintenance process, thus greatly reducing the number of database rescans and improving performance. Experimental results verify the performance of the proposed approach. The proposed algorithm provides real-time knowledge that can be used for decision making.

Sign in / Sign up

Export Citation Format

Share Document