scholarly journals When algorithm selection meets Bi-linear Learning to Rank: accuracy and inference time trade off with candidates expansion

2020 ◽  
Author(s):  
Jing Yuan ◽  
Christian Geissler ◽  
Weijia Shao ◽  
Andreas Lommatzsch ◽  
Brijnesh Jain
Keyword(s):  
Optimal Algorithm ◽  
Learning To Rank ◽  
Mapping Function ◽  
Evaluation Process ◽  
Objective Evaluation ◽  
Problem Instance ◽  
Cost Calculation ◽  
Algorithm Selection ◽  
Trade Off ◽  
Problem Instances

Abstract Algorithm selection (AS) tasks are dedicated to find the optimal algorithm for an unseen problem instance. With the knowledge of problem instances’ meta-features and algorithms’ landmark performances, Machine Learning (ML) approaches are applied to solve AS problems. However, the standard training process of benchmark ML approaches in AS either needs to train the models specifically for every algorithm or relies on the sparse one-hot encoding as the algorithms’ representation. To escape these intermediate steps and form the mapping function directly, we borrow the learning to rank framework from Recommender System (RS) and embed the bi-linear factorization to model the algorithms’ performances in AS. This Bi-linear Learning to Rank (BLR) has proven to work with competence in some AS scenarios and thus is also proposed as a benchmark approach. Thinking from the evaluation perspective in the modern AS challenges, precisely predicting the performance is usually the measuring goal. Though approaches’ inference time also needs to be counted for the running time cost calculation, it’s always overlooked in the evaluation process. The multi-objective evaluation metric Adjusted Ratio of Root Ratios (A3R) is therefore advocated in this paper to balance the trade-off between the accuracy and inference time in AS. Concerning A3R, BLR outperforms other benchmarks when expanding the candidates range to TOP3. The better effect of this candidates expansion results from the cumulative optimum performance during the AS process. We take the further step in the experimentation to represent the advantage of such TOPK expansion, and illustrate that such expansion can be considered as the supplement for the convention of TOP1 selection during the evaluation process.

scholarly journals Neural Networks for Predicting Algorithm Runtime Distributions

2018 ◽  
Author(s):  
Katharina Eggensperger ◽  
Marius Lindauer ◽  
Frank Hutter
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Combinatorial Problems ◽  
Problem Instance ◽  
Ai Planning ◽  
Algorithm Selection ◽  
Problem Instances ◽  
Runtime Distributions ◽  
Better Than ◽  
Do So

Many state-of-the-art algorithms for solving hard combinatorial problems in artificial intelligence (AI) include elements of stochasticity that lead to high variations in runtime, even for a fixed problem instance. Knowledge about the resulting runtime distributions (RTDs) of algorithms on given problem instances can be exploited in various meta-algorithmic procedures, such as algorithm selection, portfolios, and randomized restarts. Previous work has shown that machine learning can be used to individually predict mean, median and variance of RTDs. To establish a new state-of-the-art in predicting RTDs, we demonstrate that the parameters of an RTD should be learned jointly and that neural networks can do this well by directly optimizing the likelihood of an RTD given runtime observations. In an empirical study involving five algorithms for SAT solving and AI planning, we show that neural networks predict the true RTDs of unseen instances better than previous methods, and can even do so when only few runtime observations are available per training instance.

scholarly journals MultiETSC: automated machine learning for early time series classification

2021 ◽  
Author(s):  
Gilles Ottervanger ◽  
Mitra Baratchi ◽  
Holger H. Hoos
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Optimal Algorithm ◽  
Empirical Evaluation ◽  
Early Time ◽  
Time Series Classification ◽  
Algorithm Selection ◽  
Trade Off ◽  
Conflicting Objectives ◽  
Automated Machine Learning

AbstractEarly time series classification (EarlyTSC) involves the prediction of a class label based on partial observation of a given time series. Most EarlyTSC algorithms consider the trade-off between accuracy and earliness as two competing objectives, using a single dedicated hyperparameter. To obtain insights into this trade-off requires finding a set of non-dominated (Pareto efficient) classifiers. So far, this has been approached through manual hyperparameter tuning. Since the trade-off hyperparameters only provide indirect control over the earliness-accuracy trade-off, manual tuning is tedious and tends to result in many sub-optimal hyperparameter settings. This complicates the search for optimal hyperparameter settings and forms a hurdle for the application of EarlyTSC to real-world problems. To address these issues, we propose an automated approach to hyperparameter tuning and algorithm selection for EarlyTSC, building on developments in the fast-moving research area known as automated machine learning (AutoML). To deal with the challenging task of optimising two conflicting objectives in early time series classification, we propose MultiETSC, a system for multi-objective algorithm selection and hyperparameter optimisation (MO-CASH) for EarlyTSC. MultiETSC can potentially leverage any existing or future EarlyTSC algorithm and produces a set of Pareto optimal algorithm configurations from which a user can choose a posteriori. As an additional benefit, our proposed framework can incorporate and leverage time-series classification algorithms not originally designed for EarlyTSC for improving performance on EarlyTSC; we demonstrate this property using a newly defined, “naïve” fixed-time algorithm. In an extensive empirical evaluation of our new approach on a benchmark of 115 data sets, we show that MultiETSC performs substantially better than baseline methods, ranking highest (avg. rank 1.98) compared to conceptually simpler single-algorithm (2.98) and single-objective alternatives (4.36).

scholarly journals Using Machine Learning for Quantum Annealing Accuracy Prediction

Algorithms ◽  
2021 ◽  
Vol 14 (6) ◽  
pp. 187
Author(s):  
Aaron Barbosa ◽  
Elijah Pelofske ◽  
Georg Hahn ◽  
Hristo N. Djidjev
Keyword(s):  
Machine Learning ◽  
Maximum Clique ◽  
Classification Model ◽  
Maximum Clique Problem ◽  
Problem Instance ◽  
Np Hard ◽  
Machine Learning Classification ◽  
Hard Problems ◽  
Problem Instances ◽  
D Wave

Quantum annealers, such as the device built by D-Wave Systems, Inc., offer a way to compute solutions of NP-hard problems that can be expressed in Ising or quadratic unconstrained binary optimization (QUBO) form. Although such solutions are typically of very high quality, problem instances are usually not solved to optimality due to imperfections of the current generations quantum annealers. In this contribution, we aim to understand some of the factors contributing to the hardness of a problem instance, and to use machine learning models to predict the accuracy of the D-Wave 2000Q annealer for solving specific problems. We focus on the maximum clique problem, a classic NP-hard problem with important applications in network analysis, bioinformatics, and computational chemistry. By training a machine learning classification model on basic problem characteristics such as the number of edges in the graph, or annealing parameters, such as the D-Wave’s chain strength, we are able to rank certain features in the order of their contribution to the solution hardness, and present a simple decision tree which allows to predict whether a problem will be solvable to optimality with the D-Wave 2000Q. We extend these results by training a machine learning regression model that predicts the clique size found by D-Wave.

scholarly journals National and Organizational Culture, Performance Evaluation and Trust: Evidence from Multinational Company Subsidiary in Indonesia

2017 ◽  
Vol 6 (2) ◽  
pp. 319-344
Author(s):  
Unggul Purwohedi
Keyword(s):  
Organizational Culture ◽  
Performance Evaluation ◽  
National Culture ◽  
Evaluation Process ◽  
Objective Evaluation ◽  
Trust Building ◽  
Local Staff ◽  
Mnc Subsidiary ◽  
The Impact ◽  
The Relationship

The aim of this study is to investigate the impact of national and organizational culture on the relationship between accounting and trust in a subsidiary of a Western Multi-National Company (MNC) in Indonesia. This study use a qualitative field study of one French MNC subsidiary and interview four expatriate directors, nine Indonesian managers and 10 Indonesian employees. Key themes were identified with the assistance of NVivo software. In this study, accounting, through formal performance evaluation, contributes to trust building between supervisors and their subordinates. Formal performance evaluation through transparent and objective evaluation increases trust in the supervisor. On the other hand, informal performance evaluation tends to decrease trustful behaviour due to secrecy in the evaluation process.  It appears that Indonesian national culture does influence organizational culture preference in the local staff. Individuals share national culture as a result of values developed from family, religion, education, and experience.DOI: 10.15408/sjie.v6i2.4733 

scholarly journals Evaluation of Temporomandibular Disorders Among Dental Students of Saudi Arabia Using Diagnostic Criteria/temporomandibular Disorders (DC/TMD) and Its Association With Biographic, Academic, and Psychosocial Parameters – A Cross-sectional Study

2021 ◽  
Author(s):  
KUMAR CHANDAN SRIVAST ◽  
DEEPTI SHRIVASTAVA ◽  
Zafar Ali Khan ◽  
Anil Kumar Nagarajappa ◽  
Mohammed Assayed Mousa ◽  
...  
Keyword(s):  
Saudi Arabia ◽  
Diagnostic Criteria ◽  
Temporomandibular Disorders ◽  
Functional Limitations ◽  
Evaluation Process ◽  
Objective Evaluation ◽  
Dental Students ◽  
Training Methods ◽  
Cross Sectional ◽  
Axis I

Abstract Background: Temporomandibular disorders (TMD) are a board category of conditions arising from the various components of the temporomandibular joint (TMJ) complex. Bio-psychosocial model is the most accepted theory describing the etiopathogenesis of TMD. Dental students are vulnerable to psychological disorders including anxiety, depression and stress. Hence, the aim of the current study was to evaluate the TMD among dental students of various academic levels and explore the association of TMD with biographic, academic, and psychosocial parameters. Methods: A total of 246 students of a dental school in Saudi Arabia were chosen for the study. After getting consent, all students were examined as per the diagnostic criteria/Temporomandibular disorders (DC/TMD) including components from axis-I and axis-II. Results: The overall cross-sectional prevalence of TMD was found to be 36.99%. Pain arising from the jaw, temple, and peri-auricular area was the most commonly reported symptom and elicited sign during examination. Among the pain-related TMD, myalgia was the commonest diagnosed condition, whereas disc displacement with reduction was found prevalent in intra-articular disorder category. Female (OR=1.94; P=0.004), married (OR=1.74; P=0.04), and students in clinical levels (OR=1.65; P=0.03) were shown to have significantly higher risk to develop TMD. Among the psychosocial parameters, parafunctional habits (OR=2.10; P<0.001) and anxiety (OR=1.55; P=0.04) are shown to increase risk of developing TMD. Students with any TMD reported to have significantly higher pain intensity (OR=1.68; P=0.01) and jaw functional limitations (OR=1.45; P=0.008). Conclusion: Dental students especially in the clinical levels were shown to poses higher risk of developing TMD, hence strategies such as academic counselling and objective evaluation via rubrics should be planned to modify the administration of the curriculum, training methods and evaluation process.

scholarly journals SATzilla: Portfolio-based Algorithm Selection for SAT

2008 ◽  
Vol 32 ◽  
pp. 565-606 ◽  
Author(s):  
L. Xu ◽  
F. Hutter ◽  
H. H. Hoos ◽  
K. Leyton-Brown
Keyword(s):  
Traditional Approach ◽  
Data Sets ◽  
Algorithm Selection ◽  
New Techniques ◽  
Algorithm Portfolios ◽  
Predicting Performance ◽  
Different Types ◽  
Sat Solver ◽  
Selection For ◽  
Problem Instances

It has been widely observed that there is no single "dominant" SAT solver; instead, different solvers perform best on different instances. Rather than following the traditional approach of choosing the best solver for a given class of instances, we advocate making this decision online on a per-instance basis. Building on previous work, we describe SATzilla, an automated approach for constructing per-instance algorithm portfolios for SAT that use so-called empirical hardness models to choose among their constituent solvers. This approach takes as input a distribution of problem instances and a set of component solvers, and constructs a portfolio optimizing a given objective function (such as mean runtime, percent of instances solved, or score in a competition). The excellent performance of SATzilla was independently verified in the 2007 SAT Competition, where our SATzilla07 solvers won three gold, one silver and one bronze medal. In this article, we go well beyond SATzilla07 by making the portfolio construction scalable and completely automated, and improving it by integrating local search solvers as candidate solvers, by predicting performance score instead of runtime, and by using hierarchical hardness models that take into account different types of SAT instances. We demonstrate the effectiveness of these new techniques in extensive experimental results on data sets including instances from the most recent SAT competition.

scholarly journals Simple Constructive, Insertion, and Improvement Heuristics Based on the Girding Polygon for the Euclidean Traveling Salesman Problem

Algorithms ◽  
2019 ◽  
Vol 13 (1) ◽  
pp. 5 ◽  
Author(s):  
Víctor Pacheco-Valencia ◽  
José Alberto Hernández ◽  
José María Sigarreta ◽  
Nodari Vakhania
Keyword(s):  
Traveling Salesman Problem ◽  
Real Life ◽  
Traveling Salesman ◽  
Computational Time ◽  
Problem Instance ◽  
Dimensional Euclidean Space ◽  
Benchmark Problem ◽  
Small Constant ◽  
Euclidean Traveling Salesman Problem ◽  
Problem Instances

The Traveling Salesman Problem (TSP) aims at finding the shortest trip for a salesman, who has to visit each of the locations from a given set exactly once, starting and ending at the same location. Here, we consider the Euclidean version of the problem, in which the locations are points in the two-dimensional Euclidean space and the distances are correspondingly Euclidean distances. We propose simple, fast, and easily implementable heuristics that work well, in practice, for large real-life problem instances. The algorithm works on three phases, the constructive, the insertion, and the improvement phases. The first two phases run in time O ( n 2 ) and the number of repetitions in the improvement phase, in practice, is bounded by a small constant. We have tested the practical behavior of our heuristics on the available benchmark problem instances. The approximation provided by our algorithm for the tested benchmark problem instances did not beat best known results. At the same time, comparing the CPU time used by our algorithm with that of the earlier known ones, in about 92% of the cases our algorithm has required less computational time. Our algorithm is also memory efficient: for the largest tested problem instance with 744,710 cities, it has used about 50 MiB, whereas the average memory usage for the remained 217 instances was 1.6 MiB.

scholarly journals An Instance Data Repository for the Round-robin Sports Timetabling Problem

2020 ◽  
Vol 45 (2) ◽  
pp. 184-200
Author(s):  
David Van Bulck ◽  
Dries Goossens ◽  
Jo¨rn Scho¨nberger ◽  
Mario Guajardo
Keyword(s):  
Optimization Problem ◽  
Real Life ◽  
Short Description ◽  
Problem Instance ◽  
Data Repository ◽  
Timetabling Problem ◽  
Classification Framework ◽  
Different Types ◽  
Problem Instances ◽  
Complex Matter

The sports timetabling problem is a combinatorial optimization problem that consists of creating a timetable that defines against whom, when and where teams play games. This is a complex matter, since real-life sports timetabling applications are typically highly constrained. The vast amount and variety of constraints and the lack of generally accepted benchmark problem instances make that timetable algorithms proposed in the literature are often tested on just one or two specific seasons of the competition under consideration. This is problematic since only a few algorithmic insights are gained. To mitigate this issue, this article provides a problem instance repository containing over 40 different types of instances covering artificial and real-life problem instances. The construction of such a repository is not trivial, since there are dozens of constraints that need to be expressed in a standardized format. For this, our repository relies on RobinX, an XML-supported classification framework. The resulting repository provides a (non-exhaustive) overview of most real-life sports timetabling applications published over the last five decades. For every problem, a short description highlights the most distinguishing characteristics of the problem. The repository is publicly available and will be continuously updated as new instances or better solutions become available.

scholarly journals Optimal Algorithm Selection in Multimodal Medical Image Registration

2021 ◽  
Vol 15 (4) ◽  
pp. 55
Author(s):  
Husein Elkeshreu ◽  
Otman Basir
Keyword(s):  
Image Registration ◽  
Medical Image ◽  
Optimal Algorithm ◽  
Operating Conditions ◽  
Selection Strategy ◽  
Medical Image Registration ◽  
Control Parameters ◽  
Multiple Sources ◽  
Algorithm Selection ◽  
Wide Range

Many medical applications benefit from the diversity inherent in imaging technologies to obtain more reliable diagnoses and assessments. Typically, the images obtained from multiple sources are acquired at distinct times and from different viewpoints, rendering a multitude of challenges for the registration process. Furthermore, different areas of the human body require disparate registration functional capabilities and degrees of accuracy. Thus, the benefit attained from the image multiplicity hinges heavily on the imaging modalities employed as well as the accuracy of the alignment process.  It is no surprise then that a wide range of registration techniques has emerged in the last two decades. Nevertheless, it is widely acknowledged that despite the many attempts, no registration technique has been able to deliver the required accuracy consistently under diverse operating conditions.  This paper introduces a novel method for achieving multimodal medical image registration based on exploiting the complementary and competitive nature of the algorithmic approaches behind a wide range of registration techniques. First, a thorough investigation of a wide range of registration algorithms is conducted for the purpose of understanding and quantifying their registration capabilities as well as the influence of their control parameters. Subsequently, a supervised randomized machine learning strategy is proposed for selecting the best registration algorithm for a given registration instance, and for determining the optimal control parameters for such algorithm. Several experiments have been conducted to verify the capabilities of the proposed selection strategy with respect to registration reliability, accuracy, and robustness.

scholarly journals Solving the Large-Scale TSP Problem in 1 h: Santa Claus Challenge 2020

2021 ◽  
Vol 8 ◽  
Author(s):  
Radu Mariescu-Istodor ◽  
Pasi Fränti
Keyword(s):  
Large Scale ◽  
Computing Time ◽  
Real Life ◽  
Problem Instance ◽  
Santa Claus ◽  
Design Choice ◽  
Long Time ◽  
Problem Instances ◽  
The Given ◽  
Important Design

The scalability of traveling salesperson problem (TSP) algorithms for handling large-scale problem instances has been an open problem for a long time. We arranged a so-called Santa Claus challenge and invited people to submit their algorithms to solve a TSP problem instance that is larger than 1 M nodes given only 1 h of computing time. In this article, we analyze the results and show which design choices are decisive in providing the best solution to the problem with the given constraints. There were three valid submissions, all based on local search, including k-opt up to k = 5. The most important design choice turned out to be the localization of the operator using a neighborhood graph. The divide-and-merge strategy suffers a 2% loss of quality. However, via parallelization, the result can be obtained within less than 2 min, which can make a key difference in real-life applications.

Sign in / Sign up

Export Citation Format

Share Document