Assessing Algorithmic Performance by Frontier Analysis

2018 ◽  
Vol 9 (1) ◽  
pp. 78-94
Author(s):  
Jose Humberto Ablanedo-Rosas ◽  
Cesar Rego
Keyword(s):  
Performance Metrics ◽  
Heuristic Algorithms ◽  
Analysis Of Algorithms ◽  
Statistical Significance ◽  
Individual Performance ◽  
Set Covering ◽  
Covering Problem ◽  
Frontier Analysis ◽  
Surrogate Constraint ◽  
Significance Levels

In Combinatorial Optimization the evaluation of heuristic algorithms often requires the consideration of multiple performance metrics that are relevant for the application of interest. Traditional empirical analysis of algorithms relies on evaluating individual performance metrics where the overall assessment is conducted by subjective judgment without the support of rigorous scientific methods. The authors propose an analytical approach based on data envelopment analysis (DEA) to rank algorithms by their relative efficiency scores that result from combining multiple performance metrics. To evaluate their approach, they perform a pilot study examining the relative performance of ten surrogate constraint algorithms for different classes of the set covering problem. The analysis shows their DEA-based approach is highly effective, establishing a clear difference between the algorithms' performances at appropriate statistical significance levels, and in consequence providing useful insights into the selection of algorithms to address each class of instances. Their approach is general and can be used with all types of performance metrics and algorithms.

scholarly journals Modeling Optimal Location Distribution for Deployment of Flying Base Stations as On-Demand Connectivity Enablers in Real-World Scenarios

Sensors ◽  
2021 ◽  
Vol 21 (16) ◽  
pp. 5580
Author(s):  
Jiri Pokorny ◽  
Pavel Seda ◽  
Milos Seda ◽  
Jiri Hosek
Keyword(s):  
Differential Evolution ◽  
Large Scale ◽  
Heuristic Algorithms ◽  
High Capacity ◽  
Network Capacity ◽  
Base Stations ◽  
Set Covering ◽  
Covering Problem ◽  
Network Service ◽  
Aerial Vehicles

The amount of internet traffic generated during mass public events is significantly growing in a way that requires methods to increase the overall performance of the wireless network service. Recently, legacy methods in form of mobile cell sites, frequently called cells on wheels, were used. However, modern technologies are allowing the use of unmanned aerial vehicles (UAV) as a platform for network service extension instead of ground-based techniques. This results in the development of flying base stations (FBS) where the number of deployed FBSs depends on the demanded network capacity and specific user requirements. Large-scale events, such as outdoor music festivals or sporting competitions, requiring deployment of more than one FBS need a method to optimally distribute these aerial vehicles to achieve high capacity and minimize the cost. In this paper, we present a mathematical model for FBS deployment in large-scale scenarios. The model is based on a location set covering problem and the goal is to minimize the number of FBSs by finding their optimal locations. It is restricted by users’ throughput requirements and FBSs’ available throughput, also, all users that require connectivity must be served. Two meta-heuristic algorithms (cuckoo search and differential evolution) were implemented and verified on a real example of a music festival scenario. The results show that both algorithms are capable of finding a solution. The major difference is in the performance where differential evolution solves the problem six to eight times faster, thus it is more suitable for repetitive calculation. The obtained results can be used in commercial scenarios similar to the one used in this paper where providing sufficient connectivity is crucial for good user experience. The designed algorithms will serve for the network infrastructure design and for assessing the costs and feasibility of the use-case.

scholarly journals Q-Learnheuristics: Towards Data-Driven Balanced Metaheuristics

Mathematics ◽  
2021 ◽  
Vol 9 (16) ◽  
pp. 1839
Author(s):  
Broderick Crawford ◽  
Ricardo Soto ◽  
José Lemus-Romani ◽  
Marcelo Becerra-Rozas ◽  
José M. Lanza-Gutiérrez ◽  
...  
Keyword(s):  
Combinatorial Problems ◽  
Set Covering ◽  
Covering Problem ◽  
Integration Framework ◽  
Q Learning ◽  
Whale Optimization ◽  
Sine Cosine Algorithm ◽  
Exploration Exploitation ◽  
Selection Of

One of the central issues that must be resolved for a metaheuristic optimization process to work well is the dilemma of the balance between exploration and exploitation. The metaheuristics (MH) that achieved this balance can be called balanced MH, where a Q-Learning (QL) integration framework was proposed for the selection of metaheuristic operators conducive to this balance, particularly the selection of binarization schemes when a continuous metaheuristic solves binary combinatorial problems. In this work the use of this framework is extended to other recent metaheuristics, demonstrating that the integration of QL in the selection of operators improves the exploration-exploitation balance. Specifically, the Whale Optimization Algorithm and the Sine-Cosine Algorithm are tested by solving the Set Covering Problem, showing statistical improvements in this balance and in the quality of the solutions.

scholarly journals An Analysis of a KNN Perturbation Operator: An Application to the Binarization of Continuous Metaheuristics

Mathematics ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. 225
Author(s):  
José García ◽  
Gino Astorga ◽  
Víctor Yepes
Keyword(s):  
Transfer Functions ◽  
Optimization Problems ◽  
Optimization Methods ◽  
Set Covering ◽  
Covering Problem ◽  
Perturbation Operator ◽  
K Nearest Neighbors ◽  
Combinatorial Optimization Problems ◽  
Box Plots ◽  
Metaheuristic Techniques

The optimization methods and, in particular, metaheuristics must be constantly improved to reduce execution times, improve the results, and thus be able to address broader instances. In particular, addressing combinatorial optimization problems is critical in the areas of operational research and engineering. In this work, a perturbation operator is proposed which uses the k-nearest neighbors technique, and this is studied with the aim of improving the diversification and intensification properties of metaheuristic algorithms in their binary version. Random operators are designed to study the contribution of the perturbation operator. To verify the proposal, large instances of the well-known set covering problem are studied. Box plots, convergence charts, and the Wilcoxon statistical test are used to determine the operator contribution. Furthermore, a comparison is made using metaheuristic techniques that use general binarization mechanisms such as transfer functions or db-scan as binarization methods. The results obtained indicate that the KNN perturbation operator improves significantly the results.

scholarly journals A Self-Adaptive Cuckoo Search Algorithm Using a Machine Learning Technique

Mathematics ◽  
2021 ◽  
Vol 9 (16) ◽  
pp. 1840
Author(s):  
Nicolás Caselli ◽  
Ricardo Soto ◽  
Broderick Crawford ◽  
Sergio Valdivia ◽  
Rodrigo Olivares
Keyword(s):  
Machine Learning ◽  
Optimization Problems ◽  
Search Algorithm ◽  
Cuckoo Search ◽  
Cuckoo Search Algorithm ◽  
Set Covering ◽  
Covering Problem ◽  
Parameter Configuration ◽  
Self Tuning ◽  
Problem Solvers

Metaheuristics are intelligent problem-solvers that have been very efficient in solving huge optimization problems for more than two decades. However, the main drawback of these solvers is the need for problem-dependent and complex parameter setting in order to reach good results. This paper presents a new cuckoo search algorithm able to self-adapt its configuration, particularly its population and the abandon probability. The self-tuning process is governed by using machine learning, where cluster analysis is employed to autonomously and properly compute the number of agents needed at each step of the solving process. The goal is to efficiently explore the space of possible solutions while alleviating human effort in parameter configuration. We illustrate interesting experimental results on the well-known set covering problem, where the proposed approach is able to compete against various state-of-the-art algorithms, achieving better results in one single run versus 20 different configurations. In addition, the result obtained is compared with similar hybrid bio-inspired algorithms illustrating interesting results for this proposal.

scholarly journals Multi-objective database queries in combined knapsack and set covering problem domains

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Sean A. Mochocki ◽  
Gary B. Lamont ◽  
Robert C. Leishman ◽  
Kyle J. Kauffman
Keyword(s):  
Relational Database ◽  
Set Covering ◽  
Covering Problem ◽  
Set Covering Problem ◽  
Database Queries ◽  
Multi Objective ◽  
Data Representations ◽  
Np Hard Problem ◽  
Institute Of Technology ◽  
The One

AbstractDatabase queries are one of the most important functions of a relational database. Users are interested in viewing a variety of data representations, and this may vary based on database purpose and the nature of the stored data. The Air Force Institute of Technology has approximately 100 data logs which will be converted to the standardized Scorpion Data Model format. A relational database is designed to house this data and its associated sensor and non-sensor metadata. Deterministic polynomial-time queries were used to test the performance of this schema against two other schemas, with databases of 100 and 1000 logs of repeated data and randomized metadata. Of these approaches, the one that had the best performance was chosen as AFIT’s database solution, and now more complex and useful queries need to be developed to enable filter research. To this end, consider the combined Multi-Objective Knapsack/Set Covering Database Query. Algorithms which address The Set Covering Problem or Knapsack Problem could be used individually to achieve useful results, but together they could offer additional power to a potential user. This paper explores the NP-Hard problem domain of the Multi-Objective KP/SCP, proposes Genetic and Hill Climber algorithms, implements these algorithms using Java, populates their data structures using SQL queries from two test databases, and finally compares how these algorithms perform.

scholarly journals Cavalier Use of Inferential Statistics Is a Major Source of False and Irreproducible Scientific Findings

Mathematics ◽  
2021 ◽  
Vol 9 (6) ◽  
pp. 603
Author(s):  
Leonid Hanin
Keyword(s):  
Sample Size ◽  
Gaussian Approximation ◽  
Statistical Significance ◽  
Statistical Analyses ◽  
Random Sample Size ◽  
P Values ◽  
The Central Limit Theorem ◽  
Fixed Sample ◽  
Large Numbers ◽  
Significance Levels

I uncover previously underappreciated systematic sources of false and irreproducible results in natural, biomedical and social sciences that are rooted in statistical methodology. They include the inevitably occurring deviations from basic assumptions behind statistical analyses and the use of various approximations. I show through a number of examples that (a) arbitrarily small deviations from distributional homogeneity can lead to arbitrarily large deviations in the outcomes of statistical analyses; (b) samples of random size may violate the Law of Large Numbers and thus are generally unsuitable for conventional statistical inference; (c) the same is true, in particular, when random sample size and observations are stochastically dependent; and (d) the use of the Gaussian approximation based on the Central Limit Theorem has dramatic implications for p-values and statistical significance essentially making pursuit of small significance levels and p-values for a fixed sample size meaningless. The latter is proven rigorously in the case of one-sided Z test. This article could serve as a cautionary guidance to scientists and practitioners employing statistical methods in their work.

Sign in / Sign up

Export Citation Format

Share Document