scholarly journals A Study of Learning Search Approximation in Mixed Integer Branch and Bound: Node Selection in SCIP

AI ◽  
2021 ◽  
Vol 2 (2) ◽  
pp. 150-178
Author(s):  
Kaan Yilmaz ◽  
Neil Yorke-Smith
Keyword(s):  
Machine Learning ◽  
Branch And Bound ◽  
State Of The Art ◽  
Heuristic Method ◽  
Imitation Learning ◽  
Mixed Integer ◽  
Node Selection ◽  
Selection Policy ◽  
State Of Practice ◽  
Optimality Gap

In line with the growing trend of using machine learning to help solve combinatorial optimisation problems, one promising idea is to improve node selection within a mixed integer programming (MIP) branch-and-bound tree by using a learned policy. Previous work using imitation learning indicates the feasibility of acquiring a node selection policy, by learning an adaptive node searching order. In contrast, our imitation learning policy is focused solely on learning which of a node’s children to select. We present an offline method to learn such a policy in two settings: one that comprises a heuristic by committing to pruning of nodes; one that is exact and backtracks from a leaf to guarantee finding the optimal integer solution. The former setting corresponds to a child selector during plunging, while the latter is akin to a diving heuristic. We apply the policy within the popular open-source solver SCIP, in both heuristic and exact settings. Empirical results on five MIP datasets indicate that our node selection policy leads to solutions significantly more quickly than the state-of-the-art precedent in the literature. While we do not beat the highly-optimised SCIP state-of-practice baseline node selector in terms of solving time on exact solutions, our heuristic policies have a consistently better optimality gap than all baselines, if the accuracy of the predictive model is sufficient. Further, the results also indicate that, when a time limit is applied, our heuristic method finds better solutions than all baselines in the majority of problems tested. We explain the results by showing that the learned policies have imitated the SCIP baseline, but without the latter’s early plunge abort. Our recommendation is that, despite the clear improvements over the literature, this kind of MIP child selector is better seen in a broader approach to using learning in MIP branch-and-bound tree decisions.

Mixed-integer nonlinear optimization

Acta Numerica ◽  
2013 ◽  
Vol 22 ◽  
pp. 1-131 ◽  
Author(s):  
Pietro Belotti ◽  
Christian Kirches ◽  
Sven Leyffer ◽  
Jeff Linderoth ◽  
James Luedtke ◽  
...  
Keyword(s):  
Branch And Bound ◽  
Convex Functions ◽  
State Of The Art ◽  
Small Sample ◽  
The State ◽  
Decision Problems ◽  
Mixed Integer ◽  
Feasible Region ◽  
Convex Relaxations ◽  
Single Tree

Many optimal decision problems in scientific, engineering, and public sector applications involve both discrete decisions and nonlinear system dynamics that affect the quality of the final design or plan. These decision problems lead to mixed-integer nonlinear programming (MINLP) problems that combine the combinatorial difficulty of optimizing over discrete variable sets with the challenges of handling nonlinear functions. We review models and applications of MINLP, and survey the state of the art in methods for solving this challenging class of problems.Most solution methods for MINLP apply some form of tree search. We distinguish two broad classes of methods: single-tree and multitree methods. We discuss these two classes of methods first in the case where the underlying problem functions are convex. Classical single-tree methods include nonlinear branch-and-bound and branch-and-cut methods, while classical multitree methods include outer approximation and Benders decomposition. The most efficient class of methods for convex MINLP are hybrid methods that combine the strengths of both classes of classical techniques.Non-convex MINLPs pose additional challenges, because they contain non-convex functions in the objective function or the constraints; hence even when the integer variables are relaxed to be continuous, the feasible region is generally non-convex, resulting in many local minima. We discuss a range of approaches for tackling this challenging class of problems, including piecewise linear approximations, generic strategies for obtaining convex relaxations for non-convex functions, spatial branch-and-bound methods, and a small sample of techniques that exploit particular types of non-convex structures to obtain improved convex relaxations.We finish our survey with a brief discussion of three important aspects of MINLP. First, we review heuristic techniques that can obtain good feasible solution in situations where the search-tree has grown too large or we require real-time solutions. Second, we describe an emerging area of mixed-integer optimal control that adds systems of ordinary differential equations to MINLP. Third, we survey the state of the art in software for MINLP.

scholarly journals Adapted Branch-and-Bound Algorithm Using SVM With Model Selection

2019 ◽  
Vol 9 (4) ◽  
pp. 2481 ◽  
Author(s):  
Mohamed Mustapha Kabbaj ◽  
El Afia Abdellatif
Keyword(s):  
Model Selection ◽  
Branch And Bound ◽  
Mixed Integer Linear Programming ◽  
Cross Validation ◽  
Branch And Bound Algorithm ◽  
Mixed Integer ◽  
Selection Strategy ◽  
Support Vector ◽  
Information Channel ◽  
Node Selection

<span lang="EN-US">Branch-and-Bound algorithm is the basis for the majority of solving methods in mixed integer linear programming. It has been proving its efficiency in different fields. In fact, it creates little by little a tree of nodes by adopting two strategies. These strategies are variable selection strategy and node selection strategy. In our previous work, we experienced a methodology of learning branch-and-bound strategies </span><span lang="EN-US">using regression-based support vector machine twice. That methodology allowed firstly to exploit information from previous executions of Branch-and-Bound algorithm on other instances. Secondly, it created information channel between node selection strategy and variable branching strategy. And thirdly, it gave good results in term of running time comparing to standard Branch-and-Bound algorithm. In this work, we will focus on increasing SVM performance by using cross validation coupled with model selection. </span>

scholarly journals Optimal versus approximate channel selection methods for EEG decoding with application to topology-constrained neuro-sensor networks

2020 ◽  
Author(s):  
Abhijith Mundanad Narayanan ◽  
Panagiotis Patrinos ◽  
Alexander Bertrand
Keyword(s):  
Sensor Networks ◽  
Large Scale ◽  
State Of The Art ◽  
Channel Selection ◽  
Mixed Integer ◽  
Electrode Placement ◽  
Quadratic Program ◽  
Neural Decoding ◽  
Selection Methods ◽  
Optimality Gap

AbstractChannel selection or electrode placement for neural decoding is a commonly encountered problem in electroencephalography (EEG). Since evaluating all possible channel combinations is usually infeasible, one usually has to settle for heuristic methods or convex approximations without optimality guarantees. To date, it remains unclear how large the gap is between the selection made by these approximate methods and the truly optimal selection. The goal of this paper is to quantify this optimality gap for several state-of-the-art channel selection methods in the context of least-squares based neural decoding. To this end, we reformulate the channel selection problem as a mixed-integer quadratic program (MIQP), which allows the use of efficient MIQP solvers to find the optimal channel combination in a feasible computation time for up to 100 candidate channels. As this reveals the exact solution to the combinatorial problem, it allows to quantify the performance losses when using state-of-the-art sub-optimal (yet faster) channel selection methods. In a context of auditory attention decoding, we find that a greedy channel selection based on the utility metric does not show a significant optimality gap compared to optimal channel selection, whereas other state-of-the-art greedy or l1-norm penalized methods do show a significant loss in performance. Furthermore, we demonstrate that the MIQP formulation also provides a natural way to incorporate topology constraints in the selection, e.g., for electrode placement in neuro-sensor networks with galvanic separation constraints. Furthermore, a combination of this utility-based greedy selection with an MIQP solver allows to perform a topology constrained electrode placement, even in large scale problems with more than 100 candidate positions.

Data science in economics: comprehensive review of advanced machine learning and deep learning methods

2020 ◽  
Author(s):  
Saeed Nosratabadi ◽  
Amir Mosavi ◽  
Puhong Duan ◽  
Pedram Ghamisi ◽  
Ferdinand Filip ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Data Science ◽  
State Of The Art ◽  
Science Methods ◽  
Learning Models ◽  
Diverse Range ◽  
Hybrid Machine ◽  
Economics Research

This paper provides a state-of-the-art investigation of advances in data science in emerging economic applications. The analysis was performed on novel data science methods in four individual classes of deep learning models, hybrid deep learning models, hybrid machine learning, and ensemble models. Application domains include a wide and diverse range of economics research from the stock market, marketing, and e-commerce to corporate banking and cryptocurrency. Prisma method, a systematic literature review methodology, was used to ensure the quality of the survey. The findings reveal that the trends follow the advancement of hybrid models, which, based on the accuracy metric, outperform other learning algorithms. It is further expected that the trends will converge toward the advancements of sophisticated hybrid deep learning models.

Deep Learning for text in limted data settings

2020 ◽  
Author(s):  
Pathikkumar Patel ◽  
Bhargav Lad ◽  
Jinan Fiaidhi
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Transfer Learning ◽  
Text Classification ◽  
State Of The Art ◽  
Time Series Forecasting ◽  
Text Data ◽  
Performance Levels

During the last few years, RNN models have been extensively used and they have proven to be better for sequence and text data. RNNs have achieved state-of-the-art performance levels in several applications such as text classification, sequence to sequence modelling and time series forecasting. In this article we will review different Machine Learning and Deep Learning based approaches for text data and look at the results obtained from these methods. This work also explores the use of transfer learning in NLP and how it affects the performance of models on a specific application of sentiment analysis.

scholarly journals Multi-hop assortativities for network classification

2018 ◽  
Vol 7 (4) ◽  
pp. 603-622 ◽  
Author(s):  
Leonardo Gutiérrez-Gómez ◽  
Jean-Charles Delvenne
Keyword(s):  
Machine Learning ◽  
Scientific Collaboration ◽  
State Of The Art ◽  
Medical Engineering ◽  
Research Field ◽  
Classification Task ◽  
Collaboration Network ◽  
Structural Patterns ◽  
Art Methods

Abstract Several social, medical, engineering and biological challenges rely on discovering the functionality of networks from their structure and node metadata, when it is available. For example, in chemoinformatics one might want to detect whether a molecule is toxic based on structure and atomic types, or discover the research field of a scientific collaboration network. Existing techniques rely on counting or measuring structural patterns that are known to show large variations from network to network, such as the number of triangles, or the assortativity of node metadata. We introduce the concept of multi-hop assortativity, that captures the similarity of the nodes situated at the extremities of a randomly selected path of a given length. We show that multi-hop assortativity unifies various existing concepts and offers a versatile family of ‘fingerprints’ to characterize networks. These fingerprints allow in turn to recover the functionalities of a network, with the help of the machine learning toolbox. Our method is evaluated empirically on established social and chemoinformatic network benchmarks. Results reveal that our assortativity based features are competitive providing highly accurate results often outperforming state of the art methods for the network classification task.

scholarly journals Advances in the Application of Machine Learning Techniques for Power System Analytics: A Survey

Energies ◽  
2021 ◽  
Vol 14 (16) ◽  
pp. 4776
Author(s):  
Seyed Mahdi Miraftabzadeh ◽  
Michela Longo ◽  
Federica Foiadelli ◽  
Marco Pasetti ◽  
Raul Igual
Keyword(s):  
Machine Learning ◽  
Power Systems ◽  
Smart Grids ◽  
State Of The Art ◽  
Smart Cities ◽  
Power Grids ◽  
Machine Learning Techniques ◽  
Learning Techniques ◽  
New Research ◽  
Traditional Approaches

The recent advances in computing technologies and the increasing availability of large amounts of data in smart grids and smart cities are generating new research opportunities in the application of Machine Learning (ML) for improving the observability and efficiency of modern power grids. However, as the number and diversity of ML techniques increase, questions arise about their performance and applicability, and on the most suitable ML method depending on the specific application. Trying to answer these questions, this manuscript presents a systematic review of the state-of-the-art studies implementing ML techniques in the context of power systems, with a specific focus on the analysis of power flows, power quality, photovoltaic systems, intelligent transportation, and load forecasting. The survey investigates, for each of the selected topics, the most recent and promising ML techniques proposed by the literature, by highlighting their main characteristics and relevant results. The review revealed that, when compared to traditional approaches, ML algorithms can handle massive quantities of data with high dimensionality, by allowing the identification of hidden characteristics of (even) complex systems. In particular, even though very different techniques can be used for each application, hybrid models generally show better performances when compared to single ML-based models.

scholarly journals gbt-HIPS: Explaining the Classifications of Gradient Boosted Tree Ensembles

2021 ◽  
Vol 11 (6) ◽  
pp. 2511
Author(s):  
Julian Hatwell ◽  
Mohamed Medhat Gaber ◽  
R. Muhammad Atif Azad
Keyword(s):  
State Of The Art ◽  
Heuristic Method ◽  
Good Explanation ◽  
Classification Rule ◽  
Data Sets ◽  
Classification Models ◽  
Boundary Values ◽  
Class Label ◽  
Input Space ◽  
Boosted Tree

This research presents Gradient Boosted Tree High Importance Path Snippets (gbt-HIPS), a novel, heuristic method for explaining gradient boosted tree (GBT) classification models by extracting a single classification rule (CR) from the ensemble of decision trees that make up the GBT model. This CR contains the most statistically important boundary values of the input space as antecedent terms. The CR represents a hyper-rectangle of the input space inside which the GBT model is, very reliably, classifying all instances with the same class label as the explanandum instance. In a benchmark test using nine data sets and five competing state-of-the-art methods, gbt-HIPS offered the best trade-off between coverage (0.16–0.75) and precision (0.85–0.98). Unlike competing methods, gbt-HIPS is also demonstrably guarded against under- and over-fitting. A further distinguishing feature of our method is that, unlike much prior work, our explanations also provide counterfactual detail in accordance with widely accepted recommendations for what makes a good explanation.

scholarly journals Cable tree wiring - benchmarking solvers on a real-world scheduling problem with a variety of precedence constraints

Constraints ◽  
2021 ◽  
Author(s):  
Jana Koehler ◽  
Josef Bürgler ◽  
Urs Fontana ◽  
Etienne Fux ◽  
Florian Herzog ◽  
...  
Keyword(s):  
State Of The Art ◽  
Precedence Constraints ◽  
Traveling Salesman ◽  
Mixed Integer ◽  
Automated Manufacturing ◽  
Scheduling Problem ◽  
Tasks Scheduling ◽  
Optimization Modulo Theories ◽  
Coupled Tasks ◽  
Np Hardness

AbstractCable trees are used in industrial products to transmit energy and information between different product parts. To this date, they are mostly assembled by humans and only few automated manufacturing solutions exist using complex robotic machines. For these machines, the wiring plan has to be translated into a wiring sequence of cable plugging operations to be followed by the machine. In this paper, we study and formalize the problem of deriving the optimal wiring sequence for a given layout of a cable tree. We summarize our investigations to model this cable tree wiring problem (CTW). as a traveling salesman problem with atomic, soft atomic, and disjunctive precedence constraints as well as tour-dependent edge costs such that it can be solved by state-of-the-art constraint programming (CP), Optimization Modulo Theories (OMT), and mixed-integer programming (MIP). solvers. It is further shown, how the CTW problem can be viewed as a soft version of the coupled tasks scheduling problem. We discuss various modeling variants for the problem, prove its NP-hardness, and empirically compare CP, OMT, and MIP solvers on a benchmark set of 278 instances. The complete benchmark set with all models and instance data is available on github and was included in the MiniZinc challenge 2020.

Sign in / Sign up

Export Citation Format

Share Document