scholarly journals Applications of MaxSAT in Data Analysis

10.29007/3qkh ◽  
2019 ◽  
Author(s):  
Jeremias Berg ◽  
Antti Hyttinen ◽  
Matti Järvisalo
Keyword(s):  
Machine Learning ◽  
Data Analysis ◽  
Real World ◽  
Optimization Problems ◽  
Learning Problems ◽  
Integral Role ◽  
Benchmark Sets

We highlight important real-world optimization problems arising from data analysis and machine learning, representing somewhat atypical applications of SAT-based solver technology, to which the SAT community could focus more attention on. To address the problem of current lack of heterogeneity in benchmark sets available for evaluating MaxSAT solvers, we provide a benchmark library of MaxSAT instances encoding different data analysis and machine learning problems. By doing so, we also advocate extending MaxSAT solvers to accept real-valued weights for soft clauses as input via the presented problem domains in which the use of real-valued costs plays an integral role.

scholarly journals The Strength of Nesterov's Extrapolation2019

2020 ◽  
Author(s):  
Qing Tao
Keyword(s):  
Machine Learning ◽  
Large Scale ◽  
Convergence Rates ◽  
Optimization Problems ◽  
Gradient Methods ◽  
Learning Problems ◽  
Smooth Convex ◽  
Simple Modification ◽  
Convex Problems ◽  
Hinge Loss

The extrapolation strategy raised by Nesterov, which can accelerate the convergence rate of gradient descent methods by orders of magnitude when dealing with smooth convex objective, has led to tremendous success in training machine learning tasks. In this paper, we theoretically study its strength in the convergence of individual iterates of general non-smooth convex optimization problems, which we name \textit{individual convergence}. We prove that Nesterov's extrapolation is capable of making the individual convergence of projected gradient methods optimal for general convex problems, which is now a challenging problem in the machine learning community. In light of this consideration, a simple modification of the gradient operation suffices to achieve optimal individual convergence for strongly convex problems, which can be regarded as making an interesting step towards the open question about SGD posed by Shamir \cite{shamir2012open}. Furthermore, the derived algorithms are extended to solve regularized non-smooth learning problems in stochastic settings. {\color{blue}They can serve as an alternative to the most basic SGD especially in coping with machine learning problems, where an individual output is needed to guarantee the regularization structure while keeping an optimal rate of convergence.} Typically, our method is applicable as an efficient tool for solving large-scale $l_1$-regularized hinge-loss learning problems. Several real experiments demonstrate that the derived algorithms not only achieve optimal individual convergence rates but also guarantee better sparsity than the averaged solution.

scholarly journals Entropy-Penalized Semidefinite Programming

2019 ◽  
Author(s):  
Mikhail Krechetov ◽  
Jakub Marecek ◽  
Yury Maximov ◽  
Martin Takac
Keyword(s):  
Machine Learning ◽  
Time Complexity ◽  
Optimization Problems ◽  
Linear Time ◽  
Broad Class ◽  
Low Rank ◽  
Learning Problems ◽  
Unified Framework ◽  
Gradient Computation ◽  
Machine Learning Applications

Low-rank methods for semi-definite programming (SDP) have gained a lot of interest recently, especially in machine learning applications. Their analysis often involves determinant-based or Schatten-norm penalties, which are difficult to implement in practice due to high computational efforts. In this paper, we propose Entropy-Penalized Semi-Definite Programming (EP-SDP), which provides a unified framework for a broad class of penalty functions used in practice to promote a low-rank solution. We show that EP-SDP problems admit an efficient numerical algorithm, having (almost) linear time complexity of the gradient computation; this makes it useful for many machine learning and optimization problems. We illustrate the practical efficiency of our approach on several combinatorial optimization and machine learning problems.

A Survey of Recent Variants and Applications of Antlion Optimizer

2021 ◽  
Vol 10 (2) ◽  
pp. 48-73
Author(s):  
Shail Dinkar ◽  
Kusum Deep
Keyword(s):  
Machine Learning ◽  
Real World ◽  
Optimization Problems ◽  
Environmental Engineering ◽  
Metaheuristic Algorithm ◽  
Natural Phenomena ◽  
Communication Engineering ◽  
Real World Applications ◽  
Working Principle ◽  
Complex Optimization

This work proposes a review of a recently developed swarm intelligence-based metaheuristic algorithm called Antlion Optimizer (ALO), its variants, and applications. The suitable blending of a random walk with an adaptive shrinking of hypersphere radius makes this algorithm more effective and impressive over other recent optimization algorithms. This paper elaborates on the recent variants of ALO by reviewing the concerned publications. It also summarized the applications of ALO for solving real-world complex optimization problems of a wide variety of areas. So, this paper comprises of summarized review of various recently published ALO papers. Firstly, the natural phenomena of ALO and the working principle of its various operators are described. Then the recently developed variants of ALO are described in detail depicting in various categories. The real-world applications using ALO and its variants are also described under global optimization, power and system engineering, electronics and communication engineering, machine learning, environmental engineering, and networking.

scholarly journals Supervised Learning Using Homology Stable Rank Kernels

2021 ◽  
Vol 7 ◽  
Author(s):  
Jens Agerberg ◽  
Ryan Ramanujam ◽  
Martina Scolamiero ◽  
Wojciech Chachólski
Keyword(s):  
Machine Learning ◽  
Data Analysis ◽  
Real World ◽  
Fundamental Property ◽  
Stable Rank ◽  
Topological Data Analysis ◽  
Improve Accuracy ◽  
Recent Developments ◽  
Classification Tasks ◽  
Real World Datasets

Exciting recent developments in Topological Data Analysis have aimed at combining homology-based invariants with Machine Learning. In this article, we use hierarchical stabilization to bridge between persistence and kernel-based methods by introducing the so-called stable rank kernels. A fundamental property of the stable rank kernels is that they depend on metrics to compare persistence modules. We illustrate their use on artificial and real-world datasets and show that by varying the metric we can improve accuracy in classification tasks.

scholarly journals Learning Deep Attention Network from Incremental and Decremental Features for Evolving Features

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Chuxin Wang ◽  
Haoran Mo
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Real World ◽  
Learning Problems ◽  
Data Sets ◽  
Attention Network ◽  
Benchmark Data ◽  
Multiple Cross ◽  
Feature Attention ◽  
The Cross

In many real-world machine learning problems, the features are changing along the time, with some old features vanishing and some other new features augmented, while the remaining features survived. In this paper, we propose the cross-feature attention network to handle the incremental and decremental features. This network is composed of multiple cross-feature attention encoding-decoding layers. In each layer, the data samples are firstly encoded by the combination of other samples with vanished/augmented features and weighted by the attention weights calculated by the survived features. Then, the samples are encoded by the combination of samples with the survived features weighted by the attention weights calculated from the encoded vanished/augmented feature data. The encoded vanished/augmented/survived features are then decoded and fed to the next cross-feature attention layer. In this way, the incremental and decremental features are bridged by paying attention to each other, and the gap between data samples with a different set of features are filled by the attention mechanism. The outputs of the cross-feature attention network are further concatenated and fed to the class-specific attention and global attention network for the purpose of classification. We evaluate the proposed network with benchmark data sets of computer vision, IoT, and bio-informatics, with incremental and decremental features. Encouraging experimental results show the effectiveness of our algorithm.

scholarly journals The Strength of Nesterov's Extrapolation2019

2020 ◽  
Author(s):  
Qing Tao
Keyword(s):  
Machine Learning ◽  
Large Scale ◽  
Convergence Rates ◽  
Optimization Problems ◽  
Gradient Methods ◽  
Learning Problems ◽  
Smooth Convex ◽  
Simple Modification ◽  
Convex Problems ◽  
Hinge Loss

The extrapolation strategy raised by Nesterov, which can accelerate the convergence rate of gradient descent methods by orders of magnitude when dealing with smooth convex objective, has led to tremendous success in training machine learning tasks. In this paper, we theoretically study its strength in the convergence of individual iterates of general non-smooth convex optimization problems, which we name \textit{individual convergence}. We prove that Nesterov's extrapolation is capable of making the individual convergence of projected gradient methods optimal for general convex problems, which is now a challenging problem in the machine learning community. In light of this consideration, a simple modification of the gradient operation suffices to achieve optimal individual convergence for strongly convex problems, which can be regarded as making an interesting step towards the open question about SGD posed by Shamir \cite{shamir2012open}. Furthermore, the derived algorithms are extended to solve regularized non-smooth learning problems in stochastic settings. {\color{blue}They can serve as an alternative to the most basic SGD especially in coping with machine learning problems, where an individual output is needed to guarantee the regularization structure while keeping an optimal rate of convergence.} Typically, our method is applicable as an efficient tool for solving large-scale $l_1$-regularized hinge-loss learning problems. Several real experiments demonstrate that the derived algorithms not only achieve optimal individual convergence rates but also guarantee better sparsity than the averaged solution.

scholarly journals Recognizing Power-law Graphs by Machine Learning Algorithms using a Reduced Set of Structural Features

2019 ◽  
Author(s):  
Alane Lima ◽  
André Vignatti ◽  
Murilo Silva
Keyword(s):  
Machine Learning ◽  
Power Law ◽  
Real World ◽  
Optimization Problems ◽  
Learning Algorithms ◽  
Structural Features ◽  
Machine Learning Algorithms ◽  
Graph Representation ◽  
Gradient Boosting ◽  
Graph Properties

The empirical study of large real world networks in the last 20 years showed that a variety of real-world graphs are power-law. There are evidence that optimization problems seem easier in these graphs; however, for a given graph, classifying it as power-law is a problem in itself. In this work, we propose using machine learning algorithms (KNN, SVM, Gradient Boosting and Random Forests) for this task. We suggest a graph representation based on [Canning et al. 2018], but using a much simplified set of structural properties of the graph. We compare the proposed representation with the one generated by the graph2vec framework. The experiments attained high accuracy, indicating that a reduced set of structural graph properties is enough for the presented problem.

scholarly journals Topological Machine Learning for Mixed Numeric and Categorical Data

2021 ◽  
Vol 30 (05) ◽  
pp. 2150025
Author(s):  
Chengyuan Wu ◽  
Carol Anne Hargreaves
Keyword(s):  
Machine Learning ◽  
Heart Disease ◽  
Data Analysis ◽  
Real World ◽  
Persistent Homology ◽  
Topological Data Analysis ◽  
Mixed Data ◽  
Topological Methods ◽  
Cloud Data ◽  
Topological Data

Topological data analysis is a relatively new branch of machine learning that excels in studying high-dimensional data, and is theoretically known to be robust against noise. Meanwhile, data objects with mixed numeric and categorical attributes are ubiquitous in real-world applications. However, topological methods are usually applied to point cloud data, and to the best of our knowledge there is no available framework for the classification of mixed data using topological methods. In this paper, we propose a novel topological machine learning method for mixed data classification. In the proposed method, we use theory from topological data analysis such as persistent homology, persistence diagrams and Wasserstein distance to study mixed data. The performance of the proposed method is demonstrated by experiments on a real-world heart disease dataset. Experimental results show that our topological method outperforms several state-of-the-art algorithms in the prediction of heart disease.

Sign in / Sign up

Export Citation Format

Share Document