scholarly journals Robust Learning from Noisy Side-information by Semidefinite Programming

2019 ◽  
Author(s):  
En-Liang Hu ◽  
Quanming Yao
Keyword(s):  
Machine Learning ◽  
Empirical Study ◽  
Learning Community ◽  
Critical Points ◽  
Efficient Algorithm ◽  
State Of The Art ◽  
Side Information ◽  
Low Rank ◽  
Robust Learning ◽  
Speed And Accuracy

Robustness recently becomes one of the major concerns among machine learning community, since learning algorithms are usually vulnerable to outliers or corruptions. Motivated by such a trend and needs, we pursue robustness in semi-definite programming (SDP) in this paper. Specifically, this is done by replacing the commonly used squared loss with the more robust L1-loss in the low-rank SDP. However, the resulting objective becomes neither convex nor smooth. As no existing algorithms can be applied, we design an efficient algorithm, based on majorization-minimization, to optimize the objective. The proposed algorithm not only has cheap iterations and low space complexity but also theoretically converges to some critical points. Finally, empirical study shows that the new objective armed with proposed algorithm outperforms state-of-the-art in terms of both speed and accuracy.

scholarly journals Stochastic Perturbations on Low-Rank Hyperspectral Data for Image Classification

2021 ◽  
Vol 5 (1) ◽  
pp. 1
Author(s):  
Alex Sumarsono ◽  
Farnaz Ganjeizadeh ◽  
Ryan Tomasi
Keyword(s):  
Machine Learning ◽  
Least Squares ◽  
Classification Accuracy ◽  
State Of The Art ◽  
Hyperspectral Data ◽  
Low Rank ◽  
Stochastic Perturbations ◽  
Sparse Decomposition ◽  
Spectral Signatures ◽  
Material Substances

Hyperspectral imagery (HSI) contains hundreds of narrow contiguous bands of spectral signals. These signals, which form spectral signatures, provide a wealth of information that can be used to characterize material substances. In recent years machine learning has been used extensively to classify HSI data. While many excellent HSI classifiers have been proposed and deployed, the focus has been more on the design of the algorithms. This paper presents a novel data preprocessing method (LRSP) to improve classification accuracy by applying stochastic perturbations to the low-rank constituent of the dataset. The proposed architecture is composed of a low-rank and sparse decomposition, a degradation function and a constraint least squares filter. Experimental results confirm that popular state-of-the-art HSI classifiers can produce better classification results if supplied by LRSP-altered datasets rather than the original HSI datasets. 

scholarly journals Learning Aggregation Functions

2021 ◽  
Author(s):  
Giovanni Pellegrini ◽  
Alessandro Tibo ◽  
Paolo Frasconi ◽  
Andrea Passerini ◽  
Manfred Jaeger
Keyword(s):  
Machine Learning ◽  
Learning Community ◽  
State Of The Art ◽  
Real Data ◽  
Universal Function ◽  
Aggregation Function ◽  
Function Representation ◽  
Aggregation Functions ◽  
Complex Functions ◽  
Latent Dimension

Learning on sets is increasingly gaining attention in the machine learning community, due to its widespread applicability. Typically, representations over sets are computed by using fixed aggregation functions such as sum or maximum. However, recent results showed that universal function representation by sum- (or max-) decomposition requires either highly discontinuous (and thus poorly learnable) mappings, or a latent dimension equal to the maximum number of elements in the set. To mitigate this problem, we introduce LAF (Learning Aggregation Function), a learnable aggregator for sets of arbitrary cardinality. LAF can approximate several extensively used aggregators (such as average, sum, maximum) as well as more complex functions (e.g. variance and skewness). We report experiments on semi-synthetic and real data showing that LAF outperforms state-of-the-art sum- (max-) decomposition architectures such as DeepSets and library-based architectures like Principal Neighborhood Aggregation, and can be effectively combined with attention-based architectures.

scholarly journals Tensor Completion with Side Information: A Riemannian Manifold Approach

2017 ◽  
Author(s):  
Tengfei Zhou ◽  
Hui Qian ◽  
Zebang Shen ◽  
Chao Zhang ◽  
Congfu Xu
Keyword(s):  
Gradient Descent ◽  
State Of The Art ◽  
Side Information ◽  
Optimization Methods ◽  
Original Model ◽  
Low Rank ◽  
Tensor Completion ◽  
Riemannian Optimization ◽  
Rank Tensor ◽  
Completion Problems

By restricting the iterate on a nonlinear manifold, the recently proposed Riemannian optimization methods prove to be both efficient and effective in low rank tensor completion problems. However, existing methods fail to exploit the easily accessible side information, due to their format mismatch. Consequently, there is still room for improvement. To fill the gap, in this paper, a novel Riemannian model is proposed to tightly integrate the original model and the side information by overcoming their inconsistency. For this model, an efficient Riemannian conjugate gradient descent solver is devised based on a new metric that captures the curvature of the objective. Numerical experiments suggest that our method is more accurate than the state-of-the-art without compromising the efficiency.

scholarly journals Approximate Optimal Transport for Continuous Densities with Copulas

2019 ◽  
Author(s):  
Jinjin Chi ◽  
Jihong Ouyang ◽  
Ximing Li ◽  
Yang Wang ◽  
Meng Wang
Keyword(s):  
Machine Learning ◽  
Stochastic Optimization ◽  
Image Retrieval ◽  
Learning Community ◽  
Optimal Transport ◽  
State Of The Art ◽  
Probability Distributions ◽  
Synthetic Data ◽  
Computational Burden ◽  
Copula Parameter

Optimal Transport (OT) formulates a powerful framework by comparing probability distributions, and it has increasingly attracted great attention within the machine learning community. However, it suffers from severe computational burden, due to the intractable objective with respect to the distributions of interest. Especially, there still exist very few attempts for continuous OT, i.e., OT for comparing continuous densities. To this end, we develop a novel continuous OT method, namely Copula OT (Cop-OT). The basic idea is to transform the primal objective of continuous OT into a tractable form with respect to the copula parameter, which can be efficiently solved by stochastic optimization with less time and memory requirements. Empirical results on real applications of image retrieval and synthetic data demonstrate that our Cop-OT can gain more accurate approximations to continuous OT values than the state-of-the-art baselines.

ORGANIC (1).pdf

2017 ◽  
Author(s):  
Benjamin Sanchez-Lengeling ◽  
Carlos Outeiral ◽  
Gabriel L. Guimaraes ◽  
Alan Aspuru-Guzik
Keyword(s):  
Machine Learning ◽  
Learning Community ◽  
Chemical Species ◽  
Material Design ◽  
Organic Photovoltaic ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Photovoltaic Material

Molecular discovery seeks to generate chemical species tailored to very specific needs. In this paper, we present ORGANIC, a framework based on Objective-Reinforced Generative Adversarial Networks (ORGAN), capable of producing a distribution over molecular space that matches with a certain set of desirable metrics. This methodology combines two successful techniques from the machine learning community: a Generative Adversarial Network (GAN), to create non-repetitive sensible molecular species, and Reinforcement Learning (RL), to bias this generative distribution towards certain attributes. We explore several applications, from optimization of random physicochemical properties to candidates for drug discovery and organic photovoltaic material design.

Data science in economics: comprehensive review of advanced machine learning and deep learning methods

2020 ◽  
Author(s):  
Saeed Nosratabadi ◽  
Amir Mosavi ◽  
Puhong Duan ◽  
Pedram Ghamisi ◽  
Ferdinand Filip ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Data Science ◽  
State Of The Art ◽  
Science Methods ◽  
Learning Models ◽  
Diverse Range ◽  
Hybrid Machine ◽  
Economics Research

This paper provides a state-of-the-art investigation of advances in data science in emerging economic applications. The analysis was performed on novel data science methods in four individual classes of deep learning models, hybrid deep learning models, hybrid machine learning, and ensemble models. Application domains include a wide and diverse range of economics research from the stock market, marketing, and e-commerce to corporate banking and cryptocurrency. Prisma method, a systematic literature review methodology, was used to ensure the quality of the survey. The findings reveal that the trends follow the advancement of hybrid models, which, based on the accuracy metric, outperform other learning algorithms. It is further expected that the trends will converge toward the advancements of sophisticated hybrid deep learning models.

Deep Learning for text in limted data settings

2020 ◽  
Author(s):  
Pathikkumar Patel ◽  
Bhargav Lad ◽  
Jinan Fiaidhi
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Transfer Learning ◽  
Text Classification ◽  
State Of The Art ◽  
Time Series Forecasting ◽  
Text Data ◽  
Performance Levels

During the last few years, RNN models have been extensively used and they have proven to be better for sequence and text data. RNNs have achieved state-of-the-art performance levels in several applications such as text classification, sequence to sequence modelling and time series forecasting. In this article we will review different Machine Learning and Deep Learning based approaches for text data and look at the results obtained from these methods. This work also explores the use of transfer learning in NLP and how it affects the performance of models on a specific application of sentiment analysis.

scholarly journals Multi-hop assortativities for network classification

2018 ◽  
Vol 7 (4) ◽  
pp. 603-622 ◽  
Author(s):  
Leonardo Gutiérrez-Gómez ◽  
Jean-Charles Delvenne
Keyword(s):  
Machine Learning ◽  
Scientific Collaboration ◽  
State Of The Art ◽  
Medical Engineering ◽  
Research Field ◽  
Classification Task ◽  
Collaboration Network ◽  
Structural Patterns ◽  
Art Methods

Abstract Several social, medical, engineering and biological challenges rely on discovering the functionality of networks from their structure and node metadata, when it is available. For example, in chemoinformatics one might want to detect whether a molecule is toxic based on structure and atomic types, or discover the research field of a scientific collaboration network. Existing techniques rely on counting or measuring structural patterns that are known to show large variations from network to network, such as the number of triangles, or the assortativity of node metadata. We introduce the concept of multi-hop assortativity, that captures the similarity of the nodes situated at the extremities of a randomly selected path of a given length. We show that multi-hop assortativity unifies various existing concepts and offers a versatile family of ‘fingerprints’ to characterize networks. These fingerprints allow in turn to recover the functionalities of a network, with the help of the machine learning toolbox. Our method is evaluated empirically on established social and chemoinformatic network benchmarks. Results reveal that our assortativity based features are competitive providing highly accurate results often outperforming state of the art methods for the network classification task.

scholarly journals Advances in the Application of Machine Learning Techniques for Power System Analytics: A Survey

Energies ◽  
2021 ◽  
Vol 14 (16) ◽  
pp. 4776
Author(s):  
Seyed Mahdi Miraftabzadeh ◽  
Michela Longo ◽  
Federica Foiadelli ◽  
Marco Pasetti ◽  
Raul Igual
Keyword(s):  
Machine Learning ◽  
Power Systems ◽  
Smart Grids ◽  
State Of The Art ◽  
Smart Cities ◽  
Power Grids ◽  
Machine Learning Techniques ◽  
Learning Techniques ◽  
New Research ◽  
Traditional Approaches

The recent advances in computing technologies and the increasing availability of large amounts of data in smart grids and smart cities are generating new research opportunities in the application of Machine Learning (ML) for improving the observability and efficiency of modern power grids. However, as the number and diversity of ML techniques increase, questions arise about their performance and applicability, and on the most suitable ML method depending on the specific application. Trying to answer these questions, this manuscript presents a systematic review of the state-of-the-art studies implementing ML techniques in the context of power systems, with a specific focus on the analysis of power flows, power quality, photovoltaic systems, intelligent transportation, and load forecasting. The survey investigates, for each of the selected topics, the most recent and promising ML techniques proposed by the literature, by highlighting their main characteristics and relevant results. The review revealed that, when compared to traditional approaches, ML algorithms can handle massive quantities of data with high dimensionality, by allowing the identification of hidden characteristics of (even) complex systems. In particular, even though very different techniques can be used for each application, hybrid models generally show better performances when compared to single ML-based models.

Sign in / Sign up

Export Citation Format

Share Document