scholarly journals Contrastive Losses and Solution Caching for Predict-and-Optimize

2021 ◽  
Author(s):  
Maxime Mulamba ◽  
Jayanta Mandi ◽  
Michelangelo Diligenti ◽  
Michele Lombardi ◽  
Victor Bucarey ◽  
...  
Keyword(s):  
Optimization Problem ◽  
Computational Cost ◽  
Learning Approaches ◽  
Optimal Solutions ◽  
Training Time ◽  
Slow Growth Rate ◽  
Historic Data ◽  
Uncertain Input ◽  
Feasible Space ◽  
Inner Approximation

Many decision-making processes involve solving a combinatorial optimization problem with uncertain input that can be estimated from historic data. Recently, problems in this class have been successfully addressed via end-to-end learning approaches, which rely on solving one optimization problem for each training instance at every epoch. In this context, we provide two distinct contributions. First, we use a Noise Contrastive approach to motivate a family of surrogate loss functions, based on viewing non-optimal solutions as negative examples. Second, we address a major bottleneck of all predict-and-optimize approaches, i.e. the need to frequently recompute optimal solutions at training time. This is done via a solver-agnostic solution caching scheme, and by replacing optimization calls with a lookup in the solution cache. The method is formally based on an inner approximation of the feasible space and, combined with a cache lookup strategy, provides a controllable trade-off between training time and accuracy of the loss approximation. We empirically show that even a very slow growth rate is enough to match the quality of state-of-the-art methods, at a fraction of the computational cost.

scholarly journals Label consistent transform learning for pattern classification

2019 ◽  
Vol 13 ◽  
pp. 174830261988139
Author(s):  
He-Feng Yin ◽  
Xiao-Jun Wu
Keyword(s):  
Image Processing ◽  
Pattern Classification ◽  
Dictionary Learning ◽  
Optimization Problem ◽  
Classification Performance ◽  
Alternative Strategy ◽  
Learning Approaches ◽  
Training Time ◽  
Practical Applications ◽  
Consistency Constraint

Transform learning has been successfully applied to various image processing tasks in recent years. Nevertheless, transform learning learns the representation in an unsupervised fashion. To make transform learning suitable for pattern classification, we introduce a label consistency constraint into transform learning and propose a new label consistent transform learning to enhance the classification performance of transform learning. The resulting optimization problem can be solved elegantly by employing the alternative strategy. Experimental results on publicly available databases demonstrate that label consistent transform learning outperforms several dictionary learning approaches and the recently proposed discriminative transform learning. More importantly, label consistent transform learning has the least training time which has the potential in practical applications.

scholarly journals Evaluation of Mixed Deep Neural Networks for Reverberant Speech Enhancement

Biomimetics ◽  
2019 ◽  
Vol 5 (1) ◽  
pp. 1 ◽  
Author(s):  
Michelle Gutiérrez-Muñoz ◽  
Astryd González-Salazar ◽  
Marvin Coto-Jiménez
Keyword(s):  
Neural Networks ◽  
Short Term Memory ◽  
Computational Cost ◽  
Real Life ◽  
Fixed Number ◽  
Training Procedure ◽  
Statistical Validation ◽  
Significant Drop ◽  
Training Time ◽  
Important Solution

Speech signals are degraded in real-life environments, as a product of background noise or other factors. The processing of such signals for voice recognition and voice analysis systems presents important challenges. One of the conditions that make adverse quality difficult to handle in those systems is reverberation, produced by sound wave reflections that travel from the source to the microphone in multiple directions. To enhance signals in such adverse conditions, several deep learning-based methods have been proposed and proven to be effective. Recently, recurrent neural networks, especially those with long short-term memory (LSTM), have presented surprising results in tasks related to time-dependent processing of signals, such as speech. One of the most challenging aspects of LSTM networks is the high computational cost of the training procedure, which has limited extended experimentation in several cases. In this work, we present a proposal to evaluate the hybrid models of neural networks to learn different reverberation conditions without any previous information. The results show that some combinations of LSTM and perceptron layers produce good results in comparison to those from pure LSTM networks, given a fixed number of layers. The evaluation was made based on quality measurements of the signal’s spectrum, the training time of the networks, and statistical validation of results. In total, 120 artificial neural networks of eight different types were trained and compared. The results help to affirm the fact that hybrid networks represent an important solution for speech signal enhancement, given that reduction in training time is on the order of 30%, in processes that can normally take several days or weeks, depending on the amount of data. The results also present advantages in efficiency, but without a significant drop in quality.

scholarly journals A Fast and Exact Greedy Algorithm for the Core–Periphery Problem

Symmetry ◽  
2020 ◽  
Vol 12 (1) ◽  
pp. 94 ◽  
Author(s):  
Dario Fasino ◽  
Franca Rinaldi
Keyword(s):  
Structural Analysis ◽  
Optimization Problem ◽  
Polynomial Time Algorithm ◽  
Time Algorithm ◽  
Combinatorial Optimization Problem ◽  
Connected Subgraph ◽  
Optimal Solutions ◽  
The Core ◽  
Core Set ◽  
Key Concepts

The core–periphery structure is one of the key concepts in the structural analysis of complex networks. It consists of a partitioning of the node set of a given graph or network into two groups, called core and periphery, where the core nodes induce a well-connected subgraph and share connections with peripheral nodes, while the peripheral nodes are loosely connected to the core nodes and other peripheral nodes. We propose a polynomial-time algorithm to detect core–periphery structures in networks having a symmetric adjacency matrix. The core set is defined as the solution of a combinatorial optimization problem, which has a pleasant symmetry with respect to graph complementation. We provide a complete description of the optimal solutions to that problem and an exact and efficient algorithm to compute them. The proposed approach is extended to networks with loops and oriented edges. Numerical simulations are carried out on both synthetic and real-world networks to demonstrate the effectiveness and practicability of the proposed algorithm.

Accelerated 3D blind separation of convolved mixtures based on the fast iterative shrinkage thresholding algorithm for adaptive multiple subtraction

Geophysics ◽  
2018 ◽  
Vol 83 (2) ◽  
pp. V99-V113 ◽  
Author(s):  
Zhong-Xiao Li ◽  
Zhen-Chun Li
Keyword(s):  
Field Data ◽  
Optimization Problem ◽  
Computational Cost ◽  
Computation Time ◽  
Blind Separation ◽  
L1 Norm ◽  
Norm Minimization ◽  
Iterative Shrinkage ◽  
Thresholding Algorithm ◽  
Iterative Shrinkage Thresholding

After multiple prediction, adaptive multiple subtraction is essential for the success of multiple removal. The 3D blind separation of convolved mixtures (3D BSCM) method, which is effective in conducting adaptive multiple subtraction, needs to solve an optimization problem containing L1-norm minimization constraints on primaries by the iterative reweighted least-squares (IRLS) algorithm. The 3D BSCM method can better separate primaries and multiples than the 1D/2D BSCM method and the method with energy minimization constraints on primaries. However, the 3D BSCM method has high computational cost because the IRLS algorithm achieves nonquadratic optimization with an LS optimization problem solved in each iteration. In general, it is good to have a faster 3D BSCM method. To improve the adaptability of field data processing, the fast iterative shrinkage thresholding algorithm (FISTA) is introduced into the 3D BSCM method. The proximity operator of FISTA can solve the L1-norm minimization problem efficiently. We demonstrate that our FISTA-based 3D BSCM method achieves similar accuracy of estimating primaries as that of the reference IRLS-based 3D BSCM method. Furthermore, our FISTA-based 3D BSCM method reduces computation time by approximately 60% compared with the reference IRLS-based 3D BSCM method in the synthetic and field data examples.

Adaptive Grouping Brain Storm Optimization for Multimodal Optimization Problems

2021 ◽  
Vol 12 (4) ◽  
pp. 81-100
Author(s):  
Yao Peng ◽  
Zepeng Shen ◽  
Shiqi Wang
Keyword(s):  
Optimization Problem ◽  
Optimization Problems ◽  
Optimal Algorithm ◽  
Peak Detection ◽  
Multimodal Optimization ◽  
Optimal Solutions ◽  
Grouping Strategy ◽  
Different Dimensions ◽  
Global Optimal ◽  
Localization Ability

Multimodal optimization problem exists in multiple global and many local optimal solutions. The difficulty of solving these problems is finding as many local optimal peaks as possible on the premise of ensuring global optimal precision. This article presents adaptive grouping brainstorm optimization (AGBSO) for solving these problems. In this article, adaptive grouping strategy is proposed for achieving adaptive grouping without providing any prior knowledge by users. For enhancing the diversity and accuracy of the optimal algorithm, elite reservation strategy is proposed to put central particles into an elite pool, and peak detection strategy is proposed to delete particles far from optimal peaks in the elite pool. Finally, this article uses testing functions with different dimensions to compare the convergence, accuracy, and diversity of AGBSO with BSO. Experiments verify that AGBSO has great localization ability for local optimal solutions while ensuring the accuracy of the global optimal solutions.

scholarly journals Shift Invariant Dictionary Learning for Human Action Recognition

2019 ◽  
Vol 9 (2) ◽  
pp. 2107-2111
Keyword(s):  
Sparse Representation ◽  
Action Recognition ◽  
Dictionary Learning ◽  
Human Action Recognition ◽  
Human Action ◽  
Signal Denoising ◽  
Learning Approaches ◽  
Training Time ◽  
Initial Stage ◽  
Shift Invariance

Sparse representation is an emerging topic among researchers. The method to represent the huge volume of dense data as sparse data is much needed for various fields such as classification, compression and signal denoising. The base of the sparse representation is dictionary learning. In most of the dictionary learning approaches, the dictionary is learnt based on the input training signals which consumes more time. To solve this issue, the shift-invariant dictionary is used for action recognition in this work. Shift-Invariant Dictionary (SID) is that the dictionary is constructed in the initial stage with shift-invariance of initial atoms. The advantage of the proposed SID based action recognition method is that it requires minimum training time and achieves highest accuracy.

scholarly journals Evaluation of Mixed Deep Neural Networks for Reverberant Speech Enhancement

2019 ◽  
Author(s):  
Michelle Gutiérrez-Muñoz ◽  
Astryd González-Salazar ◽  
Marvin Coto-Jiménez
Keyword(s):  
Neural Networks ◽  
Short Term Memory ◽  
Computational Cost ◽  
Real Life ◽  
Fixed Number ◽  
Training Procedure ◽  
Statistical Validation ◽  
Training Time ◽  
Important Solution ◽  
Reverberant Speech

Speech signals are degraded in real-life environments, product of background noise or other factors. The processing of such signals for voice recognition and voice analysis systems presents important challenges. One of the conditions that make adverse quality difficult to handle in those systems is reverberation, produced by sound wave reflections that travel from the source to the microphone in multiple directions.To enhance signals in such adverse conditions, several deep learning-based methods have been proposed and proven to be effective. Recently, recurrent neural networks, especially those with long and short-term memory (LSTM), have presented surprising results in tasks related to time-dependent processing of signals, such as speech. One of the most challenging aspects of LSTM networks is the high computational cost of the training procedure, which has limited extended experimentation in several cases. In this work, we present a proposal to evaluate the hybrid models of neural networks to learn different reverberation conditions without any previous information. The results show that some combination of LSTM and perceptron layers produce good results in comparison to those from pure LSTM networks, given a fixed number of layers. The evaluation has been made based on quality measurements of the signal's spectrum, training time of the networks and statistical validation of results. Results help to affirm the fact that hybrid networks represent an important solution for speech signal enhancement, with advantages in efficiency, but without a significan drop in quality.

Space Contraction and Partition Algorithm for Combinatorial Optimization

2011 ◽  
Vol 421 ◽  
pp. 559-563
Author(s):  
Yong Chao Gao ◽  
Li Mei Liu ◽  
Heng Qian ◽  
Ding Wang
Keyword(s):  
Combinatorial Optimization ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Solution Space ◽  
Search Space ◽  
Small Scale ◽  
Optimal Solutions ◽  
Solution Quality ◽  
Intelligent Algorithms ◽  
Combinatorial Optimization Problems

The scale and complexity of search space are important factors deciding the solving difficulty of an optimization problem. The information of solution space may lead searching to optimal solutions. Based on this, an algorithm for combinatorial optimization is proposed. This algorithm makes use of the good solutions found by intelligent algorithms, contracts the search space and partitions it into one or several optimal regions by backbones of combinatorial optimization solutions. And optimization of small-scale problems is carried out in optimal regions. Statistical analysis is not necessary before or through the solving process in this algorithm, and solution information is used to estimate the landscape of search space, which enhances the speed of solving and solution quality. The algorithm breaks a new path for solving combinatorial optimization problems, and the results of experiments also testify its efficiency.

Tiled Sparse Coding in Eigenspaces for Image Classification

2021 ◽  
Author(s):  
Juan E Arco ◽  
Andrés Ortiz ◽  
Javier Ramírez ◽  
Yu-Dong Zhang ◽  
Juan M Górriz
Keyword(s):  
Sparse Coding ◽  
Computational Cost ◽  
Learning Approaches ◽  
Classification Framework ◽  
Large Databases ◽  
Class Context ◽  
Control Versus ◽  
Complex Solutions ◽  
Aided Diagnosis ◽  
High Computational Cost

The automation in the diagnosis of medical images is currently a challenging task. The use of Computer Aided Diagnosis (CAD) systems can be a powerful tool for clinicians, especially in situations when hospitals are overflowed. These tools are usually based on artificial intelligence (AI), a field that has been recently revolutionized by deep learning approaches. blackThese alternatives usually obtain a large performance based on complex solutions, leading to a high computational cost and the need of having large databases. In this work, we propose a classification framework based on sparse coding. Images are blackfirst partitioned into different tiles, and a dictionary is built after applying PCA to these tiles. The original signals are then transformed as a linear combination of the elements of the dictionary. blackThen, they are reconstructed by iteratively deactivating the elements associated with each component. Classification is finally performed employing as features the subsequent reconstruction errors. Performance is evaluated in a real context where distinguishing between four different pathologies: control versus bacterial pneumonia versus viral pneumonia versus COVID-19. blackOur system differentiates between pneumonia patients and controls with an accuracy of 97.74%, whereas in the 4-class context the accuracy is 86.73%. The excellent results and the pioneering use of sparse coding in this scenario evidence that our proposal can assist clinicians when their workload is high.

scholarly journals Evaluation Method of Multiobjective Functions’ Combination and Its Application in Hydrological Model Evaluation

2020 ◽  
Vol 2020 ◽  
pp. 1-23 ◽  
Author(s):  
Jiuyuan Huo ◽  
Liqun Liu
Keyword(s):  
Objective Function ◽  
Parameter Optimization ◽  
Optimization Problem ◽  
Hydrological Model ◽  
Pareto Optimal ◽  
Pareto Optimal Solutions ◽  
Optimal Solutions ◽  
Objective Functions ◽  
Hydrological Forecasting ◽  
Model Parameter Optimization

Parameter optimization of a hydrological model is intrinsically a high dimensional, nonlinear, multivariable, combinatorial optimization problem which involves a set of different objectives. Currently, the assessment of optimization results for the hydrological model is usually made through calculations and comparisons of objective function values of simulated and observed variables. Thus, the proper selection of objective functions’ combination for model parameter optimization has an important impact on the hydrological forecasting. There exist various objective functions, and how to analyze and evaluate the objective function combinations for selecting the optimal parameters has not been studied in depth. Therefore, to select the proper objective function combination which can balance the trade-off among various design objectives and achieve the overall best benefit, a simple and convenient framework for the comparison of the influence of different objective function combinations on the optimization results is urgently needed. In this paper, various objective functions related to parameters optimization of hydrological models were collected from the literature and constructed to nine combinations. Then, a selection and evaluation framework of objective functions is proposed for hydrological model parameter optimization, in which a multiobjective artificial bee colony algorithm named RMOABC is employed to optimize the hydrological model and obtain the Pareto optimal solutions. The parameter optimization problem of the Xinanjiang hydrological model was taken as the application case for long-term runoff prediction in the Heihe River basin. Finally, the technique for order preference by similarity to ideal solution (TOPSIS) based on the entropy theory is adapted to sort the Pareto optimal solutions to compare these combinations of objective functions and obtain the comprehensive optimal objective functions’ combination. The experiments results demonstrate that the combination 2 of objective functions can provide more comprehensive and reliable dominant options (i.e., parameter sets) for practical hydrological forecasting in the study area. The entropy-based method has been proved that it is effective to analyze and evaluate the performance of different combinations of objective functions and can provide more comprehensive and impersonal decision support for hydrological forecasting.

Sign in / Sign up

Export Citation Format

Share Document