scholarly journals A Unifying Representer Theorem for Inverse Problems and Machine Learning

2020 ◽  
Author(s):  
Michael Unser
Keyword(s):  
Machine Learning ◽  
Banach Spaces ◽  
Tikhonov Regularization ◽  
Optimization Problems ◽  
Broad Class ◽  
Training Problem ◽  
Optimization Task ◽  
Machine Leaning ◽  
Sparse Solutions ◽  
Standard Strategy

Abstract Regularization addresses the ill-posedness of the training problem in machine learning or the reconstruction of a signal from a limited number of measurements. The method is applicable whenever the problem is formulated as an optimization task. The standard strategy consists in augmenting the original cost functional by an energy that penalizes solutions with undesirable behavior. The effect of regularization is very well understood when the penalty involves a Hilbertian norm. Another popular configuration is the use of an $$\ell _1$$ ℓ 1 -norm (or some variant thereof) that favors sparse solutions. In this paper, we propose a higher-level formulation of regularization within the context of Banach spaces. We present a general representer theorem that characterizes the solutions of a remarkably broad class of optimization problems. We then use our theorem to retrieve a number of known results in the literature such as the celebrated representer theorem of machine leaning for RKHS, Tikhonov regularization, representer theorems for sparsity promoting functionals, the recovery of spikes, as well as a few new ones.

scholarly journals Entropy-Penalized Semidefinite Programming

2019 ◽  
Author(s):  
Mikhail Krechetov ◽  
Jakub Marecek ◽  
Yury Maximov ◽  
Martin Takac
Keyword(s):  
Machine Learning ◽  
Time Complexity ◽  
Optimization Problems ◽  
Linear Time ◽  
Broad Class ◽  
Low Rank ◽  
Learning Problems ◽  
Unified Framework ◽  
Gradient Computation ◽  
Machine Learning Applications

Low-rank methods for semi-definite programming (SDP) have gained a lot of interest recently, especially in machine learning applications. Their analysis often involves determinant-based or Schatten-norm penalties, which are difficult to implement in practice due to high computational efforts. In this paper, we propose Entropy-Penalized Semi-Definite Programming (EP-SDP), which provides a unified framework for a broad class of penalty functions used in practice to promote a low-rank solution. We show that EP-SDP problems admit an efficient numerical algorithm, having (almost) linear time complexity of the gradient computation; this makes it useful for many machine learning and optimization problems. We illustrate the practical efficiency of our approach on several combinatorial optimization and machine learning problems.

scholarly journals Practical CO2—WAG Field Operational Designs Using Hybrid Numerical-Machine-Learning Approaches

Energies ◽  
2021 ◽  
Vol 14 (4) ◽  
pp. 1055
Author(s):  
Qian Sun ◽  
William Ampomah ◽  
Junyu You ◽  
Martha Cather ◽  
Robert Balch
Keyword(s):  
Machine Learning ◽  
Oil Recovery ◽  
History Matching ◽  
Optimization Problems ◽  
Learning Technologies ◽  
Petroleum Engineering ◽  
Support Vector ◽  
Learning Approaches ◽  
Field Development ◽  
Proxy Models

Machine-learning technologies have exhibited robust competences in solving many petroleum engineering problems. The accurate predictivity and fast computational speed enable a large volume of time-consuming engineering processes such as history-matching and field development optimization. The Southwest Regional Partnership on Carbon Sequestration (SWP) project desires rigorous history-matching and multi-objective optimization processes, which fits the superiorities of the machine-learning approaches. Although the machine-learning proxy models are trained and validated before imposing to solve practical problems, the error margin would essentially introduce uncertainties to the results. In this paper, a hybrid numerical machine-learning workflow solving various optimization problems is presented. By coupling the expert machine-learning proxies with a global optimizer, the workflow successfully solves the history-matching and CO2 water alternative gas (WAG) design problem with low computational overheads. The history-matching work considers the heterogeneities of multiphase relative characteristics, and the CO2-WAG injection design takes multiple techno-economic objective functions into accounts. This work trained an expert response surface, a support vector machine, and a multi-layer neural network as proxy models to effectively learn the high-dimensional nonlinear data structure. The proposed workflow suggests revisiting the high-fidelity numerical simulator for validation purposes. The experience gained from this work would provide valuable guiding insights to similar CO2 enhanced oil recovery (EOR) projects.

scholarly journals A Self-Adaptive Cuckoo Search Algorithm Using a Machine Learning Technique

Mathematics ◽  
2021 ◽  
Vol 9 (16) ◽  
pp. 1840
Author(s):  
Nicolás Caselli ◽  
Ricardo Soto ◽  
Broderick Crawford ◽  
Sergio Valdivia ◽  
Rodrigo Olivares
Keyword(s):  
Machine Learning ◽  
Optimization Problems ◽  
Search Algorithm ◽  
Cuckoo Search ◽  
Cuckoo Search Algorithm ◽  
Set Covering ◽  
Covering Problem ◽  
Parameter Configuration ◽  
Self Tuning ◽  
Problem Solvers

Metaheuristics are intelligent problem-solvers that have been very efficient in solving huge optimization problems for more than two decades. However, the main drawback of these solvers is the need for problem-dependent and complex parameter setting in order to reach good results. This paper presents a new cuckoo search algorithm able to self-adapt its configuration, particularly its population and the abandon probability. The self-tuning process is governed by using machine learning, where cluster analysis is employed to autonomously and properly compute the number of agents needed at each step of the solving process. The goal is to efficiently explore the space of possible solutions while alleviating human effort in parameter configuration. We illustrate interesting experimental results on the well-known set covering problem, where the proposed approach is able to compete against various state-of-the-art algorithms, achieving better results in one single run versus 20 different configurations. In addition, the result obtained is compared with similar hybrid bio-inspired algorithms illustrating interesting results for this proposal.

scholarly journals An Overview of Machine Learning-Based Techniques for Solving Optimization Problems in Communications and Signal Processing

IEEE Access ◽  
2021 ◽  
pp. 1-1
Author(s):  
Hayssam Dahrouj ◽  
Rawan Alghamdi ◽  
Hibatallah Alwazani ◽  
Sarah Bahanshal ◽  
Alaa Alameer Ahmad ◽  
...  
Keyword(s):  
Machine Learning ◽  
Signal Processing ◽  
Optimization Problems

scholarly journals The iterative solutions of split common fixed point problem for asymptotically nonexpansive mappings in Banach spaces

2020 ◽  
Vol 2020 (1) ◽  
Author(s):  
Yuanheng Wang ◽  
Xiuping Wu ◽  
Chanjuan Pan
Keyword(s):  
Fixed Point ◽  
Common Fixed Point ◽  
Banach Spaces ◽  
Optimization Problems ◽  
Nonexpansive Mappings ◽  
Fixed Point Problem ◽  
Iteration Algorithm ◽  
Point Problem ◽  
Asymptotically Nonexpansive ◽  
Split Common Fixed Point

AbstractIn this paper, we propose an iteration algorithm for finding a split common fixed point of an asymptotically nonexpansive mapping in the frameworks of two real Banach spaces. Under some suitable conditions imposed on the sequences of parameters, some strong convergence theorems are proved, which also solve some variational inequalities that are closely related to optimization problems. The results here generalize and improve the main results of other authors.

scholarly journals Structure of Pareto Solutions of Generalized Polyhedral-Valued Vector Optimization Problems in Banach Spaces

2013 ◽  
Vol 2013 ◽  
pp. 1-10
Author(s):  
Qinghai He ◽  
Weili Kong
Keyword(s):  
Banach Spaces ◽  
Vector Optimization ◽  
Optimization Problems ◽  
Vector Optimization Problem ◽  
Solution Set ◽  
Pareto Optimal ◽  
Pareto Solution ◽  
Value Set ◽  
Set Valued Mapping ◽  
Optimal Value

In general Banach spaces, we consider a vector optimization problem (SVOP) in which the objective is a set-valued mapping whose graph is the union of finitely many polyhedra or the union of finitely many generalized polyhedra. Dropping the compactness assumption, we establish some results on structure of the weak Pareto solution set, Pareto solution set, weak Pareto optimal value set, and Pareto optimal value set of (SVOP) and on connectedness of Pareto solution set and Pareto optimal value set of (SVOP). In particular, we improved and generalize, Arrow, Barankin, and Blackwell’s classical results in Euclidean spaces and Zheng and Yang’s results in general Banach spaces.

scholarly journals Some relations between variational-like inequality problems and vectorial optimization problems in Banach spaces

2008 ◽  
Vol 55 (8) ◽  
pp. 1808-1814 ◽  
Author(s):  
Lucelina Batista dos Santos ◽  
Gabriel Ruiz-Garzón ◽  
Marko A. Rojas-Medar ◽  
Antonio Rufián-Lizana
Keyword(s):  
Banach Spaces ◽  
Optimization Problems

Swarm Intelligence Optimization: An Exploration and Application of Machine Learning Technology

2021 ◽  
Vol 30 (1) ◽  
pp. 460-469
Author(s):  
Yinying Cai ◽  
Amit Sharma
Keyword(s):  
Machine Learning ◽  
Swarm Intelligence ◽  
Research Result ◽  
Machine Learning Algorithms ◽  
Learning Technology ◽  
Data Set ◽  
Rice Pests ◽  
Machine Leaning ◽  
Smart Agriculture ◽  
Swarm Intelligence Optimization

Abstract In the agriculture development and growth, the efficient machinery and equipment plays an important role. Various research studies are involved in the implementation of the research and patents to aid the smart agriculture and authors and reviewers that machine leaning technologies are providing the best support for this growth. To explore machine learning technology and machine learning algorithms, the most of the applications are studied based on the swarm intelligence optimization. An optimized V3CFOA-RF model is built through V3CFOA. The algorithm is tested in the data set collected concerning rice pests, later analyzed and compared in detail with other existing algorithms. The research result shows that the model and algorithm proposed are not only more accurate in recognition and prediction, but also solve the time lagging problem to a degree. The model and algorithm helped realize a higher accuracy in crop pest prediction, which ensures a more stable and higher output of rice. Thus they can be employed as an important decision-making instrument in the agricultural production sector.

Sign in / Sign up

Export Citation Format

Share Document