scholarly journals Using Benson’s Algorithm for Regularization Parameter Tracking

2019 ◽  
Vol 33 ◽  
pp. 3689-3696
Author(s):  
Joachim Giesen ◽  
Sӧren Laue ◽  
Andreas Lӧhne ◽  
Christopher Schneider
Keyword(s):  
Approximate Solution ◽  
Minimization Problem ◽  
Regularization Parameter ◽  
Approximate Solutions ◽  
Loss Minimization ◽  
Real World Data ◽  
Approximation Accuracy ◽  
Parameter Domain ◽  
Parameter Tracking ◽  
Regularization Parameters

Regularized loss minimization, where a statistical model is obtained from minimizing the sum of a loss function and weighted regularization terms, is still in widespread use in machine learning. The statistical performance of the resulting models depends on the choice of weights (regularization parameters) that are typically tuned by cross-validation. For finding the best regularization parameters, the regularized minimization problem needs to be solved for the whole parameter domain. A practically more feasible approach is covering the parameter domain with approximate solutions of the loss minimization problem for some prescribed approximation accuracy. The problem of computing such a covering is known as the approximate solution gamut problem. Existing algorithms for the solution gamut problem suffer from several problems. For instance, they require a grid on the parameter domain whose spacing is difficult to determine in practice, and they are not generic in the sense that they rely on problem specific plug-in functions. Here, we show that a well-known algorithm from vector optimization, namely the Benson algorithm, can be used directly for computing approximate solution gamuts while avoiding the problems of existing algorithms. Experiments for the Elastic Net on real world data sets demonstrate the effectiveness of Benson’s algorithm for regularization parameter tracking.

scholarly journals On fractional Tikhonov regularization

2015 ◽  
Vol 23 (6) ◽  
Author(s):  
Daniel Gerth ◽  
Esther Klann ◽  
Ronny Ramlau ◽  
Lothar Reichel
Keyword(s):  
Exact Solution ◽  
Approximate Solution ◽  
Tikhonov Regularization ◽  
Regularization Parameter ◽  
Standard Form ◽  
Approximate Solutions ◽  
Discrepancy Principle ◽  
Convergence Properties

AbstractIt is well known that Tikhonov regularization in standard form may determine approximate solutions that are too smooth, i.e., the approximate solution may lack many details that the desired exact solution might possess. Two different approaches, both referred to as fractional Tikhonov methods have been introduced to remedy this shortcoming. This paper investigates the convergence properties of these methods by reviewing results published previously by various authors. We show that both methods are order optimal when the regularization parameter is chosen according to the discrepancy principle. The theory developed suggests situations in which the fractional methods yield approximate solutions of higher quality than Tikhonov regularization in standard form. Computed examples that illustrate the behavior of the methods are presented.

A new algorithm for finding CRM-model coefficients

2019 ◽  
Vol 5 (3) ◽  
pp. 164-185 ◽  
Author(s):  
Alexander D. Bekman ◽  
Sergey V. Stepanov ◽  
Alexander A. Ruchkin ◽  
Dmitry V. Zelenin
Keyword(s):  
Approximate Solution ◽  
Optimization Problem ◽  
Optimal Solution ◽  
Complex Form ◽  
Approximate Solutions ◽  
The Other ◽  
Programming Algorithm ◽  
Stochastic Algorithms ◽  
Large Set ◽  
Flow Rates

The quantitative evaluation of producer and injector well interference based on well operation data (profiles of flow rates/injectivities and bottomhole/reservoir pressures) with the help of CRM (Capacitance-Resistive Models) is an optimization problem with large set of variables and constraints. The analytical solution cannot be found because of the complex form of the objective function for this problem. Attempts to find the solution with stochastic algorithms take unacceptable time and the result may be far from the optimal solution. Besides, the use of universal (commercial) optimizers hides the details of step by step solution from the user, for example&nbsp;— the ambiguity of the solution as the result of data inaccuracy.<br> The present article concerns two variants of CRM problem. The authors present a new algorithm of solving the problems with the help of “General Quadratic Programming Algorithm”. The main advantage of the new algorithm is the greater performance in comparison with the other known algorithms. Its other advantage is the possibility of an ambiguity analysis. This article studies the conditions which guarantee that the first variant of problem has a unique solution, which can be found with the presented algorithm. Another algorithm for finding the approximate solution for the second variant of the problem is also considered. The method of visualization of approximate solutions set is presented. The results of experiments comparing the new algorithm with some previously known are given.

scholarly journals Approximate solution of system of equations arising in interior-point methods for bound-constrained optimization

2021 ◽  
Author(s):  
David Ek ◽  
Anders Forsgren
Keyword(s):  
Approximate Solution ◽  
Interior Point ◽  
Interior Point Methods ◽  
Linear Equations ◽  
Approximate Solutions ◽  
System Of Nonlinear Equations ◽  
Intermediate Step ◽  
System Of Linear Equations ◽  
Constrained Nonlinear Optimization ◽  
Asymptotic Error

AbstractThe focus in this paper is interior-point methods for bound-constrained nonlinear optimization, where the system of nonlinear equations that arise are solved with Newton’s method. There is a trade-off between solving Newton systems directly, which give high quality solutions, and solving many approximate Newton systems which are computationally less expensive but give lower quality solutions. We propose partial and full approximate solutions to the Newton systems. The specific approximate solution depends on estimates of the active and inactive constraints at the solution. These sets are at each iteration estimated by basic heuristics. The partial approximate solutions are computationally inexpensive, whereas a system of linear equations needs to be solved for the full approximate solution. The size of the system is determined by the estimate of the inactive constraints at the solution. In addition, we motivate and suggest two Newton-like approaches which are based on an intermediate step that consists of the partial approximate solutions. The theoretical setting is introduced and asymptotic error bounds are given. We also give numerical results to investigate the performance of the approximate solutions within and beyond the theoretical framework.

scholarly journals Algebraic multigrid methods

Acta Numerica ◽  
2017 ◽  
Vol 26 ◽  
pp. 591-721 ◽  
Author(s):  
Jinchao Xu ◽  
Ludmil Zikatanov
Keyword(s):  
Minimization Problem ◽  
Large Scale ◽  
Multigrid Method ◽  
Approximate Solutions ◽  
Multigrid Methods ◽  
Algebraic Multigrid ◽  
Unified Framework ◽  
Smoothed Aggregation ◽  
Algebraic Multigrid Methods ◽  
Coarse Space

This paper provides an overview of AMG methods for solving large-scale systems of equations, such as those from discretizations of partial differential equations. AMG is often understood as the acronym of ‘algebraic multigrid’, but it can also be understood as ‘abstract multigrid’. Indeed, we demonstrate in this paper how and why an algebraic multigrid method can be better understood at a more abstract level. In the literature, there are many different algebraic multigrid methods that have been developed from different perspectives. In this paper we try to develop a unified framework and theory that can be used to derive and analyse different algebraic multigrid methods in a coherent manner. Given a smoother$R$for a matrix$A$, such as Gauss–Seidel or Jacobi, we prove that the optimal coarse space of dimension$n_{c}$is the span of the eigenvectors corresponding to the first$n_{c}$eigenvectors$\bar{R}A$(with$\bar{R}=R+R^{T}-R^{T}AR$). We also prove that this optimal coarse space can be obtained via a constrained trace-minimization problem for a matrix associated with$\bar{R}A$, and demonstrate that coarse spaces of most existing AMG methods can be viewed as approximate solutions of this trace-minimization problem. Furthermore, we provide a general approach to the construction of quasi-optimal coarse spaces, and we prove that under appropriate assumptions the resulting two-level AMG method for the underlying linear system converges uniformly with respect to the size of the problem, the coefficient variation and the anisotropy. Our theory applies to most existing multigrid methods, including the standard geometric multigrid method, classical AMG, energy-minimization AMG, unsmoothed and smoothed aggregation AMG and spectral AMGe.

scholarly journals Second Approximate Solution of Duffing Equation with Strong Nonlinearity by Homotopy Perturbation Method

1970 ◽  
Vol 30 ◽  
pp. 59-75
Author(s):  
M Alhaz Uddin ◽  
M Abdus Sattar
Keyword(s):  
Approximate Solution ◽  
Nonlinear Systems ◽  
Perturbation Method ◽  
Differential System ◽  
Homotopy Perturbation Method ◽  
Approximate Solutions ◽  
Second Order ◽  
Strong Nonlinearity ◽  
Analytical Technique ◽  
Homotopy Perturbation

 In this paper, the second order approximate solution of a general second order nonlinear ordinary differential system, modeling damped oscillatory process is considered. The new analytical technique based on the work of He’s homotopy perturbation method is developed to find the periodic solution of a second order ordinary nonlinear differential system with damping effects. Usually the second or higher order approximate solutions are able to give better results than the first order approximate solutions. The results show that the analytical approximate solutions obtained by homotopy perturbation method are uniformly valid on the whole solutions domain and they are suitable not only for strongly nonlinear systems, but also for weakly nonlinear systems. Another advantage of this new analytical technique is that it also works for strongly damped, weakly damped and undamped systems. Figures are provided to show the comparison between the analytical and the numerical solutions. Keywords: Homotopy perturbation method; damped oscillation; nonlinear equation; strong nonlinearity. GANIT J. Bangladesh Math. Soc. (ISSN 1606-3694) 30 (2010) 59-75  DOI: http://dx.doi.org/10.3329/ganit.v30i0.8504

scholarly journals A note on time-fractional Navier–Stokes equation and multi-Laplace transform decomposition method

2020 ◽  
Vol 2020 (1) ◽  
Author(s):  
Hassan Eltayeb ◽  
Imed Bachar ◽  
Yahya T. Abdalla
Keyword(s):  
Numerical Simulation ◽  
Approximate Solution ◽  
Decomposition Method ◽  
Adomian Decomposition Method ◽  
Approximate Solutions ◽  
Navier Stokes ◽  
Two Dimensional ◽  
Navier Stokes Equation ◽  
Adomian Decomposition ◽  
Stokes Problems

Abstract In this study, the double Laplace Adomian decomposition method and the triple Laplace Adomian decomposition method are employed to solve one- and two-dimensional time-fractional Navier–Stokes problems, respectively. In order to examine the applicability of these methods some examples are provided. The presented results confirm that the proposed methods are very effective in the search of exact and approximate solutions for the problems. Numerical simulation is used to sketch the exact and approximate solution.

Stress Distribution in a Uniformly Rotating Equilateral Triangular Shaft

1955 ◽  
Vol 22 (2) ◽  
pp. 255-259
Author(s):  
H. T. Johnson
Keyword(s):  
Approximate Solution ◽  
Stress Distribution ◽  
Boundary Conditions ◽  
Plane Strain ◽  
Cross Section ◽  
Plane Stress ◽  
Biharmonic Equation ◽  
Approximate Solutions ◽  
Distribution Of Stresses ◽  
General Method

Abstract An approximate solution for the distribution of stresses in a rotating prismatic shaft, of triangular cross section, is presented in this paper. A general method is employed which may be applied in obtaining approximate solutions for the stress distribution for rotating prismatic shapes, for the cases of either generalized plane stress or plane strain. Polynomials are used which exactly satisfy the biharmonic equation and the symmetry conditions, and which approximately satisfy the boundary conditions.

scholarly journals On conjugate difference schemes: the midpoint scheme and the trapezoidal scheme

2021 ◽  
Vol 29 (1) ◽  
pp. 63-72
Author(s):  
Yu Ying ◽  
Mikhail D. Malykh
Keyword(s):  
Approximate Solution ◽  
Differential Equations ◽  
Ordinary Differential Equations ◽  
Projective Geometry ◽  
Autonomous Systems ◽  
Approximate Solutions ◽  
Difference Schemes ◽  
Duality Principle ◽  
Integrals Of Motion ◽  
Quadratic Integral

The preservation of quadratic integrals on approximate solutions of autonomous systems of ordinary differential equations x=f(x), found by the trapezoidal scheme, is investigated. For this purpose, a relation has been established between the trapezoidal scheme and the midpoint scheme, which preserves all quadratic integrals of motion by virtue of Coopers theorem. This relation allows considering the trapezoidal scheme as dual to the midpoint scheme and to find a dual analogue for Coopers theorem by analogy with the duality principle in projective geometry. It is proved that on the approximate solution found by the trapezoidal scheme, not the quadratic integral itself is preserved, but a more complicated expression, which turns into an integral in the limit as t0.Thus the concept of conjugate difference schemes is investigated in pure algebraic way. The results are illustrated by examples of linear and elliptic oscillators. In both cases, expressions preserved by the trapezoidal scheme are presented explicitly.

scholarly journals Range-relaxed criteria for choosing the Lagrange multipliers in nonstationary iterated Tikhonov method

2018 ◽  
Vol 40 (1) ◽  
pp. 606-627 ◽  
Author(s):  
R Boiger ◽  
A Leitão ◽  
B F Svaiter
Keyword(s):  
Lagrange Multipliers ◽  
Approximate Solutions ◽  
Identification Problem ◽  
Linear Operators ◽  
Type Method ◽  
Geometrical Properties ◽  
Regularization Parameters ◽  
New Strategy ◽  
Parameter Identification Problem ◽  
Ill Posed

Abstract In this article we propose a novel nonstationary iterated Tikhonov (NIT)-type method for obtaining stable approximate solutions to ill-posed operator equations modeled by linear operators acting between Hilbert spaces. Geometrical properties of the problem are used to derive a new strategy for choosing the sequence of regularization parameters (Lagrange multipliers) for the NIT iteration. Convergence analysis for this new method is provided. Numerical experiments are presented for two distinct applications: (I) a two-dimensional elliptic parameter identification problem (inverse potential problem); and (II) an image-deblurring problem. The results obtained validate the efficiency of our method compared with standard implementations of the NIT method (where a geometrical choice is typically used for the sequence of Lagrange multipliers).

Sign in / Sign up

Export Citation Format

Share Document