scholarly journals Multiplicative Updates for Optimization Problems with Dynamics

2017 ◽  
Author(s):  
Abbas Kazemipour ◽  
Behtash Babadi ◽  
Min Wu ◽  
Kaspar Podgorski ◽  
Shaul Druckmann
Keyword(s):  
Calcium Imaging ◽  
Optimization Problems ◽  
Nonnegative Matrix ◽  
Objective Functions ◽  
Ml Estimation ◽  
Multiplicative Updates ◽  
Nonnegativity Constraints ◽  
General Convex ◽  
Temporal And Spatial ◽  
Update Rules

AbstractWe consider the problem of optimizing general convex objective functions with nonnegativity constraints. Using the Karush-Kuhn-Tucker (KKT) conditions for the nonnegativity constraints we will derive fast multiplicative update rules for several problems of interest in signal processing, including non-negative deconvolution, point-process smoothing, ML estimation for Poisson Observations, nonnegative least squares and nonnegative matrix factorization (NMF). Our algorithm can also account for temporal and spatial structure and regularization. We will analyze the performance of our algorithm on simultaneously recorded neuronal calcium imaging and electrophysiology data.

scholarly journals On Nonnegative Matrix Factorization Algorithms for Signal-Dependent Noise with Application to Electromyography Data

2014 ◽  
Vol 26 (6) ◽  
pp. 1128-1168 ◽  
Author(s):  
Karthik Devarajan ◽  
Vincent C. K. Cheung
Keyword(s):  
Language Processing ◽  
Matrix Factorization ◽  
Large Scale ◽  
Goodness Of Fit ◽  
Nonnegative Matrix Factorization ◽  
Gaussian Model ◽  
Nonnegative Matrix ◽  
Theoretic Approach ◽  
Multiplicative Updates ◽  
Nonnegativity Constraints

Nonnegative matrix factorization (NMF) by the multiplicative updates algorithm is a powerful machine learning method for decomposing a high-dimensional nonnegative matrix V into two nonnegative matrices, W and H, where [Formula: see text]. It has been successfully applied in the analysis and interpretation of large-scale data arising in neuroscience, computational biology, and natural language processing, among other areas. A distinctive feature of NMF is its nonnegativity constraints that allow only additive linear combinations of the data, thus enabling it to learn parts that have distinct physical representations in reality. In this letter, we describe an information-theoretic approach to NMF for signal-dependent noise based on the generalized inverse gaussian model. Specifically, we propose three novel algorithms in this setting, each based on multiplicative updates, and prove monotonicity of updates using the EM algorithm. In addition, we develop algorithm-specific measures to evaluate their goodness of fit on data. Our methods are demonstrated using experimental data from electromyography studies, as well as simulated data in the extraction of muscle synergies, and compared with existing algorithms for signal-dependent noise.

Scalable Computing Architecture for Time-Dependent Transportation Optimization Problems Based on High-Performance Computing Techniques

2019 ◽  
Vol 2673 (4) ◽  
pp. 205-216 ◽  
Author(s):  
Pengfei (Taylor) Li ◽  
Peirong (Slade) Wang ◽  
Farzana Chowdhury ◽  
Li Zhang
Keyword(s):  
Objective Function ◽  
High Performance Computing ◽  
High Performance ◽  
Optimization Problems ◽  
Dynamic Network ◽  
Alternative Formulation ◽  
High Fidelity ◽  
Objective Functions ◽  
Performance Computing ◽  
Transportation Optimization

Traditional formulations for transportation optimization problems mostly build complicating attributes into constraints while keeping the succinctness of objective functions. A popular solution is the Lagrangian decomposition by relaxing complicating constraints and then solving iteratively. Although this approach is effective for many problems, it generates intractability in other problems. To address this issue, this paper presents an alternative formulation for transportation optimization problems in which the complicating attributes of target problems are partially or entirely built into the objective function instead of into the constraints. Many mathematical complicating constraints in transportation problems can be efficiently modeled in dynamic network loading (DNL) models based on the demand–supply equilibrium, such as the various road or vehicle capacity constraints or “IF–THEN” type constraints. After “pre-building” complicating constraints into the objective functions, the objective function can be approximated well with customized high-fidelity DNL models. Three types of computing benefits can be achieved in the alternative formulation: ( a) the original problem will be kept the same; ( b) computing complexity of the new formulation may be significantly reduced because of the disappearance of hard constraints; ( c) efficiency loss on the objective function side can be mitigated via multiple high-performance computing techniques. Under this new framework, high-fidelity and problem-specific DNL models will be critical to maintain the attributes of original problems. Therefore, the authors’ recent efforts in enhancing the DNL’s fidelity and computing efficiency are also described in the second part of this paper. Finally, a demonstration case study is conducted to validate the new approach.

Uniform and Lipschitz continuity of objective functions in metamaterial band gap optimization problems

2020 ◽  
Author(s):  
Giorgio Gnecco ◽  
Francesca Fantoni ◽  
Andrea Bacigalupo
Keyword(s):  
Band Gap ◽  
Lipschitz Continuity ◽  
Optimization Problems ◽  
Objective Functions

scholarly journals A Method for Integration of Preferences to a Multi-Objective Evolutionary Algorithm Using Ordinal Multi-Criteria Classification

2021 ◽  
Vol 26 (2) ◽  
pp. 27
Author(s):  
Alejandro Castellanos-Alvarez ◽  
Laura Cruz-Reyes ◽  
Eduardo Fernandez ◽  
Nelson Rangel-Valdez ◽  
Claudia Gómez-Santillán ◽  
...  
Keyword(s):  
Genetic Algorithm ◽  
Selective Pressure ◽  
Optimization Problems ◽  
Region Of Interest ◽  
Decision Makers ◽  
Multiple Objective ◽  
Objective Functions ◽  
Multi Objective ◽  
Imperfect Knowledge ◽  
Real World Problems

Most real-world problems require the optimization of multiple objective functions simultaneously, which can conflict with each other. The environment of these problems usually involves imprecise information derived from inaccurate measurements or the variability in decision-makers’ (DMs’) judgments and beliefs, which can lead to unsatisfactory solutions. The imperfect knowledge can be present either in objective functions, restrictions, or decision-maker’s preferences. These optimization problems have been solved using various techniques such as multi-objective evolutionary algorithms (MOEAs). This paper proposes a new MOEA called NSGA-III-P (non-nominated sorting genetic algorithm III with preferences). The main characteristic of NSGA-III-P is an ordinal multi-criteria classification method for preference integration to guide the algorithm to the region of interest given by the decision-maker’s preferences. Besides, the use of interval analysis allows the expression of preferences with imprecision. The experiments contrasted several versions of the proposed method with the original NSGA-III to analyze different selective pressure induced by the DM’s preferences. In these experiments, the algorithms solved three-objectives instances of the DTLZ problem. The obtained results showed a better approximation to the region of interest for a DM when its preferences are considered.

Assessing the performances of recent global search algorithms using analytic objective functions and seismic optimization problems

Geophysics ◽  
2019 ◽  
Vol 84 (5) ◽  
pp. R767-R781 ◽  
Author(s):  
Mattia Aleardi ◽  
Silvio Pierini ◽  
Angelo Sajeva
Keyword(s):  
Particle Swarm Optimization ◽  
Optimization Problems ◽  
Particle Swarm ◽  
Model Space ◽  
Global Search ◽  
Objective Functions ◽  
Swarm Optimization ◽  
Fireworks Algorithm ◽  
Highly Nonlinear ◽  
Whale Optimization

We have compared the performances of six recently developed global optimization algorithms: imperialist competitive algorithm, firefly algorithm (FA), water cycle algorithm (WCA), whale optimization algorithm (WOA), fireworks algorithm (FWA), and quantum particle swarm optimization (QPSO). These methods have been introduced in the past few years and have found very limited or no applications to geophysical exploration problems thus far. We benchmark the algorithms’ results against the particle swarm optimization (PSO), which is a popular and well-established global search method. In particular, we are interested in assessing the exploration and exploitation capabilities of each method as the dimension of the model space increases. First, we test the different algorithms on two multiminima and two convex analytic objective functions. Then, we compare them using the residual statics corrections and 1D elastic full-waveform inversion, which are highly nonlinear geophysical optimization problems. Our results demonstrate that FA, FWA, and WOA are characterized by optimal exploration capabilities because they outperform the other approaches in the case of optimization problems with multiminima objective functions. Differently, QPSO and PSO have good exploitation capabilities because they easily solve ill-conditioned optimizations characterized by a nearly flat valley in the objective function. QPSO, PSO, and WCA offer a good compromise between exploitation and exploration.

scholarly journals GBUO: “The Good, the Bad, and the Ugly” Optimizer

2021 ◽  
Vol 11 (5) ◽  
pp. 2042
Author(s):  
Hadi Givi ◽  
Mohammad Dehghani ◽  
Zeinab Montazeri ◽  
Ruben Morales-Menendez ◽  
Ricardo A. Ramirez-Mendoza ◽  
...  
Keyword(s):  
High Dimension ◽  
Essential Role ◽  
Optimization Problems ◽  
Optimization Algorithms ◽  
Stochastic Search ◽  
Objective Functions ◽  
Science And Engineering ◽  
Fixed Dimension ◽  
Dimension Functions ◽  
Simulation Results

Optimization problems in various fields of science and engineering should be solved using appropriate methods. Stochastic search-based optimization algorithms are a widely used approach for solving optimization problems. In this paper, a new optimization algorithm called “the good, the bad, and the ugly” optimizer (GBUO) is introduced, based on the effect of three members of the population on the population updates. In the proposed GBUO, the algorithm population moves towards the good member and avoids the bad member. In the proposed algorithm, a new member called ugly member is also introduced, which plays an essential role in updating the population. In a challenging move, the ugly member leads the population to situations contrary to society’s movement. GBUO is mathematically modeled, and its equations are presented. GBUO is implemented on a set of twenty-three standard objective functions to evaluate the proposed optimizer’s performance for solving optimization problems. The mentioned standard objective functions can be classified into three groups: unimodal, multimodal with high-dimension, and multimodal with fixed dimension functions. There was a further analysis carried-out for eight well-known optimization algorithms. The simulation results show that the proposed algorithm has a good performance in solving different optimization problems models and is superior to the mentioned optimization algorithms.

Multi Objective Design Optimization of Two Bar Truss Using NSGA II and TOPSIS

2014 ◽  
Vol 984-985 ◽  
pp. 419-424
Author(s):  
P. Sabarinath ◽  
M.R. Thansekhar ◽  
R. Saravanan
Keyword(s):  
Design Optimization ◽  
Optimization Problems ◽  
Optimization Methods ◽  
Optimal Solutions ◽  
Objective Functions ◽  
Nsga Ii ◽  
Multi Objective ◽  
Total Displacement ◽  
Design Variables ◽  
Conflicting Objectives

Arriving optimal solutions is one of the important tasks in engineering design. Many real-world design optimization problems involve multiple conflicting objectives. The design variables are of continuous or discrete in nature. In general, for solving Multi Objective Optimization methods weight method is preferred. In this method, all the objective functions are converted into a single objective function by assigning suitable weights to each objective functions. The main drawback lies in the selection of proper weights. Recently, evolutionary algorithms are used to find the nondominated optimal solutions called as Pareto optimal front in a single run. In recent years, Non-dominated Sorting Genetic Algorithm II (NSGA-II) finds increasing applications in solving multi objective problems comprising of conflicting objectives because of low computational requirements, elitism and parameter-less sharing approach. In this work, we propose a methodology which integrates NSGA-II and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) for solving a two bar truss problem. NSGA-II searches for the Pareto set where two bar truss is evaluated in terms of minimizing the weight of the truss and minimizing the total displacement of the joint under the given load. Subsequently, TOPSIS selects the best compromise solution.

scholarly journals An evolutionary-based optimization algorithm for truss sizing design

2016 ◽  
Vol 38 (4) ◽  
pp. 307-317
Author(s):  
Pham Hoang Anh
Keyword(s):  
Local Search ◽  
Objective Function ◽  
Optimization Algorithm ◽  
Optimization Problems ◽  
Optimal Solution ◽  
The Other ◽  
Truss Structures ◽  
Benchmark Problems ◽  
Discrete Variables ◽  
Objective Functions

In this paper, the optimal sizing of truss structures is solved using a novel evolutionary-based optimization algorithm. The efficiency of the proposed method lies in the combination of global search and local search, in which the global move is applied for a set of random solutions whereas the local move is performed on the other solutions in the search population. Three truss sizing benchmark problems with discrete variables are used to examine the performance of the proposed algorithm. Objective functions of the optimization problems are minimum weights of the whole truss structures and constraints are stress in members and displacement at nodes. Here, the constraints and objective function are treated separately so that both function and constraint evaluations can be saved. The results show that the new algorithm can find optimal solution effectively and it is competitive with some recent metaheuristic algorithms in terms of number of structural analyses required.

Numerical experiments on stochastic block proximal-gradient type method for convex constrained optimization involving coordinatewise separable problems

2019 ◽  
Vol 35 (3) ◽  
pp. 371-378
Author(s):  
PORNTIP PROMSINCHAI ◽  
NARIN PETROT ◽  
◽  
◽  
Keyword(s):  
Constrained Optimization ◽  
Optimization Problems ◽  
Gradient Algorithm ◽  
Objective Functions ◽  
Constrained Optimization Problems ◽  
Type Method ◽  
Coordinate Descent Algorithm ◽  
Proximal Gradient Algorithm ◽  
Separable Problems ◽  
Gradient Type

In this paper, we consider convex constrained optimization problems with composite objective functions over the set of a minimizer of another function. The main aim is to test numerically a new algorithm, namely a stochastic block coordinate proximal-gradient algorithm with penalization, by comparing both the number of iterations and CPU times between this introduced algorithm and the other well-known types of block coordinate descent algorithm for finding solutions of the randomly generated optimization problems with regularization term.

Sign in / Sign up

Export Citation Format

Share Document