scholarly journals Two-Sided Wasserstein Procrustes Analysis

2021 ◽  
Author(s):  
Kun Jin ◽  
Chaoyue Liu ◽  
Cathy Xia
Keyword(s):  
Optimization Problem ◽  
Optimal Transport ◽  
State Of The Art ◽  
Procrustes Analysis ◽  
Convex Optimization Problem ◽  
Feature Spaces ◽  
Joint Problem ◽  
Unsupervised Approach ◽  
Different Dimensions ◽  
Optimal Coupling

Learning correspondence between sets of objects is a key component in many machine learning tasks.Recently, optimal Transport (OT) has been successfully applied to such correspondence problems and it is appealing as a fully unsupervised approach. However, OT requires pairwise instances be directly comparable in a common metric space. This limits its applicability when feature spaces are of different dimensions or not directly comparable. In addition, OT only focuses on pairwise correspondence without sensing global transformations. To address these challenges, we propose a new method to jointly learn the optimal coupling between twosets, and the optimal transformations (e.g. rotation, projection and scaling) of each set based on a two-sided Wassertein Procrustes analysis (TWP). Since the joint problem is a non-convex optimization problem, we present a reformulation that renders the problem component-wise convex. We then propose a novel algorithm to solve the problem harnessing a Gauss–Seidel method. We further present competitive results of TWP on various applicationscompared with state-of-the-art methods.

scholarly journals Learning Discriminative Correlation Subspace for Heterogeneous Domain Adaptation

2017 ◽  
Author(s):  
Yuguang Yan ◽  
Wen Li ◽  
Michael Ng ◽  
Mingkui Tan ◽  
Hanrui Wu ◽  
...  
Keyword(s):  
Optimization Problem ◽  
Domain Adaptation ◽  
Data Sets ◽  
Target Domain ◽  
Real World Data ◽  
Discriminative Ability ◽  
Convex Optimization Problem ◽  
Alternating Direction ◽  
Feature Spaces ◽  
Target Data

Domain adaptation aims to reduce the effort on collecting and annotating target data by leveraging knowledge from a different source domain. The domain adaptation problem will become extremely challenging when the feature spaces of the source and target domains are different, which is also known as the heterogeneous domain adaptation (HDA) problem. In this paper, we propose a novel HDA method to find the optimal discriminative correlation subspace for the source and target data. The discriminative correlation subspace is inherited from the canonical correlation subspace between the source and target data, and is further optimized to maximize the discriminative ability for the target domain classifier. We formulate a joint objective in order to simultaneously learn the discriminative correlation subspace and the target domain classifier. We then apply an alternating direction method of multiplier (ADMM) algorithm to address the resulting non-convex optimization problem. Comprehensive experiments on two real-world data sets demonstrate the effectiveness of the proposed method compared to the state-of-the-art methods.

Quantum entropic regularization of matrix-valued optimal transport

2017 ◽  
Vol 30 (6) ◽  
pp. 1079-1102 ◽  
Author(s):  
GABRIEL PEYRÉ ◽  
LÉNAÏC CHIZAT ◽  
FRANÇOIS-XAVIER VIALARD ◽  
JUSTIN SOLOMON
Keyword(s):  
Optimization Problem ◽  
Optimal Transport ◽  
Diffusion Tensor ◽  
Texture Synthesis ◽  
Positive Semidefinite ◽  
Tensor Fields ◽  
Convex Optimization Problem ◽  
Von Neumann ◽  
Entropic Regularization ◽  
Quantum Formulation

This article introduces a new notion of optimal transport (OT) between tensor fields, which are measures whose values are positive semidefinite (PSD) matrices. This “quantum” formulation of optimal transport (Q-OT) corresponds to a relaxed version of the classical Kantorovich transport problem, where the fidelity between the input PSD-valued measures is captured using the geometry of the Von-Neumann quantum entropy. We propose a quantum-entropic regularization of the resulting convex optimization problem, which can be solved efficiently using an iterative scaling algorithm. This method is a generalization of the celebrated Sinkhorn algorithm to the quantum setting of PSD matrices. We extend this formulation and the quantum Sinkhorn algorithm to compute barycentres within a collection of input tensor fields. We illustrate the usefulness of the proposed approach on applications to procedural noise generation, anisotropic meshing, diffusion tensor imaging and spectral texture synthesis.

scholarly journals Inverse reinforcement learning in contextual MDPs

2021 ◽  
Author(s):  
Stav Belogolovsky ◽  
Philip Korsunsky ◽  
Shie Mannor ◽  
Chen Tessler ◽  
Tom Zahavy
Keyword(s):  
Reinforcement Learning ◽  
Optimization Problem ◽  
Decision Processes ◽  
Inverse Reinforcement Learning ◽  
Convex Optimization Problem ◽  
Reward Function ◽  
Dynamic Treatment Regime ◽  
Markov Decision ◽  
Dynamic Treatment ◽  
Recorded Data

AbstractWe consider the task of Inverse Reinforcement Learning in Contextual Markov Decision Processes (MDPs). In this setting, contexts, which define the reward and transition kernel, are sampled from a distribution. In addition, although the reward is a function of the context, it is not provided to the agent. Instead, the agent observes demonstrations from an optimal policy. The goal is to learn the reward mapping, such that the agent will act optimally even when encountering previously unseen contexts, also known as zero-shot transfer. We formulate this problem as a non-differential convex optimization problem and propose a novel algorithm to compute its subgradients. Based on this scheme, we analyze several methods both theoretically, where we compare the sample complexity and scalability, and empirically. Most importantly, we show both theoretically and empirically that our algorithms perform zero-shot transfer (generalize to new and unseen contexts). Specifically, we present empirical experiments in a dynamic treatment regime, where the goal is to learn a reward function which explains the behavior of expert physicians based on recorded data of them treating patients diagnosed with sepsis.

scholarly journals Robust CNN Compression Framework for Security-Sensitive Embedded Systems

2021 ◽  
Vol 11 (3) ◽  
pp. 1093
Author(s):  
Jeonghyun Lee ◽  
Sangkyun Lee
Keyword(s):  
Embedded Systems ◽  
Optimization Problem ◽  
State Of The Art ◽  
Classification Problems ◽  
Proximal Gradient Method ◽  
Knowledge Distillation ◽  
New Type ◽  
Adversarial Examples ◽  
Adversarial Training ◽  
Memory Efficient

Convolutional neural networks (CNNs) have achieved tremendous success in solving complex classification problems. Motivated by this success, there have been proposed various compression methods for downsizing the CNNs to deploy them on resource-constrained embedded systems. However, a new type of vulnerability of compressed CNNs known as the adversarial examples has been discovered recently, which is critical for security-sensitive systems because the adversarial examples can cause malfunction of CNNs and can be crafted easily in many cases. In this paper, we proposed a compression framework to produce compressed CNNs robust against such adversarial examples. To achieve the goal, our framework uses both pruning and knowledge distillation with adversarial training. We formulate our framework as an optimization problem and provide a solution algorithm based on the proximal gradient method, which is more memory-efficient than the popular ADMM-based compression approaches. In experiments, we show that our framework can improve the trade-off between adversarial robustness and compression rate compared to the existing state-of-the-art adversarial pruning approach.

The smallest extraction problem

2021 ◽  
Vol 14 (11) ◽  
pp. 2445-2458
Author(s):  
Valerio Cetorelli ◽  
Paolo Atzeni ◽  
Valter Crescenzi ◽  
Franco Milicchio
Keyword(s):  
Unsupervised Learning ◽  
Optimization Problem ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Data Extraction ◽  
Source Code ◽  
Web Data ◽  
Web Data Extraction ◽  
New Family ◽  
Context Free

We introduce landmark grammars , a new family of context-free grammars aimed at describing the HTML source code of pages published by large and templated websites and therefore at effectively tackling Web data extraction problems. Indeed, they address the inherent ambiguity of HTML, one of the main challenges of Web data extraction, which, despite over twenty years of research, has been largely neglected by the approaches presented in literature. We then formalize the Smallest Extraction Problem (SEP), an optimization problem for finding the grammar of a family that best describes a set of pages and contextually extract their data. Finally, we present an unsupervised learning algorithm to induce a landmark grammar from a set of pages sharing a common HTML template, and we present an automatic Web data extraction system. The experiments on consolidated benchmarks show that the approach can substantially contribute to improve the state-of-the-art.

Adaptive Grouping Brain Storm Optimization for Multimodal Optimization Problems

2021 ◽  
Vol 12 (4) ◽  
pp. 81-100
Author(s):  
Yao Peng ◽  
Zepeng Shen ◽  
Shiqi Wang
Keyword(s):  
Optimization Problem ◽  
Optimization Problems ◽  
Optimal Algorithm ◽  
Peak Detection ◽  
Multimodal Optimization ◽  
Optimal Solutions ◽  
Grouping Strategy ◽  
Different Dimensions ◽  
Global Optimal ◽  
Localization Ability

Multimodal optimization problem exists in multiple global and many local optimal solutions. The difficulty of solving these problems is finding as many local optimal peaks as possible on the premise of ensuring global optimal precision. This article presents adaptive grouping brainstorm optimization (AGBSO) for solving these problems. In this article, adaptive grouping strategy is proposed for achieving adaptive grouping without providing any prior knowledge by users. For enhancing the diversity and accuracy of the optimal algorithm, elite reservation strategy is proposed to put central particles into an elite pool, and peak detection strategy is proposed to delete particles far from optimal peaks in the elite pool. Finally, this article uses testing functions with different dimensions to compare the convergence, accuracy, and diversity of AGBSO with BSO. Experiments verify that AGBSO has great localization ability for local optimal solutions while ensuring the accuracy of the global optimal solutions.

Solving Mono- and Multi-Objective Problems Using Hybrid Evolutionary Algorithms and Nelder-Mead Method

2021 ◽  
Vol 12 (4) ◽  
pp. 98-116
Author(s):  
Noureddine Boukhari ◽  
Fatima Debbat ◽  
Nicolas Monmarché ◽  
Mohamed Slimane
Keyword(s):  
Convergence Rate ◽  
Optimization Problem ◽  
Optimization Problems ◽  
State Of The Art ◽  
Hybrid Approach ◽  
Optimization Methods ◽  
Global Optimum ◽  
Stochastic Methods ◽  
Local Optima ◽  
Portfolio Optimization Problem

Evolution strategies (ES) are a family of strong stochastic methods for global optimization and have proved their capability in avoiding local optima more than other optimization methods. Many researchers have investigated different versions of the original evolution strategy with good results in a variety of optimization problems. However, the convergence rate of the algorithm to the global optimum stays asymptotic. In order to accelerate the convergence rate, a hybrid approach is proposed using the nonlinear simplex method (Nelder-Mead) and an adaptive scheme to control the local search application, and the authors demonstrate that such combination yields significantly better convergence. The new proposed method has been tested on 15 complex benchmark functions and applied to the bi-objective portfolio optimization problem and compared with other state-of-the-art techniques. Experimental results show that the performance is improved by this hybridization in terms of solution eminence and strong convergence.

Extended state observer-based robust non-linear integral dynamic surface control for triaxial MEMS gyroscope

Robotica ◽  
2018 ◽  
Vol 37 (3) ◽  
pp. 481-501 ◽  
Author(s):  
Mehran Hosseini-Pishrobat ◽  
Jafar Keighobadi
Keyword(s):  
Optimization Problem ◽  
Dynamic Surface Control ◽  
Extended State ◽  
Gain Function ◽  
Dynamic Surface ◽  
Convex Optimization Problem ◽  
Mems Gyroscope ◽  
Surface Control ◽  
Non Linear ◽  
Linear Integral

SUMMARYThis paper reports an extended state observer (ESO)-based robust dynamic surface control (DSC) method for triaxial MEMS gyroscope applications. An ESO with non-linear gain function is designed to estimate both velocity and disturbance vectors of the gyroscope dynamics via measured position signals. Using the sector-bounded property of the non-linear gain function, the design of an $\mathcal{L}_2$-robust ESO is phrased as a convex optimization problem in terms of linear matrix inequalities (LMIs). Next, by using the estimated velocity and disturbance, a certainty equivalence tracking controller is designed based on DSC. To achieve an improved robustness and to remove static steady-state tracking errors, new non-linear integral error surfaces are incorporated into the DSC. Based on the energy-to-peak ($\mathcal{L}_2$-$\mathcal{L}_\infty$) performance criterion, a finite number of LMIs are derived to obtain the DSC gains. In order to prevent amplification of the measurement noise in the DSC error dynamics, a multi-objective convex optimization problem, which guarantees a prescribed $\mathcal{L}_2$-$\mathcal{L}_\infty$ performance bound, is considered. Finally, the efficacy of the proposed control method is illustrated by detailed software simulations.

scholarly journals Shearlet-based regularized reconstruction in region-of-interest computed tomography

2018 ◽  
Vol 13 (4) ◽  
pp. 34
Author(s):  
T.A. Bubba ◽  
D. Labate ◽  
G. Zanghirati ◽  
S. Bonettini
Keyword(s):  
Optimization Problem ◽  
Ad Hoc ◽  
Iterative Scheme ◽  
Tomographic Reconstruction ◽  
Region Of Interest ◽  
Projection Data ◽  
Reconstruction Problem ◽  
Convex Optimization Problem ◽  
Scanning Time ◽  
Novel Approach

Region of interest (ROI) tomography has gained increasing attention in recent years due to its potential to reducing radiation exposure and shortening the scanning time. However, tomographic reconstruction from ROI-focused illumination involves truncated projection data and typically results in higher numerical instability even when the reconstruction problem has unique solution. To address this problem, bothad hocanalytic formulas and iterative numerical schemes have been proposed in the literature. In this paper, we introduce a novel approach for ROI tomographic reconstruction, formulated as a convex optimization problem with a regularized term based on shearlets. Our numerical implementation consists of an iterative scheme based on the scaled gradient projection method and it is tested in the context of fan-beam CT. Our results show that our approach is essentially insensitive to the location of the ROI and remains very stable also when the ROI size is rather small.

scholarly journals Low Complexity Sparse Beamspace DOA Estimation Via Single Measurement Vectors for Uniform Circular Array

2021 ◽  
Author(s):  
Di Zhao ◽  
Weijie Tan ◽  
Zhongliang Deng ◽  
Gang Li
Keyword(s):  
Optimization Problem ◽  
Signal To Noise Ratio ◽  
Estimation Method ◽  
Doa Estimation ◽  
Low Complexity ◽  
Estimation Methods ◽  
Circular Array ◽  
Single Measurement ◽  
Signal Model ◽  
Convex Optimization Problem

Abstract In this paper, we present a low complexity beamspace direction-of-arrival (DOA) estimation method for uniform circular array (UCA), which is based on the single measurement vectors (SMVs) via vectorization of sparse covariance matrix. In the proposed method, we rstly transform the signal model of UCA to that of virtual uniform linear array (ULA) in beamspace domain using the beamspace transformation (BT). Subsequently, by applying the vectorization operator on the virtual ULA-like array signal model, a new dimension-reduction array signal model consists of SMVs based on Khatri-Rao (KR) product is derived. And then, the DOA estimation is converted to the convex optimization problem. Finally, simulations are carried out to verify the eectiveness of the proposed method, the results show that without knowledge of the signal number, the proposed method not only has higher DOA resolution than subspace-based methods in low signal-to-noise ratio (SNR), but also has much lower computational complexity comparing other sparse-like DOA estimation methods.

Sign in / Sign up

Export Citation Format

Share Document