scholarly journals Understanding Entropic Regularization in GANs

2021 ◽  
Author(s):  
Daria Reshetova ◽  
Yikun Bai ◽  
Xiugang Wu ◽  
Ayfer Ozgur
Keyword(s):  
Entropic Regularization

scholarly journals Entropic Regularization of Markov Decision Processes

Entropy ◽  
2019 ◽  
Vol 21 (7) ◽  
pp. 674
Author(s):  
Boris Belousov ◽  
Jan Peters
Keyword(s):  
Data Gathering ◽  
Direct Interaction ◽  
Learning Problems ◽  
Function Estimation ◽  
Policy Improvement ◽  
Reward Function ◽  
Learning Agent ◽  
Markov Decision ◽  
Entropic Regularization ◽  
Policy Optimization

An optimal feedback controller for a given Markov decision process (MDP) can in principle be synthesized by value or policy iteration. However, if the system dynamics and the reward function are unknown, a learning agent must discover an optimal controller via direct interaction with the environment. Such interactive data gathering commonly leads to divergence towards dangerous or uninformative regions of the state space unless additional regularization measures are taken. Prior works proposed bounding the information loss measured by the Kullback–Leibler (KL) divergence at every policy improvement step to eliminate instability in the learning dynamics. In this paper, we consider a broader family of f-divergences, and more concretely α -divergences, which inherit the beneficial property of providing the policy improvement step in closed form at the same time yielding a corresponding dual objective for policy evaluation. Such entropic proximal policy optimization view gives a unified perspective on compatible actor-critic architectures. In particular, common least-squares value function estimation coupled with advantage-weighted maximum likelihood policy improvement is shown to correspond to the Pearson χ 2 -divergence penalty. Other actor-critic pairs arise for various choices of the penalty-generating function f. On a concrete instantiation of our framework with the α -divergence, we carry out asymptotic analysis of the solutions for different values of α and demonstrate the effects of the divergence function choice on common standard reinforcement learning problems.

scholarly journals A tumor growth model of Hele-Shaw type as a gradient flow

2020 ◽  
Vol 26 ◽  
pp. 103
Author(s):  
Simone Di Marino ◽  
Lénaïc Chizat
Keyword(s):  
Tumor Growth ◽  
Optimal Transport ◽  
Gradient Flow ◽  
Convex Domains ◽  
Wasserstein Metric ◽  
Uniqueness Of Solutions ◽  
Tumor Growth Model ◽  
Entropic Regularization ◽  
Minimizing Movements ◽  
Convergence Of The Scheme

In this paper, we characterize a degenerate PDE as the gradient flow in the space of nonnegative measures endowed with an optimal transport-growth metric. The PDE of concern, of Hele-Shaw type, was introduced by Perthame et. al. as a mechanical model for tumor growth and the metric was introduced recently in several articles as the analogue of the Wasserstein metric for nonnegative measures. We show existence of solutions using minimizing movements and show uniqueness of solutions on convex domains by proving the Evolutional Variational Inequality. Our analysis does not require any regularity assumption on the initial condition. We also derive a numerical scheme based on the discretization of the gradient flow and the idea of entropic regularization. We assess the convergence of the scheme on explicit solutions. In doing this analysis, we prove several new properties of the optimal transport-growth metric, which generally have a known counterpart for the Wasserstein metric.

scholarly journals Thermalization of rate-independent processes by entropic regularization

2013 ◽  
Vol 6 (1) ◽  
pp. 215-233
Author(s):  
T. J. Sullivan ◽  
◽  
M. Koslowski ◽  
F. Theil ◽  
Michael Ortiz ◽  
...  
Keyword(s):  
Entropic Regularization

Total variation regularization for depth-to-basement estimate: Part 2 — Physicogeologic meaning and comparisons with previous inversion methods

Geophysics ◽  
2011 ◽  
Vol 76 (1) ◽  
pp. I13-I20 ◽  
Author(s):  
Williams A. Lima ◽  
Cristiano M. Martins ◽  
João B. Silva ◽  
Valeria C. Barbosa
Keyword(s):  
Total Variation ◽  
Regularization Parameter ◽  
Synthetic Data ◽  
Gravity Inversion ◽  
Total Variation Regularization ◽  
Discontinuous Solutions ◽  
Mathematical Basis ◽  
Inversion Methods ◽  
Tv Regularization ◽  
Entropic Regularization

We applied the mathematical basis of the total variation (TV) regularization to analyze the physicogeologic meaning of the TV method and compared it with previous gravity inversion methods (weighted smoothness and entropic Regularization) to estimate discontinuous basements. In the second part, we analyze the physicogeologic meaning of the TV method and compare it with previous gravity inversion methods (weighted smoothness and entropic regularization) to estimate discontinuous basements. Presenting a mathematical review of these methods, we show that minimizing the TV stabilizing function favors discontinuous solutions because a smooth solution, to honor the data, must oscillate, and the presence of these oscillations increases the value of the TV stabilizing function. These three methods are applied to synthetic data produced by a simulated 2D graben bordered by step faults. TV regularization and weighted smoothness are also applied to the real anomaly of Steptoe Valley, Nevada, U.S.A. In all applications, the three methods perform similarly. TV regularization, however, has the advantage, compared with weighted smoothness, of requiring no a priori information about the maximum depth of the basin. As compared with entropic regularization, TV regularization is much simpler to use because it requires, in general, the tuning of just one regularization parameter.

Gravity inversion of 2D basement relief using entropic regularization

Geophysics ◽  
2010 ◽  
Vol 75 (3) ◽  
pp. I29-I35 ◽  
Author(s):  
João B. Silva ◽  
Alexandre S. Oliveira ◽  
Valéria C. Barbosa
Keyword(s):  
A Priori ◽  
Synthetic Data ◽  
Entropy Measure ◽  
Gravity Inversion ◽  
Solution Vector ◽  
Good Definition ◽  
Northern Border ◽  
Global Smoothness ◽  
Entropic Regularization ◽  
Gravity Interpretation

We have developed a gravity interpretation method for estimating the discontinuous basement relief of a sedimentary basin. The density contrast between the basement and the sediments is assumed to be known, and it could be either constant or vary monotonically with depth. The interpretation model consists of a set of vertical, juxtaposed prisms, whose thicknesses are the parameters to be estimated. We used the entropic regularization that combines the minimization of the first-order entropy measure with the maximization of the zeroth-order entropy measure of the solution vector. We validated the method by applying it to synthetic data produced by a simulated basin bordered by high-angle step faults; we obtained a good definition of the relief, particularly of the discontinuities. We also applied the method to a profile across the Büyük Menderes Valley in West Turkey and obtained a solution exhibiting a gravity fault with large slip on the northern border of the valley. When applied to the interpretation of a discontinuous basement relief, the method has a better performance than the global smoothness method. It is comparable to the weighted smoothness method, but it does not require the a priori knowledge about the maximum basin depth.

Sign in / Sign up

Export Citation Format

Share Document