Entropic gradient descent algorithms and wide flat minima*

The properties of flat minima in the empirical risk landscape of neural networks have been debated for some time. Increasing evidence suggests they possess better generalization capabilities with respect to sharp ones. In this work we first discuss the relationship between alternative measures of flatness: the local entropy, which is useful for analysis and algorithm development, and the local energy, which is easier to compute and was shown empirically in extensive tests on state-of-the-art networks to be the best predictor of generalization capabilities. We show semi-analytically in simple controlled scenarios that these two measures correlate strongly with each other and with generalization. Then, we extend the analysis to the deep learning scenario by extensive numerical validations. We study two algorithms, entropy-stochastic gradient descent and replicated-stochastic gradient descent, that explicitly include the local entropy in the optimization objective. We devise a training schedule by which we consistently find flatter minima (using both flatness measures), and improve the generalization error for common architectures (e.g. ResNet, EfficientNet).

Packing topological entropy for amenable group actions

Packing topological entropy is a dynamical analogy of the packing dimension, which can be viewed as a counterpart of Bowen topological entropy. In the present paper we give a systematic study of the packing topological entropy for a continuous G-action dynamical system $(X,G)$ , where X is a compact metric space and G is a countable infinite discrete amenable group. We first prove a variational principle for amenable packing topological entropy: for any Borel subset Z of X, the packing topological entropy of Z equals the supremum of upper local entropy over all Borel probability measures for which the subset Z has full measure. Then we obtain an entropy inequality concerning amenable packing entropy. Finally, we show that the packing topological entropy of the set of generic points for any invariant Borel probability measure $\mu $ coincides with the metric entropy if either $\mu $ is ergodic or the system satisfies a kind of specification property.

Local Properties of Entropy for Finite Family of Functions

In this paper we consider the issues of local entropy for a finite family of generators (that generates the semigroup). Our main aim is to show that any continuous function can be approximated by s-chaotic family of generators.

Using Non-Additive Entropy to Enhance Convolutional Neural Features for Texture Recognition

Here we present a study on the use of non-additive entropy to improve the performance of convolutional neural networks for texture description. More precisely, we introduce the use of a local transform that associates each pixel with a measure of local entropy and use such alternative representation as the input to a pretrained convolutional network that performs feature extraction. We compare the performance of our approach in texture recognition over well-established benchmark databases and on a practical task of identifying Brazilian plant species based on the scanned image of the leaf surface. In both cases, our method achieved interesting performance, outperforming several methods from the state-of-the-art in texture analysis. Among the interesting results we have an accuracy of 84.4% in the classification of KTH-TIPS-2b database and 77.7% in FMD. In the identification of plant species we also achieve a promising accuracy of 88.5%. Considering the challenges posed by these tasks and results of other approaches in the literature, our method managed to demonstrate the potential of computing deep learning features over an entropy representation.

Image-Based Method to Quantify Decellularization of Tissue Sections

Tissue decellularization is typically assessed through absorbance-based DNA quantification after tissue digestion. This method has several disadvantages, namely its destructive nature and inadequacy in experimental situations where tissue is scarce. Here, we present an image processing algorithm for quantitative analysis of DNA content in (de)cellularized tissues as a faster, simpler and more comprehensive alternative. Our method uses local entropy measurements of a phase contrast image to create a mask, which is then applied to corresponding nuclei labelled (UV) images to extract average fluorescence intensities as an estimate of DNA content. The method can be used on native or decellularized tissue to quantify DNA content, thus allowing quantitative assessment of decellularization procedures. We confirm that our new method yields results in line with those obtained using the standard DNA quantification method and that it is successful for both lung and heart tissues. We are also able to accurately obtain a timeline of decreasing DNA content with increased incubation time with a decellularizing agent. Finally, the identified masks can also be applied to additional fluorescence images of immunostained proteins such as collagen or elastin, thus allowing further image-based tissue characterization.

Anomalous hydrodynamics with dyonic charge

In this paper, we study anomalous hydrodynamics with a dyonic charge. We show that the local second law of thermodynamics constrains the structure of the anomaly in addition to the structure of the hydrodynamic constitutive equations. In particular, we show that not only the usual [Formula: see text] term but also [Formula: see text] term should be present in the anomaly with a specific coefficient for the local entropy production to be positive definite.