Hausdorff dimension, heavy tails, and generalization in neural networks*

Despite its success in a wide range of applications, characterizing the generalization properties of stochastic gradient descent (SGD) in non-convex deep learning problems is still an important challenge. While modeling the trajectories of SGD via stochastic differential equations (SDE) under heavy-tailed gradient noise has recently shed light over several peculiar characteristics of SGD, a rigorous treatment of the generalization properties of such SDEs in a learning theoretical framework is still missing. Aiming to bridge this gap, in this paper, we prove generalization bounds for SGD under the assumption that its trajectories can be well-approximated by a Feller process, which defines a rich class of Markov processes that include several recent SDE representations (both Brownian or heavy-tailed) as its special case. We show that the generalization error can be controlled by the Hausdorff dimension of the trajectories, which is intimately linked to the tail behavior of the driving process. Our results imply that heavier-tailed processes should achieve better generalization; hence, the tail-index of the process can be used as a notion of ‘capacity metric’. We support our theory with experiments on deep neural networks illustrating that the proposed capacity metric accurately estimates the generalization error, and it does not necessarily grow with the number of parameters unlike the existing capacity metrics in the literature.

A contraction approach to dynamic optimization problems

An infinite-horizon, multidimensional optimization problem with arbitrary yet finite periodicity in discrete time is considered. The problem can be posed as a set of coupled equations. It is shown that the problem is a special case of a more general class of contraction problems that have unique solutions. Solutions are obtained by considering a vector-valued value function and by using an iterative process. Special cases of the general class of contraction problems include the classical Bellman problem and its stochastic formulations. Thus, our approach can be viewed as an extension of the Bellman problem to the special case of nonautonomy that periodicity represents, and our approach thereby facilitates consistent and rigorous treatment of, for example, seasonality in discrete, dynamic optimization, and furthermore, certain types of dynamic games. The contraction approach is illustrated in simple examples. In the main example, which is an infinite-horizon resource management problem with a periodic price, it is found that the optimal exploitation level differs between high and low price time intervals and that the solution time paths approach a limit cycle.

The Bounds of Possibility

This book didn’t have to consist of exactly the sentences that it in fact contains: any one of its sentences could have been very different. But it could not have consisted of an entirely different collection of sentences, such as to make it a gothic novel or a treatise on wine-tasting. Other familiar objects are similarly capable of being moderately different, but not radically different, in various respects. But there are puzzling arguments which threaten these apparently obvious judgments, exploiting the fact that an appropriate sequence of small differences can add up to a radical difference. This book presents the first full-length treatment of these puzzles, using them as an entry point to a broad range of metaphysical questions about possibility, necessity, and identity. It introduces tools of higher-order modal logic which enable a rigorous treatment of the puzzles, and develops a strategy for resolving them based on a plenitudinous ontology of material objects, which induces fine-grained variability in the reference of words like ‘book’.

Rigorous treatment of pairwise and many-body electrostatic interactions among dielectric spheres at the Debye–Hückel level

Electrostatic interactions among colloidal particles are often described using the venerable (two-particle) Derjaguin–Landau–Verwey–Overbeek (DLVO) approximation and its various modifications. However, until the recent development of a many-body theory exact at the Debye–Hückel level (Yu in Phys Rev E 102:052404, 2020), it was difficult to assess the errors of such approximations and impossible to assess the role of many-body effects. By applying the exact Debye–Hückel level theory, we quantify the errors inherent to DLVO and the additional errors associated with replacing many-particle interactions by the sum of pairwise interactions (even when the latter are calculated exactly). In particular, we show that: (1) the DLVO approximation does not provide sufficient accuracy at shorter distances, especially when there is an asymmetry in charges and/or sizes of interacting dielectric spheres; (2) the pairwise approximation leads to significant errors at shorter distances and at large and moderate Debye lengths and also gets worse with increasing asymmetry in the size of the spheres or magnitude or placement of the charges. We also demonstrate that asymmetric dielectric screening, i.e., the enhanced repulsion between charged dielectric bodies immersed in media with high dielectric constant, is preserved in the presence of free ions in the medium. Graphic abstract

The Logic of Capital

This book presents the main economic argument developed by Marx in the three volumes of Capital in a coherent and comprehensive manner. The first part presents the main economic argument contained in the Capital in a coherent and comprehensive manner. It also delves into three long-standing debates in Marxist political economy: the transformation problem, the Okishio theorem, and theories of exploitation and oppression. Starting with discussions of methodology, including dialectics and historical materialism, the book explains key concepts of Marxist political economy: commodity, value, money, capital, reserve army of labour, accumulation of capital, circuit of capital, reproduction schemas, prices of production, profit, interest and rent. Scholars of economics, sociology, geography, political science, anthropology, and other kindred disciplines, will find here an accessible yet rigorous treatment of Marxist political economy.

The Chemistry and Physics of Aerogels

Discover a rigorous treatment of aerogels processing and techniques for characterization with this easy-to-use reference. Presents the basics of aerogel synthesis and gelation to open porous nanostructures, and the processing of wet gels like ambient and supercritical drying leading to aerogels. Describes their essential properties with their measurement techniques and theoretical models used to analyse relations to their nanostructure. Linking the fundamentals and with practical applications, this is a useful toolkit for advanced undergraduates, and graduate students doing research in material and polymer science, physical chemistry, and chemical and environmental engineering.

Nietzsche and Music

As the relationship between music and philosophy in Nietzsche’s thought and life continues to fascinate, new approaches to the treatment of music in Nietzsche studies have emerged which take seriously the importance of music, not only in Nietzsche’s life, but for his philosophical project as a whole. While Nietzsche’s often-quoted claim that life without music would be a mistake was once treated as a quip, the quality and breadth of the works reviewed here demonstrate that this invaluable area of Nietzsche’s thought is finally receiving the rigorous treatment it deserves. The works below each offer new and valuable insights on this exciting and growing area of Nietzsche studies which aid us in understanding where to place Nietzsche’s most loved art form in the framework of his philosophy.

Diabetes Mellitus and Heart Failure

Diabetes mellitus (DM) is a major risk factor for new-onset heart failure (HF) and vice versa. The pathogenesis of new-onset HF in DM is complex and has been largely attributed to the toxic cardiovascular effects of hyperglycemia and relevant metabolic abnormalities (diabetic cardiomyopathy) as well as the frequently coexisting morbidities such as hypertension (HTN), coronary artery disease (CAD), and diabetic nephropathy. In patients with type 1 DM (T1DM), HF develops in the setting of a dysregulated immune response, whereas in most patients with type 2 DM (T2DM), against a background of overweight/obesity. HF prevention in DM is feasible with rigorous treatment of cardiovascular risk factors and selective antidiabetic agents. Conversely, development of new-onset T2DM in HF (cardiogenic DM) is common and has been attributed to an increase in the resistance to insulin, especially in the skeletal muscle, liver, and adipose tissue as well as in diminished insulin secretory response to hyperglycemia by pancreatic β-cells. Cardiogenic DM further deteriorates cardiac dysfunction and adversely affects outcome in HF. Novel lifesaving medications employed in HF management such as sacubitril/valsartan and sodium glucose cotransporter 2 inhibitors (SGLT-2i) have a favorable metabolic profile and lower the incidence of cardiogenic diabetes. Whether mitigation of cardiogenic DM should be a treatment target in HF deserves further investigation.

Nietzsche and Music

As the relationship between music and philosophy in Nietzsche’s thought and life continues to fascinate, new approaches to the treatment of music in Nietzsche studies have emerged which take seriously the importance of music, not only in Nietzsche’s life, but for his philosophical project as a whole. While Nietzsche’s often-quoted claim that life without music would be a mistake was once treated as a quip, the quality and breadth of the works reviewed here demonstrate that this invaluable area of Nietzsche’s thought is finally receiving the rigorous treatment it deserves. The works below each offer new and valuable insights on this exciting and growing area of Nietzsche studies which aid us in understanding where to place Nietzsche’s most loved art form in the framework of his philosophy.

Antioxidant Additives versus Health Benefits of Brief/Intermittent Exposure to Potentially Toxic Physical or Chemical Agents

Although antioxidants can act locally to react with an oxidant, oral administration of “antioxidants” is quite useless in treating oxidative stress in tissues. Furthermore, it does not make sense to consider a vitamin as an antioxidant, but vitamin B3 leads to the in vivo formation of compounds that are essential for reducing this stress. A rigorous treatment of the subject indicates that to deal with oxidative stress, the most direct approach is to enhance the innate antioxidant mechanisms. The question is whether this is possible through daily activities. Diets can contain the necessary components for these mechanisms or may induce the expression of the genes involved in them. Another possibility is that pro-oxidant molecules in food increase the sensitivity and power of the detoxification pathways. This option is based on well-known DNA repair mechanisms after exposure to radiation (even from the Sun), or strong evidence of induction of antioxidant capacity after exposure to powerful pro-oxidants such as H2O2. More experimental work is required to test whether some molecules in food can increase the expression of antioxidant enzymes and/or improve antioxidant mechanisms. Identifying effective molecules to achieve such antioxidant power is critical to the food and nutraceutical industries. The potential of diet-based interventions to combat oxidative stress must be viewed from a new perspective.