scholarly journals Provably correct, asymptotically efficient, higher-order reverse-mode automatic differentiation

2022 ◽  
Vol 6 (POPL) ◽  
pp. 1-30
Author(s):  
Faustyna Krawiec ◽  
Simon Peyton Jones ◽  
Neel Krishnaswami ◽  
Tom Ellis ◽  
Richard A. Eisenberg ◽  
...  
Keyword(s):  
Automatic Differentiation ◽  
Higher Order ◽  
Efficient Implementation ◽  
Formal Description ◽  
Memory Consumption ◽  
Reverse Mode ◽  
Original Program ◽  
Run Time ◽  
Asymptotically Efficient ◽  
Higher Order Functions

In this paper, we give a simple and efficient implementation of reverse-mode automatic differentiation, which both extends easily to higher-order functions, and has run time and memory consumption linear in the run time of the original program. In addition to a formal description of the translation, we also describe an implementation of this algorithm, and prove its correctness by means of a logical relations argument.

scholarly journals Perturbation confusion in forward automatic differentiation of higher-order functions

2019 ◽  
Vol 29 ◽  
Author(s):  
OLEKSANDR MANZYUK ◽  
BARAK A. PEARLMUTTER ◽  
ALEXEY ANDREYEVICH RADUL ◽  
DAVID R. RUSH ◽  
JEFFREY MARK SISKIND
Keyword(s):  
Automatic Differentiation ◽  
Computer Programs ◽  
Higher Order ◽  
The Other ◽  
Order Function ◽  
Accumulation Mode ◽  
The Creation ◽  
One To One ◽  
Derivatives Of ◽  
Higher Order Functions

Abstract Automatic differentiation (AD) is a technique for augmenting computer programs to compute derivatives. The essence of AD in its forward accumulation mode is to attach perturbations to each number, and propagate these through the computation by overloading the arithmetic operators. When derivatives are nested, the distinct derivative calculations, and their associated perturbations, must be distinguished. This is typically accomplished by creating a unique tag for each derivative calculation and tagging the perturbations. We exhibit a subtle bug, present in fielded implementations which support derivatives of higher-order functions, in which perturbations are confused despite the tagging machinery, leading to incorrect results. The essence of the bug is as follows: a unique tag is needed for each derivative calculation, but in existing implementations unique tags are created when taking the derivative of a function at a point. When taking derivatives of higher-order functions, these need not correspond! We exhibit a simple example: a higher-order function f whose derivative at a point x, namely f′(x), is itself a function which calculates a derivative. This situation arises naturally when taking derivatives of curried functions. Two potential solutions are presented, and their deficiencies discussed. One uses eta expansion to delay the creation of fresh tags in order to put them into one-to-one correspondence with derivative calculations. The other wraps outputs of derivative operators with tag substitution machinery. Both solutions seem very difficult to implement without violating the desirable complexity guarantees of forward AD.

scholarly journals Voigt Transform and Umbral Image

2020 ◽  
Vol 25 (3) ◽  
pp. 49
Author(s):  
Silvia Licciardi ◽  
Rosa Maria Pidatella ◽  
Marcello Artioli ◽  
Giuseppe Dattoli
Keyword(s):  
Spectroscopic Studies ◽  
Higher Order ◽  
Point Of View ◽  
The Other ◽  
Pivotal Role ◽  
Umbral Calculus ◽  
Laguerre Functions ◽  
Hermite And Laguerre Functions ◽  
Higher Order Functions ◽  
Voigt Function

In this paper, we show that the use of methods of an operational nature, such as umbral calculus, allows achieving a double target: on one side, the study of the Voigt function, which plays a pivotal role in spectroscopic studies and in other applications, according to a new point of view, and on the other, the introduction of a Voigt transform and its possible use. Furthermore, by the same method, we point out that the Hermite and Laguerre functions, extension of the corresponding polynomials to negative and/or real indices, can be expressed through a definition in a straightforward and unified fashion. It is illustrated how the techniques that we are going to suggest provide an easy derivation of the relevant properties along with generalizations to higher order functions.

Higher-Order Functions

2017 ◽  
pp. 63-76
Author(s):  
Thomas Mailund
Keyword(s):  
Higher Order ◽  
Higher Order Functions

Higher-Order Functions

2017 ◽  
pp. 29-43
Author(s):  
Anto Aravinth
Keyword(s):  
Higher Order ◽  
Higher Order Functions

Higher-order functions

2012 ◽  
pp. 61-73
Author(s):  
Graham Hutton
Keyword(s):  
Higher Order ◽  
Higher Order Functions

On the Use of the Automatic Differentiation for Developing a Linear Harmonic Solver

2018 ◽  
Author(s):  
Wei Zhang ◽  
Dingxi Wang ◽  
Xiuquan Huang ◽  
Tianxiao Yang ◽  
Hong Yan ◽  
...  
Keyword(s):  
Harmonic Balance ◽  
Automatic Differentiation ◽  
Harmonic Balance Method ◽  
Source Codes ◽  
Reverse Mode ◽  
Frequency Domain Methods ◽  
The Cost ◽  
Time Periodic ◽  
Small Disturbances

The linear and nonlinear harmonic methods are efficient frequency domain methods for analyzing time periodic unsteady flow fields. They have been widely used in both academia and industry. But the cost and complexity of developing a linear harmonic solver has been limiting its wider applications. On the other hand, the automatic differentiation (AD) has long been used in the CFD community with a focus on generating adjoint codes in a reverse mode. All those AD tools can do a much better job in generating linearized codes in a tangent mode, but so far very little, if any, attention is paid to using AD for developing linear harmonic solvers. The linear harmonic method, in comparison with the harmonic balance method, has its own advantages. For example, it can capture small disturbances very effectively, and avoids aliasing errors which can lead to solution instability since each wave component is solved for separately. This paper presents the effort of using an AD tool to generate major source codes for the development of a linear harmonic solver for analyzing time periodic unsteady flows. It includes the procedures and advice of using AD for such a purpose. A case study is also presented to validate the developed linear harmonic solver.

Static Parallelization of Functional Programs: Elimination of Higher-Order Functions & Optimized Inlining

1999 ◽  
pp. 930-934
Author(s):  
Christoph A. Herrmann ◽  
Christian Lengauer ◽  
Jan Laitenberger ◽  
Christian Schaller
Keyword(s):  
Higher Order ◽  
Higher Order Functions
Sign in / Sign up

Export Citation Format

Share Document