Expression Templates and Forward Mode Automatic Differentiation

Cache-Aware and Roofline-Ideal Automatic Differentiation

10.2118/203933-ms ◽

2021 ◽

Author(s):

Yuxuan Jing ◽

Rami M. Younis

Keyword(s):

Automatic Differentiation ◽

Simulation Software ◽

Code Optimization ◽

Discrete Operator ◽

Lazy Evaluation ◽

Software Libraries ◽

Language Constructs ◽

Working Set ◽

Expression Templates ◽

Programming Interface

Abstract Automatic differentiation software libraries augment arithmetic operations with their derivatives, thereby relieving the programmer of deriving, implementing, debugging, and maintaining derivative code. With this encapsulation however, the responsibility of code optimization relies more heavily on the AD system itself (as opposed to the programmer and the compiler). Moreover, given that there are multiple contexts in reservoir simulation software for which derivatives are required (e.g. property package and discrete operator evaluations), the AD infrastructure must also be adaptable. An Operator Overloading AD design is proposed and tested to provide scalability and computational efficiency seemlessly across memory- and compute-bound applications. This is achieved by 1) use of portable and standard programming language constructs (C++17 and OpenMP 4.5 standards), 2) adopting a vectorized programming interface, 3) lazy evaluation via expression templates, and 4) multiple memory alignment and layout policies. Empirical analysis is conducted on various kernels spanning various arithmetic intensity and working set sizes. Cache- aware roofline analysis results show that the performance and scalability attained are reliably ideal. In terms of floapting point operations executed per second, the performance of the AD system matches optimized hand-code. Finally, the implementation is benchmarked using the Automatically Differentiable Expression Templates Library (ADETL).

Download Full-text

A Domain Specific Embedded Language in C++ for Automatic Differentiation, Projection, Integration and Variational Formulations

Scientific Programming ◽

10.1155/2006/150736 ◽

2006 ◽

Vol 14 (2) ◽

pp. 81-110 ◽

Cited By ~ 9

Author(s):

Christophe Prud'homme

Keyword(s):

Numerical Integration ◽

Automatic Differentiation ◽

Functional Space ◽

Test Problems ◽

Domain Specific ◽

Meta Programming ◽

Programming Techniques ◽

Expression Templates ◽

Performance Behavior ◽

Variational Formulations

In this article, we present a domain specific embedded language inC++ that can be used in various contexts such as numerical projection onto a functional space, numerical integration, variational formulations and automatic differentiation. Albeit these tools operate in different ways, the language overcomes this difficulty by decoupling expression constructions from evaluation. The language is implemented using expression templates and meta-programming techniques and uses various Boost libraries. The language is exercised on a number of non-trivial examples and a benchmark presents the performance behavior on a few test problems.

Download Full-text