scholarly journals Seismic Waveform Inversion by Stochastic Optimization

2011 ◽  
Vol 2011 ◽  
pp. 1-18 ◽  
Author(s):  
Tristan van Leeuwen ◽  
Aleksandr Y. Aravkin ◽  
Felix J. Herrmann
Keyword(s):  
Stochastic Optimization ◽  
Waveform Inversion ◽  
Computational Cost ◽  
Optimization Methods ◽  
Batch Size ◽  
Stochastic Methods ◽  
Seismic Waveform ◽  
Recent Developments ◽  
Order Of Magnitude ◽  
Batch Sizes

We explore the use of stochastic optimization methods for seismic waveform inversion. The basic principle of such methods is to randomly draw a batch of realizations of a given misfit function and goes back to the 1950s. The ultimate goal of such an approach is to dramatically reduce the computational cost involved in evaluating the misfit. Following earlier work, we introduce the stochasticity in waveform inversion problem in a rigorous way via a technique calledrandomized trace estimation. We then review theoretical results that underlie recent developments in the use of stochastic methods for waveform inversion. We present numerical experiments to illustrate the behavior of different types of stochastic optimization methods and investigate the sensitivity to the batch size and the noise level in the data. We find that it is possible to reproduce results that are qualitatively similar to the solution of the full problem with modest batch sizes, even on noisy data. Each iteration of the corresponding stochastic methods requires an order of magnitude fewer PDE solves than a comparable deterministic method applied to the full problem, which may lead to an order of magnitude speedup for waveform inversion in practice.

scholarly journals Accelerating numerical wave propagation using wavefield adapted meshes. Part I: forward and adjoint modelling

2020 ◽  
Vol 221 (3) ◽  
pp. 1580-1590 ◽  
Author(s):  
M van Driel ◽  
C Boehm ◽  
L Krischer ◽  
M Afanasiev
Keyword(s):  
Wave Propagation ◽  
Adaptive Mesh Refinement ◽  
Waveform Inversion ◽  
Computational Cost ◽  
Model Space ◽  
Adaptive Mesh ◽  
Order Of Magnitude ◽  
Speed Up ◽  
Adjoint Modelling ◽  
The Waves

SUMMARY An order of magnitude speed-up in finite-element modelling of wave propagation can be achieved by adapting the mesh to the anticipated space-dependent complexity and smoothness of the waves. This can be achieved by designing the mesh not only to respect the local wavelengths, but also the propagation direction of the waves depending on the source location, hence by anisotropic adaptive mesh refinement. Discrete gradients with respect to material properties as needed in full waveform inversion can still be computed exactly, but at greatly reduced computational cost. In order to do this, we explicitly distinguish the discretization of the model space from the discretization of the wavefield and derive the necessary expressions to map the discrete gradient into the model space. While the idea is applicable to any wave propagation problem that retains predictable smoothness in the solution, we highlight the idea of this approach with instructive 2-D examples of forward as well as inverse elastic wave propagation. Furthermore, we apply the method to 3-D global seismic wave simulations and demonstrate how meshes can be constructed that take advantage of high-order mappings from the reference coordinates of the finite elements to physical coordinates. Error level and speed-ups are estimated based on convergence tests with 1-D and 3-D models.

scholarly journals Accelerating numerical wave propagation by wavefield adapted meshes. Part II: full-waveform inversion

2020 ◽  
Vol 221 (3) ◽  
pp. 1591-1604 ◽  
Author(s):  
Solvi Thrastarson ◽  
Martin van Driel ◽  
Lion Krischer ◽  
Christian Boehm ◽  
Michael Afanasiev ◽  
...  
Keyword(s):  
Wave Propagation ◽  
Wave Equation ◽  
Waveform Inversion ◽  
Computational Cost ◽  
Spectral Element ◽  
Full Waveform Inversion ◽  
Full Waveform ◽  
Order Of Magnitude ◽  
Numerical Wave ◽  
Expected Complexity

SUMMARY We present a novel full-waveform inversion (FWI) approach which can reduce the computational cost by up to an order of magnitude compared to conventional approaches, provided that variations in medium properties are sufficiently smooth. Our method is based on the usage of wavefield adapted meshes which accelerate the forward and adjoint wavefield simulations. By adapting the mesh to the expected complexity and smoothness of the wavefield, the number of elements needed to discretize the wave equation can be greatly reduced. This leads to spectral-element meshes which are optimally tailored to source locations and medium complexity. We demonstrate a workflow which opens up the possibility to use these meshes in FWI and show the computational advantages of the approach. We provide examples in 2-D and 3-D to illustrate the concept, describe how the new workflow deviates from the standard FWI workflow, and explain the additional steps in detail.

scholarly journals Inefficiency of K-FAC for Large Batch Size Training

2020 ◽  
Vol 34 (04) ◽  
pp. 5053-5060
Author(s):  
Linjian Ma ◽  
Gabe Montague ◽  
Jiayu Ye ◽  
Zhewei Yao ◽  
Amir Gholami ◽  
...  
Keyword(s):  
Gradient Descent ◽  
Computational Cost ◽  
Parameter Tuning ◽  
Stochastic Gradient Descent ◽  
Batch Size ◽  
Common Belief ◽  
Record Times ◽  
Large Batch ◽  
Strong Scaling ◽  
Batch Sizes

There have been several recent work claiming record times for ImageNet training. This is achieved by using large batch sizes during training to leverage parallel resources to produce faster wall-clock training times per training epoch. However, often these solutions require massive hyper-parameter tuning, which is an important cost that is often ignored. In this work, we perform an extensive analysis of large batch size training for two popular methods that is Stochastic Gradient Descent (SGD) as well as Kronecker-Factored Approximate Curvature (K-FAC) method. We evaluate the performance of these methods in terms of both wall-clock time and aggregate computational cost, and study the hyper-parameter sensitivity by performing more than 512 experiments per batch size for each of these methods. We perform experiments on multiple different models on two datasets of CIFAR-10 and SVHN. The results show that beyond a critical batch size both K-FAC and SGD significantly deviate from ideal strong scaling behaviour, and that despite common belief K-FAC does not exhibit improved large-batch scalability behavior, as compared to SGD.

Whole Earth Full-Waveform Inversion With Wavefield Adapted Meshes

2020 ◽  
Author(s):  
Solvi Thrastarson ◽  
Dirk-Philip van Herwaarden ◽  
Lion Krischer ◽  
Christian Boehm ◽  
Martin van Driel ◽  
...  
Keyword(s):  
Stochastic Optimization ◽  
High Performance ◽  
Waveform Inversion ◽  
Computational Cost ◽  
Real Data ◽  
Global Scale ◽  
Full Waveform Inversion ◽  
Optimization Scheme ◽  
Continental Scale ◽  
Full Waveform

<p>With the steadily increasing availability and density of seismic data, full-waveform inversion (FWI) can reveal the Earth's subsurface with unprecedented resolution. FWI, however, carries a significant computational burden. Even with the ever-increasing power of high-performance computing resources, these massive compute requirements inhibit substantial progress, and require algorithmic and technological innovations for global and continental scale inversions.<br>In this contribution, we present an approach to FWI where we achieve significant computational savings through wavefield adapted meshing [1] combined with a stochastic optimization scheme [2]. This twofold strategy allows us (a) to solve the wave equation at lower costs, and (b) to reduce the number of required simulations. In laterally smooth media, we can construct meshes which are adapted to the expected complexity of the wavefield. By optimally designing a unique mesh for each source, we can reduce the computational cost of the forward and adjoint simulations by an order of magnitude. The stochastic optimization scheme is based on a dynamic mini-batch L-BFGS approach, which adaptively subsamples the event catalogue and requires significantly fewer wavefield simulations to converge to a model than conventional FWI. An additional benefit of the dynamic mini-batches is that they seamlessly allow for the inclusion of more sources in an inversion without a considerable additional computational cost.<br>We demonstrate a prototype FWI for this approach towards a global scale inversion with real data.<br><br>[1] Thrastarson, S., van Driel, M., Krischer, L., Afanasiev, M., Boehm, C., van Herwaarden, DP., Fichtner, A., 2019. Accelerating numerical wave propagation by wavefield adapted meshes, Part II: Full-waveform inversion. <em>Submitted to Geophysical Journal International</em><br>[2] van Herwaarden, DP., Boehm, C., Afanasiev, M., Krischer, L., van Driel, M., Thrastarson, S., Trampert, J., Fichtner, A. 2019. Accelerated full-waveform inversion using dynamic mini-batches. <em>Submitted to Geophysical Journal International</em></p>

scholarly journals Seismic shot-encoding schemes for waveform inversion

2020 ◽  
Vol 17 (5) ◽  
pp. 906-913 ◽  
Author(s):  
Edwin Fagua Duarte ◽  
Carlos A N da Costa ◽  
João M de Araújo ◽  
Yanghua Wang ◽  
Ying Rao
Keyword(s):  
Waveform Inversion ◽  
Random Phase ◽  
Computational Cost ◽  
Velocity Model ◽  
Dominant Frequency ◽  
Seismic Waveform ◽  
Rotation Scheme ◽  
Iterative Inversion ◽  
Inversion Procedure ◽  
The Time Domain

Abstract A shot-encoding technique can be used in seismic waveform inversion to significantly reduce the computational cost by reducing the number of seismic simulations in the inversion procedure. Here we developed two alternative shot-encoding schemes to perform simultaneous-sources waveform inversion. The first scheme (I) encodes shot gathers with random-phase rotations applied to seismic traces. The second scheme (II) encodes shot gathers with random static time shifts. The well-known polarity encoding scheme (III) is just a special case of the random-phase rotation scheme. The second scheme is a variation of the conventional static shift encoding (IV), but the static time shifts in the second scheme are limited to one period of the dominant frequency. All encoded shot gathers are added up into a single super-shot gather for seismic waveform inversion. We perform the time-domain waveform inversion, using these shot-encoding schemes in conjunction with a restarted L-BFGS algorithm in the iterative inversion. The effectiveness and efficiency analyses demonstrate that the two shot-encoding schemes (I and II) proposed in this paper may improve the convergence of the iterative inversion, reduce the crosstalk effect among shots and consequently produce a subsurface velocity model with a high resolution.

A subsampled truncated-Newton method for multiparameter full-waveform inversion

Geophysics ◽  
2019 ◽  
Vol 84 (3) ◽  
pp. R333-R340 ◽  
Author(s):  
Gian Matharu ◽  
Mauricio Sacchi
Keyword(s):  
Convergence Rates ◽  
Waveform Inversion ◽  
Computational Cost ◽  
Gradient Methods ◽  
Optimization Methods ◽  
Isotropic Case ◽  
Full Waveform Inversion ◽  
Full Waveform ◽  
Source Contributions ◽  
Truncated Newton

Accounting for the Hessian in full-waveform inversion (FWI) can lead to higher convergence rates, improved resolution, and better mitigation of parameter trade-off in multiparameter problems. In spite of these advantages, the adoption of second-order optimization methods (e.g., truncated Newton [TN]) has been precluded by their high computational cost. We propose a subsampled TN (STN) algorithm for time-domain FWI with applications presented for the elastic isotropic case. By using uniform or nonuniform source subsampling during the computation of Hessian-vector products, we reduce the number of partial differential equation solves required per iteration when compared to the conventional TN algorithm. We evaluate the performance of STN through synthetic inversions on the Marmousi II and BP 2.5D models, using the limited-memory Broyden–Fletcher–Goldfarb–Shanno and TN algorithms as benchmarks. We determine that STN reaches a target misfit reduction at an overall cost comparable to first-order gradient methods, while retaining favorable convergence properties of TN methods. Furthermore, we evaluate an example in which nonuniform sampling outperforms uniform sampling in STN due to highly nonuniform source contributions to the Hessian.

scholarly journals The importance of better models in stochastic optimization

2019 ◽  
Vol 116 (46) ◽  
pp. 22924-22930 ◽  
Author(s):  
Hilal Asi ◽  
John C. Duchi
Keyword(s):  
Machine Learning ◽  
Stochastic Optimization ◽  
Optimization Methods ◽  
Convergence Time ◽  
Learning Problems ◽  
Stochastic Methods ◽  
Weakly Convex ◽  
Smooth Functions ◽  
Asymptotically Optimal ◽  
Modern Machine

Standard stochastic optimization methods are brittle, sensitive to stepsize choice and other algorithmic parameters, and they exhibit instability outside of well-behaved families of objectives. To address these challenges, we investigate models for stochastic optimization and learning problems that exhibit better robustness to problem families and algorithmic parameters. With appropriately accurate models—which we call the aprox family—stochastic methods can be made stable, provably convergent, and asymptotically optimal; even modeling that the objective is nonnegative is sufficient for this stability. We extend these results beyond convexity to weakly convex objectives, which include compositions of convex losses with smooth functions common in modern machine learning. We highlight the importance of robustness and accurate modeling with experimental evaluation of convergence time and algorithm sensitivity.

scholarly journals Machine Learning for Design Optimization of Electromagnetic Devices: Recent Developments and Future Directions

2021 ◽  
Vol 11 (4) ◽  
pp. 1627
Author(s):  
Yanbin Li ◽  
Gang Lei ◽  
Gerd Bramerdorfer ◽  
Sheng Peng ◽  
Xiaodong Sun ◽  
...  
Keyword(s):  
Machine Learning ◽  
Design Optimization ◽  
Optimization Methods ◽  
Machine Learning Algorithms ◽  
Cloud Services ◽  
Robust Design Optimization ◽  
Support Vector ◽  
Future Directions ◽  
Electromagnetic Devices ◽  
Recent Developments

This paper reviews the recent developments of design optimization methods for electromagnetic devices, with a focus on machine learning methods. First, the recent advances in multi-objective, multidisciplinary, multilevel, topology, fuzzy, and robust design optimization of electromagnetic devices are overviewed. Second, a review is presented to the performance prediction and design optimization of electromagnetic devices based on the machine learning algorithms, including artificial neural network, support vector machine, extreme learning machine, random forest, and deep learning. Last, to meet modern requirements of high manufacturing/production quality and lifetime reliability, several promising topics, including the application of cloud services and digital twin, are discussed as future directions for design optimization of electromagnetic devices.

Sign in / Sign up

Export Citation Format

Share Document