Implementation of 2D Domain Decomposition in the UCAN Gyrokinetic Particle-in-Cell Code and Resulting Performance of UCAN2

2016 ◽  
Vol 19 (1) ◽  
pp. 205-225 ◽  
Author(s):  
Jean-Noel G. Leboeuf ◽  
Viktor K. Decyk ◽  
David E. Newman ◽  
Raul Sanchez
Keyword(s):  
Domain Decomposition ◽  
Parallel Implementation ◽  
Three Dimensional ◽  
Massively Parallel ◽  
Problem Size ◽  
Time Step ◽  
Particle In Cell ◽  
Minor Radius ◽  
Efficient Charge ◽  
Cartesian Geometry

AbstractThe massively parallel, nonlinear, three-dimensional (3D), toroidal, electrostatic, gyrokinetic, particle-in-cell (PIC), Cartesian geometry UCAN code, with particle ions and adiabatic electrons, has been successfully exercised to identify non-diffusive transport characteristics in present day tokamak discharges. The limitation in applying UCAN to larger scale discharges is the 1D domain decomposition in the toroidal (or z-) direction for massively parallel implementation using MPI which has restricted the calculations to a few hundred ion Larmor radii or gyroradii per plasma minor radius. To exceed these sizes, we have implemented 2D domain decomposition in UCAN with the addition of the y-direction to the processor mix. This has been facilitated by use of relevant components in the P2LIB library of field and particle management routines developed for UCLA's UPIC Framework of conventional PIC codes. The gyro-averaging specific to gyrokinetic codes is simplified by the use of replicated arrays for efficient charge accumulation and force deposition. The 2D domain-decomposed UCAN2 code reproduces the original 1D domain nonlinear results within round-off. Benchmarks of UCAN2 on the Cray XC30 Edison at NERSC demonstrate ideal scaling when problem size is increased along with processor number up to the largest power of 2 available, namely 131,072 processors. These particle weak scaling benchmarks also indicate that the 1 nanosecond per particle per time step and 1 TFlops barriers are easily broken by UCAN2 with 1 billion particles or more and 2000 or more processors.

scholarly journals Domain Decomposition Modified with Characteristic Mixed Finite Element and Numerical Analysis for Three-Dimensional Slightly Compressible Oil-Water Seepage Displacement

2017 ◽  
Vol 9 (1) ◽  
pp. 143
Author(s):  
Yirang Yuan ◽  
Luo Chang ◽  
Changfeng Li ◽  
Tongjun Sun
Keyword(s):  
Finite Element ◽  
Domain Decomposition ◽  
Method Of Characteristics ◽  
Mixed Finite Element ◽  
Three Dimensional ◽  
Optimal Error Estimate ◽  
Computational Domain ◽  
Time Step ◽  
Slightly Compressible ◽  
The Method Of Characteristics

A parallel algorithm is presented to solve three-dimensional slightly compressible seepage displacement where domain decomposition and characteristics-mixed finite element are combined. Decomposing the computational domain into several subdomains, we define a special function to approximate the derivative at interior boundary explicitly and obtain numerical solutions of the saturation implicitly on subdomains in parallel. The method of characteristics can confirm strong stability at the fronts, and can avoid numerical dispersion and nonphysical oscillation. It can adopt large-time step but can obtain small time truncation error. So a characteristic domain decomposition finite element scheme is put forward to compute the saturation. The flow equation is computed by the method of mixed finite element and numerical accuracy of Darcy velocity is improved one order. For a model problem we apply some techniques such as variation form, domain decomposition, the method of characteristics, the principle of energy, negative norm estimates, induction hypothesis, and the theory of priori estimates of differential equations to derive optimal error estimate in $l^2$ norm. Numerical example is given to testify theoretical analysis and numerical data show that this method is effective in solving actual applications. Then it can solve the well-known problem.

scholarly journals Total FETI domain decomposition method and its massively parallel implementation

2013 ◽  
Vol 60-61 ◽  
pp. 14-22 ◽  
Author(s):  
T. Kozubek ◽  
V. Vondrák ◽  
M. Menšı́k ◽  
D. Horák ◽  
Z. Dostál ◽  
...  
Keyword(s):  
Domain Decomposition ◽  
Decomposition Method ◽  
Parallel Implementation ◽  
Domain Decomposition Method ◽  
Massively Parallel

scholarly journals Particle-in-cell simulations of a lens on an f-plane

1997 ◽  
Vol 4 (2) ◽  
pp. 71-91 ◽  
Author(s):  
A. D. Kirwan, Jr. ◽  
C. E. Grosch ◽  
J. J. Holdzkom II.
Keyword(s):  
Euler Equations ◽  
Total Angular Momentum ◽  
Parallel Implementation ◽  
Lower Layer ◽  
Analytic Solutions ◽  
Integral Invariant ◽  
Time Step ◽  
Particle In Cell ◽  
Numerical Viscosity ◽  
Layered Models

Abstract. A particle-in-cell ansatz for solving the Euler equations in a rotating frame is described. The approach is ideally suited for "layered" models of flows with sharp density and velocity fronts. The material and Coriolis accelerations in the Euler equations are solved at each particle while the gradient accelerations are evaluated on a grid and interpolated at each time step to the particles. The height of each particle is fixed and, depending on the application may be constant for all particles or may vary from particle to particle. The approach is used here to predict the evolution of a lens in a layered model with lower layer outcropping. The integral invariant of the volume is conserved exactly and total energy and total angular momentum are conserved to within 3% throughout a 30 day simulation. Exceptional resolution of the density and velocity fronts is maintained during the simulation without imposing any numerical viscosity. the model also reproduces essential characteristics of analytic solutions to a parabolic shaped lens. This algorithm is well suited to parallel implementation; all of the calculations reported here were done on an IBM SP2. Performance speedup and execution time as a function of the number of processors is given.

Three-dimensional deformable-grid electromagnetic particle-in-cell for parallel computers

1999 ◽  
Vol 61 (3) ◽  
pp. 367-389 ◽  
Author(s):  
J. WANG ◽  
D. KONDRASHOV ◽  
P. C. LIEWER ◽  
S. R. KARMESIN
Keyword(s):  
Large Scale ◽  
Parallel Implementation ◽  
Three Dimensional ◽  
Physical Space ◽  
Cartesian Grid ◽  
Second Order ◽  
Surface Integral ◽  
Particle In Cell ◽  
Discrete Surface ◽  
Performance Benchmarks

We describe a new parallel, non-orthogonal-grid, three-dimensional electromagnetic particle-in-cell (EMPIC) code based on a finite-volume formulation. This code uses a logically Cartesian grid of deformable hexahedral cells, a discrete surface integral (DSI) algorithm to calculate the electromagnetic field, and a hybrid logical–physical space algorithm to push particles. We investigate the numerical instability of the DSI algorithm for non-orthogonal grids, analyse the accuracy for EMPIC simulations on non-orthogonal grids, and present performance benchmarks of this code on a parallel supercomputer. While the hybrid particle push algorithm has a second-order accuracy in space, the accuracy of the DSI field solve algorithm is between first and second order for non-orthogonal grids. The parallel implementation of this code, which is almost identical to that of a Cartesian-grid EMPIC code using domain decomposition, achieved a high parallel efficiency of over 96% for large-scale simulations.

Multiscale Modelling of Electrical Contacts Using Domain Decomposition

2012 ◽  
Author(s):  
Piergiorgio Alotto ◽  
Massimo Guarnieri ◽  
Federico Moro
Keyword(s):  
Domain Decomposition ◽  
Domain Decomposition Method ◽  
Thermal Contact ◽  
Three Dimensional ◽  
Contact Problems ◽  
Electrical Contacts ◽  
Computational Domain ◽  
Problem Size ◽  
3D Fem ◽  
Statistical Parameters

A three-dimensional (3D) domain decomposition method for analyzing electrical-thermal contact problems is presented. The computational domain is subdivided into non-overlapping regions discretized according to the Cell Method. Voltage and temperature drops at the contact interfaces are modelled by means of boundary constitutive operators. Continuity between sub-domains is enforced with Lagrange multipliers. The final non-linear algebraic system is solved by an iterative Newton procedure combined to a Schur’s complement approach in order to reduce the problem size and improve the condition number. Potential and temperature jumps across the contact interface depend on the local surface conditions according to Holm’s theory. Surface roughness and a-spot density in the contact area are modelled by means of statistical parameters that can be easily embedded into a CM formulation. The developed code has been validated by a 3D FEM commercial software package.

Parallel Computing of Two Numerical Quadratures for an Integral Formulation of Transient Radiation Transport

2003 ◽  
Author(s):  
Xiaodong Lu ◽  
Pei-Feng Hsu
Keyword(s):  
Integral Equation ◽  
Parallel Computing ◽  
Domain Decomposition ◽  
Three Dimensional ◽  
Spatial Domain ◽  
Equation Model ◽  
Discrete Ordinates ◽  
Scattering Media ◽  
Time Step ◽  
Numerical Quadratures

Parallel computing of the transient radiative transfer process in the three-dimensional homogeneous and nonhomogeneous participating media is studied with an integral equation model. The model can be used for analyzing the ultra-short light pulse propagation within the highly scattering media. Two numerical quadratures are used: the discrete rectangular volume (DRV) method and YIX method. The parallel versions of both methods are developed for one-dimensional and three-dimensional geometries, respectively. Both quadratures achieve good speedup in parallel performance. Because the integral equation model uses very small amount of memory, the parallel computing can take advantage of having each compute node or processor store the full spatial domain information without using the typical domain decomposition parallelism, which will be necessary in other solution methods, e.g., discrete ordinates and finite volume methods, for large scale simulations. The parallel computation is conducted by assigning different portion of the quadrature to different compute node. In DRV method, a variation of the spatial domain decomposition is used. In the case of YIX scheme, the angular quadrature is divided up according to the number of compute nodes, instead of the spatial domain being divided. This parallel scheme minimizes the communications overhead. The only communication needed is at the end of each time step when each node shares the partial integrated result of the current time step with all other compute nodes. The angular quadrature decomposition approach leads to very good parallel efficiency. Two new discrete ordinate sets are used in the YIX angular quadrature and their parallel performances are discussed. One of the discrete ordinates sets, called spherical ring set, is also suitable for use in the conventional discrete ordinates method.

TIME-DOMAIN PARALLEL SIMULATION OF HETEROGENEOUS WAVE PROPAGATION ON UNSTRUCTURED GRIDS USING EXPLICIT, NONDIFFUSIVE, DISCONTINUOUS GALERKIN METHODS

2006 ◽  
Vol 14 (01) ◽  
pp. 57-81 ◽  
Author(s):  
MARC BERNACKI ◽  
STEPHANE LANTERI ◽  
SERGE PIPERNO
Keyword(s):  
Wave Propagation ◽  
Discontinuous Galerkin ◽  
Wave Energy ◽  
Time Domain ◽  
Balance Equation ◽  
Message Passing ◽  
Heterogeneous Media ◽  
Parallel Implementation ◽  
Three Dimensional ◽  
Time Step

A general Discontinuous Galerkin framework is introduced for symmetric systems of conservations laws. It is applied to the three-dimensional electromagnetic wave propagation in heterogeneous media, and to the propagation of aeroacoustic perturbations of either uniform or nonuniform, steady solutions of the three-dimensional Euler equations. In all these linear contexts, the time evolution of some quadratic wave energy is given in a balance equation, with a volumic source term for aeroacoustics in a nonuniform flow. An explicit leap-frog time scheme along with centered numerical fluxes are used in the proposed Discontinuous Galerkin Time Domain (DGTD) method, in order to achieve a discrete equivalent of the balance equation for the wave energy. The scheme introduced is genuinely nondissipative. Numerical first-order boundary conditions are developed to bound the domain and stability is proved on arbitrary unstructured meshes and discontinuous finite elements, under some CFL-like stability condition on the time step. Numerical results obtained with a parallel implementation of the method based on mesh partitioning and message passing are presented to show the potential of the method.

Sign in / Sign up

Export Citation Format

Share Document