MPI Parallelization for Two-Way Coupled Euler-Lagrange Simulation of Microbubble Enhanced HIFU

Microbubble enhanced High Intensity Focused Ultrasound (HIFU) is of great interest to tissue ablation for tumor treatment such as in liver and brain cancers, in which ultrasonic contrast agent microbubbles are injected to the targeted region to promote local heating while reducing pre-focal damage. To accurately characterize the acoustic and thermal fields during this process, a compressible Euler-Lagrange model is used. The non-linear ultrasound field is modeled using compressible N-S equations on an Eulerian grid, while the microbubbles are tracked as discrete singularities in a Lagrangian fashion with their dynamics computed. Their intimate coupling is realized through the local void fraction, which is computed from the instantaneous bubble volumes and locations, and then fed to the fluid continuum model. Owing to demanding computational cost in real applications, schemes for significant speedup are highly desirable. We present here a MPI parallelization scheme based on domain decomposition for both the continuum fluid and the discrete bubbles. The Eulerian computational domain is subdivided into several subdomains having each the same number of grids, while the bubbles are subdivided based on their locations corresponding to each subdomain. During each computation time step, MPI processors, each handling one subdomain, are 1) first used to execute the fluid computation, and 2) then to execute the bubble computations, 3) followed by the coupling procedure, which maps the void fraction from the Lagrangian bubble solutions into the Eulerian grids. Steps 1) and 2) are relatively straightforward by routinely following regular MPI procedures. However, step 3) becomes challenging as the effect of the bubbles through the void fraction at an Eulerian point near a subdomain border will require information from bubbles located in different subdomains. Similarly, a bubble near a border between subdomains will spread its contribution to the void fraction of different subdomains. This is addressed by a special utilization of ghost cells surrounding each fluid subdomain, which allows bubbles to spread their void fraction effects across subdomain edges without the need of exchanging directly bubble information between subdomains and significantly increasing overhead. This void fraction implementation is verified by gas volume conservation before and after spreading the bubble effects. Other bubble effects such as thermal effects are handled in a similar way. This parallelization scheme is validated and illustrated on a typical microbubble enhanced HIFU problem, followed by parallelization scaling tests and efficiency analysis.