Trilinos Solvers Scalability on a MFiX-Trilinos Framework Applied to Fluidized Bed Simulations

2020 ◽  
Author(s):  
Arturo Rodriguez ◽  
V. M. Krushnarao Kotteda ◽  
Luis F. Rodriguez ◽  
Vinod Kumar ◽  
Jorge A. Munoz
Keyword(s):  
Linear System ◽  
Fluidized Bed ◽  
Large Scale ◽  
Fossil Fuel ◽  
State Of The Art ◽  
Iterative Solvers ◽  
Flow Solver ◽  
Nonlinear Solvers ◽  
Fuel Reactor ◽  
Preconditioned Iterative

Abstract MFiX is a multiphase open-source suite that is developed at the National Energy Technology Laboratories. It is widely used by fossil fuel reactor communities to simulate flow in a fluidized bed reactor. It does not have advanced linear iterative solvers even though it spends 70% of the run time in solving the linear system. Trilinos contains algorithms and enabling technologies for the solution of large-scale, sophisticated multi-physics engineering and scientific problems. The library developed at Sandia National Laboratories has more than 60 packages. It consists of state-of-the-art preconditioners, nonlinear solvers, direct solvers, and iterative solvers. The packages are performant and portable on various hybrid computing architectures. To improve the capabilities of MFiX, we developed a framework, MFiX-Trilinos, to integrate the advanced linear solvers in Trilinos with the FORTRAN based multiphase flow solver, MFiX. The framework changes the semantics of the array in FORTRAN and C++ and solve the linear system with packages in Trilinos and returns the solution to MFiX. The preconditioned iterative solvers considered for the analysis are BiCGStab and GMRES. The framework is verified on various fluidized bed problems. The performance of the framework is tested on the Stampede supercomputer. The wall time for multiple sizes of fluidized beds is compared.

scholarly journals Next-Generation Multiphase Flow Solver for Fluidized Bed Applications

2017 ◽  
Author(s):  
V. M. Krushnarao Kotteda ◽  
Ashesh Chattopadhyay ◽  
Vinod Kumar ◽  
William Spotz
Keyword(s):  
Multiphase Flow ◽  
Open Source ◽  
Fluidized Bed ◽  
Large Scale ◽  
General Purpose ◽  
Iterative Solvers ◽  
Next Generation ◽  
Linear Solvers ◽  
Development Platform ◽  
The Matrix

A framework is developed to integrate MFiX (Multiphase Flow with Interphase eXchanges) with advanced linear solvers in Trilinos. MFiX is a widely used open source general purpose multiphase solver developed by National Energy Technology Laboratories and written in Fortran. Trilinos is an objected-oriented open source software development platform from Sandia National Laboratories for solving large scale multiphysics problems. The framework handles the different data structures in Fortran and C++ and exchanges the information from MFiX to Trilinos and vice versa. The integrated solver, called MFiX-Trilinos hereafter, provides next-generation computational capabilities including scalable linear solvers for distributed memory massively parallel computers. In this paper, the solution from the standard linear solvers in MFiX-Trilinos is validated against the same from MFiX for 2D and 3D fluidized bed problems. The standard iterative solvers considered in this work are Bi-Conjugate Gradient Stabilized (BiCGStab) and Generalized minimal residual methods (GMRES) as the matrix is non-symmetric in nature. The stopping criterion set for the iterative solvers is same. It is observed that the solution from the integrated solver and MFiX is in good agreement.

scholarly journals Performance Models for the Spike Banded Linear System Solver

2011 ◽  
Vol 19 (1) ◽  
pp. 13-25
Author(s):  
Murat Manguoglu ◽  
Faisal Saied ◽  
Ahmed Sameh ◽  
Ananth Grama
Keyword(s):  
Linear System ◽  
Large Scale ◽  
Performance Model ◽  
Iterative Solvers ◽  
Analytical Performance ◽  
Analytical Models ◽  
Cost Models ◽  
Performance Models ◽  
Performance Characterization ◽  
Parallel Performance

With availability of large-scale parallel platforms comprised of tens-of-thousands of processors and beyond, there is significant impetus for the development of scalable parallel sparse linear system solvers and preconditioners. An integral part of this design process is the development of performance models capable of predicting performance and providing accurate cost models for the solvers and preconditioners. There has been some work in the past on characterizing performance of the iterative solvers themselves. In this paper, we investigate the problem of characterizing performance and scalability of banded preconditioners. Recent work has demonstrated the superior convergence properties and robustness of banded preconditioners, compared to state-of-the-art ILU family of preconditioners as well as algebraic multigrid preconditioners. Furthermore, when used in conjunction with efficient banded solvers, banded preconditioners are capable of significantly faster time-to-solution. Our banded solver, the Truncated Spike algorithm is specifically designed for parallel performance and tolerance to deep memory hierarchies. Its regular structure is also highly amenable to accurate performance characterization. Using these characteristics, we derive the following results in this paper: (i) we develop parallel formulations of the Truncated Spike solver, (ii) we develop a highly accurate pseudo-analytical parallel performance model for our solver, (iii) we show excellent predication capabilities of our model – based on which we argue the high scalability of our solver. Our pseudo-analytical performance model is based on analytical performance characterization of each phase of our solver. These analytical models are then parameterized using actual runtime information on target platforms. An important consequence of our performance models is that they reveal underlying performance bottlenecks in both serial and parallel formulations. All of our results are validated on diverse heterogeneous multiclusters – platforms for which performance prediction is particularly challenging. Finally, we provide predict the scalability of the Spike algorithm using up to 65,536 cores with our model. In this paper we extend the results presented in the Ninth International Symposium on Parallel and Distributed Computing.

scholarly journals A GPU-accelerated adaptive FSAI preconditioner for massively parallel simulations

2021 ◽  
pp. 109434202110171
Author(s):  
Giovanni Isotton ◽  
Carlo Janna ◽  
Massimo Bernaschi
Keyword(s):  
Linear Systems ◽  
Large Scale ◽  
Iterative Solvers ◽  
Ideal Behavior ◽  
Parallel Simulations ◽  
Numerical Experimentation ◽  
Approximate Inverses ◽  
Preconditioned Iterative ◽  
Set Up ◽  
High Degree

The solution of linear systems of equations is a central task in a number of scientific and engineering applications. In many cases the solution of linear systems may take most of the simulation time thus representing a major bottleneck in the further development of scientific and technical software. For large scale simulations, nowadays accounting for several millions or even billions of unknowns, it is quite common to resort to preconditioned iterative solvers for exploiting their low memory requirements and, at least potential, parallelism. Approximate inverses have been shown to be robust and effective preconditioners in various contexts. In this work, we show how adaptive Factored Sparse Approximate Inverse (aFSAI), characterized by a very high degree of parallelism, can be successfully implemented on a distributed memory computer equipped with GPU accelerators. Taking advantage of GPUs in adaptive FSAI set-up is not a trivial task, nevertheless we show through an extensive numerical experimentation how the proposed approach outperforms more traditional preconditioners and results in a close-to-ideal behavior in challenging linear algebra problems.

Next Generation Exascale Capable Mutliphase Solver With Trilinos

2016 ◽  
Author(s):  
Ashesh Chattopadhyay ◽  
V. M. Krushnarao Kotteda ◽  
Vinod Kumar ◽  
William Spotz
Keyword(s):  
Multiphase Flow ◽  
Linear Equation ◽  
Fluidized Bed ◽  
State Of The Art ◽  
Next Generation ◽  
Flow Solver ◽  
Flow Problems ◽  
Linear Equation Solver

A framework is developed to integrate the existing MFiX (Multiphase Flow with Interphase eXchanges) flow solver with state-of-the-art linear equation solver packages in Trilinos. The integrated solver is tested on various flow problems. The performance of the solver is evaluated on fluidized bed problems and observed that the integrated flow solver performs better compared to the native solver.

A GPU-Accelerated Linear Solver for Massively Parallel Underground Simulations

2021 ◽  
Author(s):  
Giovanni Isotton ◽  
Carlo Janna ◽  
Nicoló Spiezia ◽  
Omar Tosatto ◽  
Massimo Bernaschi ◽  
...  
Keyword(s):  
Large Scale ◽  
Iterative Solvers ◽  
Sparse Approximate Inverse ◽  
Modern Engineering ◽  
Numerical Experimentation ◽  
Approximate Inverses ◽  
Preconditioned Iterative ◽  
Set Up ◽  
High Level ◽  
Large Scale Simulations

Abstract Modern engineering applications require the solution of linear systems of millions or even billions of equations. The solution of the linear system takes most of the simulation for large scale simulations, and represent the bottleneck in developing scientific and technical software. Usually, preconditioned iterative solvers are preferred because of their low memory requirements and they can have a high level of parallelism. Approximate inverses have been proven to be robust and effective preconditioners in several contexts. In this communication, we present an adaptive Factorized Sparse Approximate Inverse (FSAI) preconditioner with a very high level of parallelism in both set-up and application. Its inherent parallelism makes FSAI an ideal candidate for a GPU-accelerated implementation, even if taking advantage of this hardware is not a trivial task, especially in the set-up stage. An extensive numerical experimentation has been performed on industrial underground applications. It is shown that the proposed approach outperforms more traditional preconditioners in challenging underground simulation, greatly reducing time-to-solution.

Solution of large-scale 3D controlled-source electromagnetic modeling problem using efficient iterative solvers

Geophysics ◽  
2021 ◽  
pp. 1-64
Author(s):  
Changkai Qiu ◽  
Changchun Yin ◽  
Yunhe Liu ◽  
Xiuyan Ren ◽  
Hui Chen ◽  
...  
Keyword(s):  
Linear System ◽  
Linear Equation ◽  
Frequency Domain ◽  
Time Domain ◽  
Large Scale ◽  
Equation System ◽  
Iterative Solvers ◽  
Controlled Source ◽  
Linear Equation System ◽  
The Time Domain

With geophysical surveys evolving from traditional 2D to 3D models, the large volume of data adds challenges to inversion, especially when aiming to resolve complex 3D structures. An iterative forward solver for a controlled-source electromagnetic method (CSEM) requires less memory than that for a direct solver; however, it is not easy to iteratively solve an ill-conditioned linear system of equations arising from finite-element discretization of Maxwell’s equations. To solve this problem, we have developed efficient and robust iterative solvers for frequency- and time-domain CSEM modeling problems. For the frequency-domain problem, we first transform the linear system into its equivalent real-number format, and then introduce an optimal block-diagonal preconditioner. Because the condition number of the preconditioned linear equation system has an upper bound of √2, we can achieve fast solution convergence when applying a flexible generalized minimum residual solver. Applying the block preconditioner further results in solving two smaller linear systems with the same coefficient matrix. For the time-domain problem, we first discretize the partial differential equation for the electric field in time using an unconditionally stable backward Euler scheme. We then solve the resulting linear equation system iteratively at each time step. After the spatial discretization in the frequency domain, or space-time discretization in the time domain, we exploit the conjugate-gradient solver with auxiliary-space preconditioners derived from the Hiptmair-Xu decomposition to solve these real linear systems. Finally, we check the efficiency and effectiveness of our iterative methods by simulating complex CSEM models. The most significant advantage of our approach is that the iterative solvers that we adopt have almost the same accuracy and robustness as direct solvers but require much less memory, rendering them more suitable for large-scale 3D CSEM forward modeling and inversion.

scholarly journals Neural UpFlow

2021 ◽  
Vol 4 (3) ◽  
pp. 1-26
Author(s):  
Bruno Roy ◽  
Pierre Poulin ◽  
Eric Paquette
Keyword(s):  
Neural Networks ◽  
High Resolution ◽  
Large Scale ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Initial Condition ◽  
Inference Model ◽  
Flow Solver ◽  
Flow Estimation ◽  
Lagrangian Advection

We present a novel up-resing technique for generating high-resolution liquids based on scene flow estimation using deep neural networks. Our approach infers and synthesizes small- and large-scale details solely from a low-resolution particle-based liquid simulation. The proposed network leverages neighborhood contributions to encode inherent liquid properties throughout convolutions. We also propose a particle-based approach to interpolate between liquids generated from varying simulation discretizations using a state-of-the-art bidirectional optical flow solver method for fluids in addition with a novel key-event topological alignment constraint. In conjunction with the neighborhood contributions, our loss formulation allows the inference model throughout epochs to reward important differences in regard to significant gaps in simulation discretizations. Even when applied in an untested simulation setup, our approach is able to generate plausible high-resolution details. Using this interpolation approach and the predicted displacements, our approach combines the input liquid properties with the predicted motion to infer semi-Lagrangian advection. We furthermore showcase how the proposed interpolation approach can facilitate generating large simulation datasets with a subset of initial condition parameters.

scholarly journals Documentary data and the study of past droughts: a global state of the art

2018 ◽  
Vol 14 (12) ◽  
pp. 1915-1960 ◽  
Author(s):  
Rudolf Brázdil ◽  
Andrea Kiss ◽  
Jürg Luterbacher ◽  
David J. Nash ◽  
Ladislava Řezníčková
Keyword(s):  
Large Scale ◽  
State Of The Art ◽  
Drought Indices ◽  
Documentary Evidence ◽  
Climatic Trends ◽  
Instrumental Observations ◽  
Spatio Temporal ◽  
Epigraphic Evidence ◽  
Administrative Evidence

Abstract. The use of documentary evidence to investigate past climatic trends and events has become a recognised approach in recent decades. This contribution presents the state of the art in its application to droughts. The range of documentary evidence is very wide, including general annals, chronicles, memoirs and diaries kept by missionaries, travellers and those specifically interested in the weather; records kept by administrators tasked with keeping accounts and other financial and economic records; legal-administrative evidence; religious sources; letters; songs; newspapers and journals; pictographic evidence; chronograms; epigraphic evidence; early instrumental observations; society commentaries; and compilations and books. These are available from many parts of the world. This variety of documentary information is evaluated with respect to the reconstruction of hydroclimatic conditions (precipitation, drought frequency and drought indices). Documentary-based drought reconstructions are then addressed in terms of long-term spatio-temporal fluctuations, major drought events, relationships with external forcing and large-scale climate drivers, socio-economic impacts and human responses. Documentary-based drought series are also considered from the viewpoint of spatio-temporal variability for certain continents, and their employment together with hydroclimate reconstructions from other proxies (in particular tree rings) is discussed. Finally, conclusions are drawn, and challenges for the future use of documentary evidence in the study of droughts are presented.

An Out-of-Core Dataflow Middleware to Reduce the Cost of Large Scale Iterative Solvers

2012 ◽  
Author(s):  
Zheng Zhou ◽  
Erik Saule ◽  
Hasan Metin Aktulga ◽  
Chao Yang ◽  
Esmond G. Ng ◽  
...  
Keyword(s):  
Large Scale ◽  
Iterative Solvers ◽  
The Cost

scholarly journals VIPPrint: Validating Synthetic Image Detection and Source Linking Methods on a Large Scale Dataset of Printed Documents

2021 ◽  
Vol 7 (3) ◽  
pp. 50
Author(s):  
Anselmo Ferreira ◽  
Ehsan Nowroozi ◽  
Mauro Barni
Keyword(s):  
Large Scale ◽  
State Of The Art ◽  
Child Pornography ◽  
Forensic Analysis ◽  
Synthetic Image ◽  
Image Detection ◽  
Face Images ◽  
Large Scale Dataset ◽  
Scanned Images ◽  
Analysis Of The Images

The possibility of carrying out a meaningful forensic analysis on printed and scanned images plays a major role in many applications. First of all, printed documents are often associated with criminal activities, such as terrorist plans, child pornography, and even fake packages. Additionally, printing and scanning can be used to hide the traces of image manipulation or the synthetic nature of images, since the artifacts commonly found in manipulated and synthetic images are gone after the images are printed and scanned. A problem hindering research in this area is the lack of large scale reference datasets to be used for algorithm development and benchmarking. Motivated by this issue, we present a new dataset composed of a large number of synthetic and natural printed face images. To highlight the difficulties associated with the analysis of the images of the dataset, we carried out an extensive set of experiments comparing several printer attribution methods. We also verified that state-of-the-art methods to distinguish natural and synthetic face images fail when applied to print and scanned images. We envision that the availability of the new dataset and the preliminary experiments we carried out will motivate and facilitate further research in this area.

Sign in / Sign up

Export Citation Format

Share Document