Hybrid Gaussian Process Models for continuous time series in bolus fed-batch cultures

Hybrid modeling, meaning the integration of data-driven and knowledge-based methods, is quickly gaining popularity among many research fields, including bioprocess engineering and development. Recently, the data-driven part of hybrid methods have been largely extended with machine learning algorithms (e.g., artificial neural network, support vector regression), while the mechanistic part is typically using differential equations to describe the dynamics of the process based on its current state. In this work we present an alternative hybrid model formulation that merges the advantages of Gaussian Process State Space Models and the numerical approximation of differential equation systems through full discretization. The use of Gaussian Process Models to describe complex bioprocesses in batch, fed-batch, has been reported in several applications. Nevertheless, handling the dynamics of the states of the system, known to have a continuous time-dependent evolution governed by implicit dynamics, has proven to be a major challenge. Discretization of the process on the sampling steps is a source of several complications, as are: 1) not being able to handle multi-rate date sets, 2) the step-size of the derivative approximation is defined by the sampling frequency, and 3) a high sensitivity to sampling and addition errors. We present a coupling of polynomial regression with Gaussian Process Models as representation of the right-hand side of the ordinary differential equation system and demonstrate the advantages in a typical fed-batch cultivation for monoclonal antibody production.

Machine Learning-Enabled Prediction of Transient Injection Map In Automotive Injectors With Uncertainty Quantification

Accurate prediction of injection profiles is a critical aspect of linking injector operation with engine performance and emissions. However, highly resolved injector simulations can take one to two weeks of wall-clock time, which is incompatible with engine design cycles with desired turnaround times of less than a day. Hence, it is important to reduce the time-to-solution of the internal flow simulations by several orders of magnitude to make it compatible with engine simulations. This work demonstrates a data-driven approach for tackling the computational overhead of injector simulations, whereby the transient injection profiles are emulated for a side-oriented, single-hole diesel injector using a Bayesian machine-learning framework. First, an interpretable Bayesian learning strategy was employed to understand the effect of design parameters on the total void fraction field. Then, autoencoders are utilized for efficient dimensionality reduction of the flowfields. Gaussian process models are finally used to predict the spatio-temporal void fraction field at the injector exit for unknown operating conditions. The Gaussian process models produce principled uncertainty estimates associated with the emulated flowfields, which provide the engine designer with valuable information of where the data-driven predictions can be trusted in the design space. The Bayesian flowfield predictions are compared with the corresponding predictions from a deep neural network, which has been transfer-learned from static needle simulations from a previous work by the authors. The emulation framework can predict the void fraction field at the exit of the orifice within a few seconds, thus achieving a speed-up factor of up to 38 million over the traditional simulation-based approach of generating transient injection maps.

Alignment of multiple metabolomics LC-MS datasets from disparate diseases to reveal fever-associated metabolites

Acute febrile illnesses are still a major cause of mortality and morbidity globally, particularly in low to middle income countries. The aim of this study was to determine any possible metabolic commonalities of patients infected with disparate pathogens that cause fever. Three liquid chromatography-mass spectrometry (LC-MS) datasets investigating the metabolic effects of malaria, leishmaniasis and Zika virus infection were used. The retention time (RT) drift between the datasets was determined using landmarks obtained from the standard reference mixtures generally used in the quality control of the LC-MS experiments. We used fitted Gaussian Process models (GPs) to perform a high level correction of the RT drift between the experiments, followed by standard peakset alignment between the samples with corrected RTs of the three LC-MS datasets. Statistical analysis, annotation and pathway analysis of the integrated peaksets were subsequently performed. Metabolic dysregulation patterns common across the datasets were identified, with kynurenine pathway being the most affected pathway between all three fever-associated datasets.

Deterministic global optimization with Gaussian processes embedded

Gaussian processes (Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in optimization problems. These optimization problems are nonconvex and global optimization is desired. However, previous literature observed computational burdens limiting deterministic global optimization to Gaussian processes trained on few data points. We propose a reduced-space formulation for deterministic global optimization with trained Gaussian processes embedded. For optimization, the branch-and-bound solver branches only on the free variables and McCormick relaxations are propagated through explicit Gaussian process models. The approach also leads to significantly smaller and computationally cheaper subproblems for lower and upper bounding. To further accelerate convergence, we derive envelopes of common covariance functions for GPs and tight relaxations of acquisition functions used in Bayesian optimization including expected improvement, probability of improvement, and lower confidence bound. In total, we reduce computational time by orders of magnitude compared to state-of-the-art methods, thus overcoming previous computational burdens. We demonstrate the performance and scaling of the proposed method and apply it to Bayesian optimization with global optimization of the acquisition function and chance-constrained programming. The Gaussian process models, acquisition functions, and training scripts are available open-source within the “MeLOn—MachineLearning Models for Optimization” toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).