Explainable autoencoder-based representation learning for gene expression data

Modern machine learning methods have been extensively utilized in gene expression data analysis. In particular, autoencoders (AE) have been employed in processing noisy and heterogenous RNA-Seq data. However, AEs usually lead to "black-box" hidden variables difficult to interpret, hindering downstream experimental validation and clinical translation. To bridge the gap between complicated models and biological interpretations, we developed a tool, XAE4Exp (eXplainable AutoEncoder for Expression data), which integrates AE and SHapley Additive exPlanations (SHAP), a flagship technique in the field of eXplainable AI (XAI). It quantitatively evaluates the contributions of each gene to the hidden structure learned by an AE, substantially improving the expandability of AE outcomes. By applying XAE4Exp to The Cancer Genome Atlas (TCGA) breast cancer gene expression data, we identified genes that are not differentially expressed, and pathways in various cancer-related classes. This tool will enable researchers and practitioners to analyze high-dimensional expression data intuitively, paving the way towards broader uses of deep learning.

POINT AND INTERVAL FORECASTS OF DEATH RATES USING NEURAL NETWORKS

The Lee–Carter model has become a benchmark in stochastic mortality modeling. However, its forecasting performance can be significantly improved upon by modern machine learning techniques. We propose a convolutional neural network (NN) architecture for mortality rate forecasting, empirically compare this model as well as other NN models to the Lee–Carter model and find that lower forecast errors are achievable for many countries in the Human Mortality Database. We provide details on the errors and forecasts of our model to make it more understandable and, thus, more trustworthy. As NN by default only yield point estimates, previous works applying them to mortality modeling have not investigated prediction uncertainty. We address this gap in the literature by implementing a bootstrapping-based technique and demonstrate that it yields highly reliable prediction intervals for our NN model.

Non-intrusive nonlinear model reduction via machine learning approximations to low-dimensional operators

Although projection-based reduced-order models (ROMs) for parameterized nonlinear dynamical systems have demonstrated exciting results across a range of applications, their broad adoption has been limited by their intrusivity: implementing such a reduced-order model typically requires significant modifications to the underlying simulation code. To address this, we propose a method that enables traditionally intrusive reduced-order models to be accurately approximated in a non-intrusive manner. Specifically, the approach approximates the low-dimensional operators associated with projection-based reduced-order models (ROMs) using modern machine-learning regression techniques. The only requirement of the simulation code is the ability to export the velocity given the state and parameters; this functionality is used to train the approximated low-dimensional operators. In addition to enabling nonintrusivity, we demonstrate that the approach also leads to very low computational complexity, achieving up to $$10^3{\times }$$ 10 3 × in run time. We demonstrate the effectiveness of the proposed technique on two types of PDEs. The domain of applications include both parabolic and hyperbolic PDEs, regardless of the dimension of full-order models (FOMs).

Key Concepts in AI Safety: Specification in Machine Learning

This paper is the fourth installment in a series on “AI safety,” an area of machine learning research that aims to identify causes of unintended behavior in machine learning systems and develop tools to ensure these systems work safely and reliably. The first paper in the series, “Key Concepts in AI Safety: An Overview,” outlined three categories of AI safety issues—problems of robustness, assurance, and specification—and the subsequent two papers described problems of robustness and assurance, respectively. This paper introduces specification as a key element in designing modern machine learning systems that operate as intended.

Learning to Disentangle the Complex Causes of Data

<p>The ability to extract and model the meaning in data has been key to the success of modern machine learning. Typically, data reflects a combination of multiple sources that are mixed together. For example, photographs of people’s faces reflect the subject of the photograph, lighting conditions, angle, and background scene. It is therefore natural to wish to extract these multiple, largely independent, sources, which is known as disentangling in the literature. Additional benefits of disentangling arise from the fact that the data is then simpler, meaning that there are fewer free parameters, which reduces the curse of dimensionality and aids learning. While there has been a lot of research into finding disentangled representations, it remains an open problem. This thesis considers a number of approaches to a particularly difficult version of this task: we wish to disentangle the complex causes of data in an entirely unsupervised setting. That is, given access only to unlabeled, entangled data, we search for algorithms that can identify the generative factors of that data, which we call causes. Further, we assume that causes can themselves be complex and require a high-dimensional representation. We consider three approaches to this challenge: as an inference problem, as an extension of independent components analysis, and as a learning problem. Each method is motivated, described, and tested on a set of datasets build from entangled combinations of images, most commonly MNIST digits. Where the results fall short of disentangling, the reasons for this are dissected and analysed. The last method that we describe, which is based on combinations of autoencoders that learn to predict each other’s output, shows some promise on this extremely challenging problem.</p>

Learning to Disentangle the Complex Causes of Data

<p>The ability to extract and model the meaning in data has been key to the success of modern machine learning. Typically, data reflects a combination of multiple sources that are mixed together. For example, photographs of people’s faces reflect the subject of the photograph, lighting conditions, angle, and background scene. It is therefore natural to wish to extract these multiple, largely independent, sources, which is known as disentangling in the literature. Additional benefits of disentangling arise from the fact that the data is then simpler, meaning that there are fewer free parameters, which reduces the curse of dimensionality and aids learning. While there has been a lot of research into finding disentangled representations, it remains an open problem. This thesis considers a number of approaches to a particularly difficult version of this task: we wish to disentangle the complex causes of data in an entirely unsupervised setting. That is, given access only to unlabeled, entangled data, we search for algorithms that can identify the generative factors of that data, which we call causes. Further, we assume that causes can themselves be complex and require a high-dimensional representation. We consider three approaches to this challenge: as an inference problem, as an extension of independent components analysis, and as a learning problem. Each method is motivated, described, and tested on a set of datasets build from entangled combinations of images, most commonly MNIST digits. Where the results fall short of disentangling, the reasons for this are dissected and analysed. The last method that we describe, which is based on combinations of autoencoders that learn to predict each other’s output, shows some promise on this extremely challenging problem.</p>

Identification of Neural Network Model of Robot to Solve the Optimal Control Problem

To calculate the optimal control, a satisfactory mathematical model of the control object is required. Further, when implementing the calculated controls on a real object, the same model can be used in robot navigation to predict its position and correct sensor data, therefore, it is important that the model adequately reflects the dynamics of the object. Model derivation is often time-consuming and sometimes even impossible using traditional methods. In view of the increasing diversity and extremely complex nature of control objects, including the variety of modern robotic systems, the identification problem is becoming increasingly important, which allows you to build a mathematical model of the control object, having input and output data about the system. The identification of a nonlinear system is of particular interest, since most real systems have nonlinear dynamics. And if earlier the identification of the system model consisted in the selection of the optimal parameters for the selected structure, then the emergence of modern machine learning methods opens up broader prospects and allows you to automate the identification process itself. In this paper, a wheeled robot with a differential drive in the Gazebo simulation environment, which is currently the most popular software package for the development and simulation of robotic systems, is considered as a control object. The mathematical model of the robot is unknown in advance. The main problem is that the existing mathematical models do not correspond to the real dynamics of the robot in the simulator. The paper considers the solution to the problem of identifying a mathematical model of a control object using machine learning technique of the neural networks. A new mixed approach is proposed. It is based on the use of well-known simple models of the object and identification of unaccounted dynamic properties of the object using a neural network based on a training sample. To generate training data, a software package was written that automates the collection process using two ROS nodes. To train the neural network, the PyTorch framework was used and an open source software package was created. Further, the identified object model is used to calculate the optimal control. The results of the computational experiment demonstrate the adequacy and performance of the resulting model. The presented approach based on a combination of a well-known mathematical model and an additional identified neural network model allows using the advantages of the accumulated physical apparatus and increasing its efficiency and accuracy through the use of modern machine learning tools.

A Multi-Component Framework for the Analysis and Design of Explainable Artificial Intelligence

The rapid growth of research in explainable artificial intelligence (XAI) follows on two substantial developments. First, the enormous application success of modern machine learning methods, especially deep and reinforcement learning, have created high expectations for industrial, commercial, and social value. Second, the emerging and growing concern for creating ethical and trusted AI systems, including compliance with regulatory principles to ensure transparency and trust. These two threads have created a kind of “perfect storm” of research activity, all motivated to create and deliver any set of tools and techniques to address the XAI demand. As some surveys of current XAI suggest, there is yet to appear a principled framework that respects the literature of explainability in the history of science and which provides a basis for the development of a framework for transparent XAI. We identify four foundational components, including the requirements for (1) explicit explanation knowledge representation, (2) delivery of alternative explanations, (3) adjusting explanations based on knowledge of the explainee, and (4) exploiting the advantage of interactive explanation. With those four components in mind, we intend to provide a strategic inventory of XAI requirements, demonstrate their connection to a basic history of XAI ideas, and then synthesize those ideas into a simple framework that can guide the design of AI systems that require XAI.