Assessment of reinforcement learning applications for industrial control based on complexity measures

Machine learning and particularly reinforcement learning methods may be applied to control tasks ranging from single control loops to the operation of whole production plants. However, their utilization in industrial contexts lacks understandability and requires suitable levels of operability and maintainability. In order to asses different application scenarios a simple measure for their complexity is proposed and evaluated on four examples in a simulated palette transport system of a cold rolling mill. The measure is based on the size of controller input and output space determined by different granularity levels in a hierarchical process control model. The impact of these decomposition strategies on system characteristics, especially operability and maintainability, are discussed, assuming solvability and a suitable quality of the reinforcement learning solution is provided.

TSInsight: A Local-global Attribution Framework for Interpretability in Time Series Data

With the rise in the employment of deep learning methods in safety-critical scenarios, interpretability is more essential than ever before. Although many different directions regarding interpretability have been explored for visual modalities, time series data has been neglected, with only a handful of methods tested due to their poor intelligibility. We approach the problem of interpretability in a novel way by proposing TSInsight, where we attach an auto-encoder to the classifier with a sparsity-inducing norm on its output and fine-tune it based on the gradients from the classifier and a reconstruction penalty. TSInsight learns to preserve features that are important for prediction by the classifier and suppresses those that are irrelevant, i.e., serves as a feature attribution method to boost the interpretability. In contrast to most other attribution frameworks, TSInsight is capable of generating both instance-based and model-based explanations. We evaluated TSInsight along with nine other commonly used attribution methods on eight different time series datasets to validate its efficacy. The evaluation results show that TSInsight naturally achieves output space contraction; therefore, it is an effective tool for the interpretability of deep time series models.

Output Space Entropy Search Framework for Multi-Objective Bayesian Optimization

We consider the problem of black-box multi-objective optimization (MOO) using expensive function evaluations (also referred to as experiments), where the goal is to approximate the true Pareto set of solutions by minimizing the total resource cost of experiments. For example, in hardware design optimization, we need to find the designs that trade-off performance, energy, and area overhead using expensive computational simulations. The key challenge is to select the sequence of experiments to uncover high-quality solutions using minimal resources. In this paper, we propose a general framework for solving MOO problems based on the principle of output space entropy (OSE) search: select the experiment that maximizes the information gained per unit resource cost about the true Pareto front. We appropriately instantiate the principle of OSE search to derive efficient algorithms for the following four MOO problem settings: 1) The most basic single-fidelity setting, where experiments are expensive and accurate; 2) Handling black-box constraints which cannot be evaluated without performing experiments; 3) The discrete multi-fidelity setting, where experiments can vary in the amount of resources consumed and their evaluation accuracy; and 4) The continuous-fidelity setting, where continuous function approximations result in a huge space of experiments. Experiments on diverse synthetic and real-world benchmarks show that our OSE search based algorithms improve over state-of-the-art methods in terms of both computational-efficiency and accuracy of MOO solutions.

Experimental force reconstruction on plates of arbitrary shape using neural networks

Measuring the forces that excite a structure into vibration is an important tool in modeling the system and investigating ways to reduce the vibration. However, determining the forces that have been applied to a vibrating structure can be a challenging inverse problem, even when the structure is instrumented with a large number of sensors. Previously, an artificial neural network was developed to identify the location of an impulsive force on a rectangular plate. In this research, the techniques were extended to plates of arbitrary shape. The principal challenge of arbitrary shapes is that some combinations of network outputs (x- and y-coordinates) are invalid. For example, for a plate with a hole in the middle, the network should not output that the force was applied in the center of the hole. Different methods of accommodating arbitrary shapes were investigated, including output space quantization and selecting the closest valid region.

A Family of Quaternion-valued Pipelined Second-order Volterra Adaptive Filters for Nonlinear System Identification

This paper primarily proposes a family of quaternion Volterra filters based on the feedforward pipelined structure (QPSOVAFs) for nonlinear quaternion system identification to reduce the computational complexity. Then, the strictly nonlinear QPSOVAF (SNL-QPSOVAF), semi-widely nonlinear QPSOVAF(SWNL-QPSOVAF), and widely nonlinear QPSOVAF (WNL-QPSOVAF), are proposed. This architecture consists of several quaternion-valued second-order Volterra (SOV) modules. The structure's nonlinear subsection executes a nonlinear mapping from the input space to an intermediate space using the feedforward SOV; the linear combiner subsection performs a linear mapping from the intermediate space to the output space. Moreover, the theoretical analysis expresses the effectiveness of the proposed QPSOVAFs in a specific condition. Finally, simulation results further prove that the proposed QPSOVAFs have good performance in identifying the quaternion-valued nonlinear system.

Learning Neural Representations and Local Embedding for Nonlinear Dimensionality Reduction Mapping

This work explores neural approximation for nonlinear dimensionality reduction mapping based on internal representations of graph-organized regular data supports. Given training observations are assumed as a sample from a high-dimensional space with an embedding low-dimensional manifold. An approximating function consisting of adaptable built-in parameters is optimized subject to given training observations by the proposed learning process, and verified for transformation of novel testing observations to images in the low-dimensional output space. Optimized internal representations sketch graph-organized supports of distributed data clusters and their representative images in the output space. On the basis, the approximating function is able to operate for testing without reserving original massive training observations. The neural approximating model contains multiple modules. Each activates a non-zero output for mapping in response to an input inside its correspondent local support. Graph-organized data supports have lateral interconnections for representing neighboring relations, inferring the minimal path between centroids of any two data supports, and proposing distance constraints for mapping all centroids to images in the output space. Following the distance-preserving principle, this work proposes Levenberg-Marquardt learning for optimizing images of centroids in the output space subject to given distance constraints, and further develops local embedding constraints for mapping during execution phase. Numerical simulations show the proposed neural approximation effective and reliable for nonlinear dimensionality reduction mapping.

A Hyper-Ellipsoid Approach for Inverse Lack-of-Knowledge Uncertainty Quantification

This paper presents a novel methodology to solve an inverse uncertainty quantification problem where only the variation of the system response is provided by a small set of experimental data. Furthermore, the method is extended for cases where the uncertainty of the response quantities is given by an incomplete set of statistical moments. For both cases, the uncertainty on the output space is represented by a minimum volume enclosing ellipsoid (MVEE). The actual inverse uncertainty quantification is conducted by identifying also a hyper-ellipsoid for the input parameters, which has an image on the output space that matches the MVEE as close as possible. Hence, the newly introduced approach is a contribution to the field of nonprobabilistic uncertainty quantification methods. Compared to literature, the new approach has often superior accuracy and especially an improved efficiency for high-dimensional problems. The method is validated first by an analytical test case and subsequently applied to a jet engine performance model, where this type of inverse uncertainty quantification has to be solved to allow for a consistent and integrated solution procedure. In both cases, the results are compared with an inverse method where the variability on the input side is quantified by a multidimensional interval. It can be shown that the hyper-ellipsoid approach is superior with respect to the computation time in high-dimensional problems encountered not only in jet engine design.

Echocardiographic phenotyping of the continuum of myocardial functional remodelling in left ventricular hypertrophy - machine learning validated with cardiac magnetic resonance

Funding Acknowledgements Type of funding sources: Public grant(s) – EU funding. Main funding source(s): Horizon 2020 European Commission Project H2020-MSCA-ITN-2016 (764738) and the Clinical Research in Cardiology grant from the Spanish Cardiac Society Background Exploring phenotypes of left ventricular hypertrophy (LVH) and interpreting the relationship of genotype and phenotype are contemporary clinical challenges. Machine learning (ML) can help by integrating whole-cardiac cycle echo data and separating patients based on subtle differences of cardiac function. The aim is to investigate if an unsupervised ML approach has the potential to explore the LVH spectrum and recognize phenotypes related to distinct disease aetiologies and genotypes. Methods The cohort consisted of 342 participants: patients with hypertrophic cardiomyopathy (HCM)(n = 27), HCM relatives (n = 31), hypertensive patients (HTN) (n = 189), and healthy individuals (n = 95). All had echocardiography performed, whereas magnetic resonance (MR) and genetic testing were performed when clinically indicated. Myocardial deformation of the LV and left atrium, aortic and mitral blood-pool Doppler, as well as the septal mitral annular tissue Doppler velocity profiles were used as input for ML. Clinical data, including echo measurements, were not part of the learning, but used to validate the ML-derived phenotypes. An unsupervised ML algorithm was used to create an output space where participants were positioned based on cardiac function. Regression was used to estimate the echo and clinical characteristics of different regions in the space. Results The ML analysis of HCM and relative data shows grouping of HCM patients in the right-most region of the output space (Fig 1B). This region was related to LV outflow tract obstruction, mitral inflow fusion, systolic impairment with septal involvement, as well as LA and LV strain impairment (Fig 1A). Clinical data concurred - showing reduced global longitudinal strain, elevated LV mass, and a pattern of systolic and diastolic impairment - defining a comprehensive phenotype of LV remodelling related to HCM. Exploration of the genotype/phenotype relationship revealed G + P- relatives grouping on the transition from the healthy to the remodelling region. Projection of the HTN and healthy individuals into the HCM space defined the LVH disease spectrum, with healthy individuals projecting in the existing healthy region and HTNs in the transition from health to extreme remodelling (Fig 1C). MR findings of late gadolinium enhancement correlated with the ML-derived functional remodelling phenotype (Fig 1C). Furthermore, 6 patients with a clinical need for septal myectomy were located in the extreme remodelling part of the output space (Fig 1C, red circles). Conclusion ML can integrate complex, whole-cardiac cycle echo data to group patients based on similarity of cardiac function. Using an interpretable ML approach, we can explore the spectrum of LV remodelling in different aetiologies and interpret the relationship between genotype and phenotype. The methodology can accommodate new patients by projecting them into the existing space to aid in clinical interpretation, risk assessment and patient management. Abstract Figure 1