scholarly journals Sensitivity analysis for interpretation of machine learning based segmentation models in cardiac MRI

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Markus J. Ankenbrand ◽  
Liliia Shainberg ◽  
Michael Hock ◽  
David Lohr ◽  
Laura M. Schreiber
Keyword(s):  
Neural Networks ◽  
Sensitivity Analysis ◽  
Cardiac Magnetic Resonance ◽  
Magnetic Resonance ◽  
Cardiac Mri ◽  
Magnetic Resonance Images ◽  
Model Interpretation ◽  
First Case ◽  
Segmentation Models

Abstract Background Image segmentation is a common task in medical imaging e.g., for volumetry analysis in cardiac MRI. Artificial neural networks are used to automate this task with performance similar to manual operators. However, this performance is only achieved in the narrow tasks networks are trained on. Performance drops dramatically when data characteristics differ from the training set properties. Moreover, neural networks are commonly considered black boxes, because it is hard to understand how they make decisions and why they fail. Therefore, it is also hard to predict whether they will generalize and work well with new data. Here we present a generic method for segmentation model interpretation. Sensitivity analysis is an approach where model input is modified in a controlled manner and the effect of these modifications on the model output is evaluated. This method yields insights into the sensitivity of the model to these alterations and therefore to the importance of certain features on segmentation performance. Results We present an open-source Python library (misas), that facilitates the use of sensitivity analysis with arbitrary data and models. We show that this method is a suitable approach to answer practical questions regarding use and functionality of segmentation models. We demonstrate this in two case studies on cardiac magnetic resonance imaging. The first case study explores the suitability of a published network for use on a public dataset the network has not been trained on. The second case study demonstrates how sensitivity analysis can be used to evaluate the robustness of a newly trained model. Conclusions Sensitivity analysis is a useful tool for deep learning developers as well as users such as clinicians. It extends their toolbox, enabling and improving interpretability of segmentation models. Enhancing our understanding of neural networks through sensitivity analysis also assists in decision making. Although demonstrated only on cardiac magnetic resonance images this approach and software are much more broadly applicable.

scholarly journals Sensitivity Analysis for Interpretation of Machine Learning Based Segmentation Models in Cardiac MRI

2020 ◽  
Author(s):  
Markus J Ankenbrand ◽  
Liliia Shainberg ◽  
Michael Hock ◽  
David Lohr ◽  
Laura M Schreiber
Keyword(s):  
Neural Networks ◽  
Sensitivity Analysis ◽  
Cardiac Magnetic Resonance ◽  
Magnetic Resonance ◽  
Cardiac Mri ◽  
Magnetic Resonance Images ◽  
Training Set ◽  
Black Boxes ◽  
Common Task ◽  
Segmentation Models

Abstract Background Image segmentation is a common task in medical imaging e.g. for volumetry analysis in cardiac MRI. Artificial neural networks are used to automate this task with performance similar to manual operators. However, this performance is only achieved in the narrow tasks networks are trained on. Performance drops dramatically when data characteristics differ from the training set properties. Moreover, neural networks are commonly considered black boxes, because it is hard to understand how they make decisions and why they fail. Therefore, it is also hard to predict whether they will generalize and work well with new data. Results We show that sensitivity analysis is a suitable approach to answer practical questions regarding use and functionality of segmentation models. We also provide an open source Python library (misas), that facilitates the use of this method with arbitrary data and models. By enabling a better understanding of neural networks through sensitivity analysis it also assists in decision making. We demonstrate this in two case studies on cardiac magnetic resonance imaging.Conclusions Sensitivity analysis is a useful tool for deep learning developers as well as users such as clinicians. It extends their toolbox with a new tool that makes segmentation models more interpretable. Although demonstrated only on cardiac magnetic resonance images this approach and software are much more broadly applicable.

Semi-automated scar detection in delayed enhanced cardiac magnetic resonance images

2015 ◽  
Vol 26 (01) ◽  
pp. 1550011
Author(s):  
Rita Morisi ◽  
Bruno Donini ◽  
Nico Lanconelli ◽  
James Rosengarden ◽  
John Morgan ◽  
...  
Keyword(s):  
Clinical Practice ◽  
Cardiac Magnetic Resonance ◽  
Magnetic Resonance ◽  
Cardiac Mri ◽  
Magnetic Resonance Images ◽  
University Hospital ◽  
Support Vector ◽  
K Nearest Neighbor ◽  
Feed Forward Neural Network ◽  
Automated Method

Late enhancement cardiac magnetic resonance images (MRI) has the ability to precisely delineate myocardial scars. We present a semi-automated method for detecting scars in cardiac MRI. This model has the potential to improve routine clinical practice since quantification is not currently offered due to time constraints. A first segmentation step was developed for extracting the target regions for potential scar and determining pre-candidate objects. Pattern recognition methods are then applied to the segmented images in order to detect the position of the myocardial scar. The database of late gadolinium enhancement (LE) cardiac MR images consists of 111 blocks of images acquired from 63 patients at the University Hospital Southampton NHS Foundation Trust (UK). At least one scar was present for each patient, and all the scars were manually annotated by an expert. A group of images (around one third of the entire set) was used for training the system which was subsequently tested on all the remaining images. Four different classifiers were trained (Support Vector Machine (SVM), k-nearest neighbor (KNN), Bayesian and feed-forward neural network) and their performance was evaluated by using Free response Receiver Operating Characteristic (FROC) analysis. Feature selection was implemented for analyzing the importance of the various features. The segmentation method proposed allowed the region affected by the scar to be extracted correctly in 96% of the blocks of images. The SVM was shown to be the best classifier for our task, and our system reached an overall sensitivity of 80% with less than 7 false positives per patient. The method we present provides an effective tool for detection of scars on cardiac MRI. This may be of value in clinical practice by permitting routine reporting of scar quantification.

Segmentation of Cardiac Magnetic Resonance Images

2009 ◽  
pp. 108-125
Author(s):  
Dimitrios G. Tsalikakis ◽  
Petros S. Karvelis ◽  
Dimitrios I. Fotiadis
Keyword(s):  
Magnetic Resonance Imaging ◽  
Cardiac Magnetic Resonance ◽  
Magnetic Resonance ◽  
Cardiac Magnetic Resonance Imaging ◽  
Cardiac Mri ◽  
Crucial Role ◽  
Magnetic Resonance Images ◽  
Regions Of Interest ◽  
Mri Segmentation ◽  
Segmentation Methods

Segmentation plays a crucial role in cardiac magnetic resonance imaging (CMRI) applications, since it permits automated detection of regions of interest. In this chapter we review semi-automated and fully automated cardiac MRI segmentation techniques and discuss their advantages. We classify those segmentation methods as classical and model-based.

Myocardial segmentation in cardiac magnetic resonance images using fully convolutional neural networks

2018 ◽  
Vol 44 ◽  
pp. 48-57 ◽  
Author(s):  
Liset Vázquez Romaguera ◽  
Francisco Perdigón Romero ◽  
Cicero Ferreira Fernandes Costa Filho ◽  
Marly Guimarães Fernandes Costa
Keyword(s):  
Neural Networks ◽  
Cardiac Magnetic Resonance ◽  
Magnetic Resonance ◽  
Convolutional Neural Networks ◽  
Magnetic Resonance Images ◽  
Fully Convolutional Neural Networks ◽  
Myocardial Segmentation

Abstract 16189: Cardiovascular Risk Prediction Using Fully Automated Artificial Intelligence Algorithms for the Assessment of Right Ventricular Function From Cardiac Magnetic Resonance Images

Circulation ◽  
2020 ◽  
Vol 142 (Suppl_3) ◽  
Author(s):  
shuo wang ◽  
Hena Patel ◽  
Tamari Miller ◽  
Keith Ameyaw ◽  
Akhil Narang ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Cardiac Magnetic Resonance ◽  
Magnetic Resonance ◽  
Right Ventricular ◽  
Roc Analysis ◽  
Right Ventricular Ejection Fraction ◽  
Magnetic Resonance Images ◽  
Major Adverse Cardiovascular Events ◽  
Ventricular Ejection Fraction ◽  
Rv Function

Background: It is unclear whether artificial intelligence (AI) can provide automatic solutions to measure right ventricular ejection fraction (RVEF), due to the complex RV geometry. Although several deep learning (DL) algorithms are available to quantify RVEF from cardiac magnetic resonance (CMR) images, there has been no systematic comparison of these algorithms, and the prognostic value of these automated measurements is unknown. We aimed to determine whether RVEF measurements made using DL algorithms could be used to risk stratify patients similarly to measurements made by an expert. Methods: We identified from a pre-existing registry 200 patients who underwent CMR. RVEF was determined using 3 fully automated commercial DL algorithms (DL-RVEF) and also by a clinical expert (CLIN-RVEF) using conventional methodology. Each of the DL-RVEF approaches was compared against CLIN-RVEF using linear regression and Bland-Altman analyses. In addition, RVEF values were classified according to clinically important cutoffs: <35%, 35-50%, ≥50%, and rates of disagreement with the reference classification were determined. ROC analysis was performed to evaluate the ability of CLIN-RVEF and each of the DL-RVEF based classifications to predict major adverse cardiovascular events (MACE). Results: The CLIN-RVEF and the three DL-RVEFs were obtained in all patients. We found only modest correlations between DL-RVEF and CLIN-RVEF (figure). The DL-RVEF algorithms had accuracy ranging from 0.59 to 0.78 for categorizing RV function. Nevertheless, ROC analysis showed no significant differences between the 4 approaches in predicting MACE, as reflected by respective AUC values of 0.68, 0.69, 0.64 and 0.63. Conclusions: Although the automated algorithms predicted patient outcomes as well as the CLIN-RVEF, the agreement between DL-RVEF and the clinical expert’s measurements was not optimal. DL approaches need further refinements to improve automated assessment of RV function.

Sign in / Sign up

Export Citation Format

Share Document