TorchEsegeta: Framework for Interpretability and Explainability of Image-based Deep Learning Models

Clinicians are often very sceptical about applying automatic image processing approaches, especially deep learning based methods, in practice. One main reason for this is the black-box nature of these approaches and the inherent problem of missing insights of the automatically derived decisions. In order to increase trust in these methods, this paper presents approaches that help to interpret and explain the results of deep learning algorithms by depicting the anatomical areas which influence the decision of the algorithm most. Moreover, this research presents a unified framework, TorchEsegeta, for applying various interpretability and explainability techniques for deep learning models and generate visual interpretations and explanations for clinicians to corroborate their clinical findings. In addition, this will aid in gaining confidence in such methods. The framework builds on existing interpretability and explainability techniques that are currently focusing on classification models, extending them to segmentation tasks. In addition, these methods have been adapted to 3D models for volumetric analysis. The proposed framework provides methods to quantitatively compare visual explanations using infidelity and sensitivity metrics. This framework can be used by data scientists to perform post-hoc interpretations and explanations of their models, develop more explainable tools and present the findings to clinicians to increase their faith in such models. The proposed framework was evaluated based on a use case scenario of vessel segmentation models trained on Time-of-fight (TOF) Magnetic Resonance Angiogram (MRA) images of the human brain. Quantitative and qualitative results of a comparative study of different models and interpretability methods are presented. Furthermore, this paper provides an extensive overview of several existing interpretability and explainability methods.

Аpplication of artificial intelligence systems in neuroradiology of acute ischemic stroke

Introduction. Artificial intelligence is one of the fastest-growing areas of great importance to radiology. Purpose. In this article, we aimed to study the current state of the use of computer-aided imaging analysis in acute ischemic stroke. Results. There are many artificial intelligence softwares that automatic image processing can successfully identify neuroradiology image in stroke: early detection by diagnostic imaging methods, assessment of the time of disease onset, segmentation of the lesion, analysis of the presence and possibility of cerebral edema, and predicting complications and treatment outcomes. Conclusion. The first results of using artificial intelligence to evaluate neuroimaging data showed that machine-learning methods could be useful as decision-making tools when choosing a treatment for acute ischemic stroke.

Hydrodynamics of a Fast and Highly Exothermic Liquid-Liquid Oxidation Process with in-situ Gas Production in Microreacto

Hydrodynamics characteristics of a fast and highly exothermic liquid-liquid oxidation process with in-situ gas production in microreactors was studied using a newly developed experimental method. In the adipic acid synthesis through the K/A oil oxidation with nitric acid, bubble generation modes were divided into four categories. The gas production became more intensive and unstable, even explosive with increasing the oil phase feed rate and the temperature. A novel automatic image processing method was established to monitor the instantaneous fluid velocity online by tracking the gas-liquid interface. The axial fluid velocity at the same location was unstable with obvious fluctuation due to the unstable gas production rate. Furthermore, the actual average residence time was obtained easily with being only 36% of the space-time minimally, beneficial for establishing accurate kinetic and mass transfer models with time participation. Finally, an empirical correlation was developed to predict the actual residence time under different conditions.

Variation of Heartwood Proportion and Wood Colour from Fast Grown 5-Year-Old Teak

The heartwood percentage and wood colour of fast plantation grown teak destined for harvest at 5 years of age were characterized using automatic image processing ’ImageJ’ routines and CieLab’s colour system with the following coefficients: L for lightness, a* for redness and b* for yellowness. Analyses were conducted on material from different dry and wet sites. Comparison with 6-year old plantation from a dry site was conducted to study differences arising in older trees. Analyses of variation of those properties between and within different tree diameter classes were also conducted. The results showed that brightness, redness and yellowness values of 5-year old teak trees were 60.7, 10.7 and 23.1, respectively. Tree clone had a more dominant effect on wood colour and heartwood proportion than site, thus if specific colour preferences are needed of plantation trees, clone selection is important. The drier site produced larger proportions of heartwood in trees, as well as a more attractive figure. The trees produced heartwood proportions of 20% and 14% from the dry and wet sites respectively. On average, these 5 year old teak trees already produced 18% heartwood. Faster tree growth (larger diameter) appeared to have produced significantly larger heartwood proportions. Radially, the palest colour (the highest L but the lowest a*b* parameters) occurred in an area between heartwood and sapwood indicating the presence of a transition zone in all the tree samples.

Computer Vision and Deep Learning Techniques for the Analysis of Drone-Acquired Forest Images, a Transfer Learning Study

Unmanned Aerial Vehicles (UAV) are becoming an essential tool for evaluating the status and the changes in forest ecosystems. This is especially important in Japan due to the sheer magnitude and complexity of the forest area, made up mostly of natural mixed broadleaf deciduous forests. Additionally, Deep Learning (DL) is becoming more popular for forestry applications because it allows for the inclusion of expert human knowledge into the automatic image processing pipeline. In this paper we study and quantify issues related to the use of DL with our own UAV-acquired images in forestry applications such as: the effect of Transfer Learning (TL) and the Deep Learning architecture chosen or whether a simple patch-based framework may produce results in different practical problems. We use two different Deep Learning architectures (ResNet50 and UNet), two in-house datasets (winter and coastal forest) and focus on two separate problem formalizations (Multi-Label Patch or MLP classification and semantic segmentation). Our results show that Transfer Learning is necessary to obtain satisfactory outcome in the problem of MLP classification of deciduous vs evergreen trees in the winter orthomosaic dataset (with a 9.78% improvement from no transfer learning to transfer learning from a a general-purpose dataset). We also observe a further 2.7% improvement when Transfer Learning is performed from a dataset that is closer to our type of images. Finally, we demonstrate the applicability of the patch-based framework with the ResNet50 architecture in a different and complex example: Detection of the invasive broadleaf deciduous black locust (Robinia pseudoacacia) in an evergreen coniferous black pine (Pinus thunbergii) coastal forest typical of Japan. In this case we detect images containing the invasive species with a 75% of True Positives (TP) and 9% False Positives (FP) while the detection of native trees was 95% TP and 10% FP.

Application of a wavelet transform for multispectral image fusion

A method of image fusion based on wavelet decomposition of the original images is considered. An integrated monochrome image is formed from images of the same scene obtained in different spectral ranges. A strategy for fusing detail coefficients by comparing the proportions of their magnitudes for all original images is proposed. The fusion procedure does not require any thresholds. This fusion procedure can be performed for any number of original images. The algorithm does not introduce additional distortions and collects all the necessary information from the original images. The quantitative and qualitative evaluation of the results was performed. The proposed algorithm can be used in optoelectronic systems for automatic image processing.

Convolutional Neural Networks–Based Image Analysis for the Detection and Quantification of Neutrophil Extracellular Traps

Over a decade ago, the formation of neutrophil extracellular traps (NETs) was described as a novel mechanism employed by neutrophils to tackle infections. Currently applied methods for NETs release quantification are often limited by the use of unspecific dyes and technical difficulties. Therefore, we aimed to develop a fully automatic image processing method for the detection and quantification of NETs based on live imaging with the use of DNA-staining dyes. For this purpose, we adopted a recently proposed Convolutional Neural Network (CNN) model called Mask R-CNN. The adopted model detected objects with quality comparable to manual counting—Over 90% of detected cells were classified in the same manner as in manual labelling. Furthermore, the inhibitory effect of GW 311616A (neutrophil elastase inhibitor) on NETs release, observed microscopically, was confirmed with the use of the CNN model but not by extracellular DNA release measurement. We have demonstrated that a modern CNN model outperforms a widely used quantification method based on the measurement of DNA release and can be a valuable tool to quantitate the formation process of NETs.