Automated Ventricular System Segmentation in Paediatric Patients Treated for Hydrocephalus Using Deep Learning Methods

Hydrocephalus is a common neurological condition that can have traumatic ramifications and can be lethal without treatment. Nowadays, during therapy radiologists have to spend a vast amount of time assessing the volume of cerebrospinal fluid (CSF) by manual segmentation on Computed Tomography (CT) images. Further, some of the segmentations are prone to radiologist bias and high intraobserver variability. To improve this, researchers are exploring methods to automate the process, which would enable faster and more unbiased results. In this study, we propose the application of U-Net convolutional neural network in order to automatically segment CT brain scans for location of CSF. U-Net is a neural network that has proven to be successful for various interdisciplinary segmentation tasks. We optimised training using state of the art methods, including “1cycle” learning rate policy, transfer learning, generalized dice loss function, mixed float precision, self-attention, and data augmentation. Even though the study was performed using a limited amount of data (80 CT images), our experiment has shown near human-level performance. We managed to achieve a 0.917 mean dice score with 0.0352 standard deviation on cross validation across the training data and a 0.9506 mean dice score on a separate test set. To our knowledge, these results are better than any known method for CSF segmentation in hydrocephalic patients, and thus, it is promising for potential practical applications.

Download Full-text

Neural Network based Whole Heart Segmentation from 3D CT images

International journal of electrical and computer engineering systems ◽

10.32985/ijeces.11.1.3 ◽

2020 ◽

Vol 11 (1) ◽

pp. 25-31

Author(s):

Marija Habijan ◽

Hrvoje Leventić ◽

Irena Galić ◽

Danilo Babin

Keyword(s):

Neural Network ◽

Data Augmentation ◽

Principal Component ◽

Ground Truth ◽

Ct Images ◽

Training Data ◽

Optimal Learning ◽

Learning Rates ◽

Augmentation Techniques ◽

Whole Heart

The most recent research is showing the importance and suitability of neural networks for medical image processing tasks. Nonetheless, their efficiency in segmentation tasks is greatly dependent on the amount of available training data. To overcome issues of using small datasets, various data augmentation techniques have been developed. In this paper, an approach for the whole heart segmentation based on the convolutional neural network, specifically on the 3D U-Net architecture, is presented. Also, we propose the incorporation of the principal component analysis as an additional data augmentation technique. The network is trained end-to-end, i.e., no pre-trained network is required. Evaluation of the proposed approach is performed on CT images from MICCAI 2017 Multi-Modality Whole Heart Segmentation Challenge dataset, delivering in a three-fold cross-validation an average dice coefficient overlap of 88.2% for the whole heart, i.e. all heart substructures. Final segmentation results show a high accuracy with the ground truth, indicating that the proposed approach is competitive to the state-of-the-art. Additionally, experiments on the influence of different learning rates are provided as well, showing the optimal learning rate of 0.005 to give the best segmentation results.

Download Full-text

Robust Approach to Supervised Deep Neural Network Training for Real-Time Object Classification in Cluttered Indoor Environment

Applied Sciences ◽

10.3390/app11157148 ◽

2021 ◽

Vol 11 (15) ◽

pp. 7148

Author(s):

Bedada Endale ◽

Abera Tullu ◽

Hayoung Shi ◽

Beom-Soo Kang

Keyword(s):

Neural Network ◽

Deep Learning ◽

Real Time ◽

Network Architecture ◽

Input Data ◽

Deep Neural Network ◽

Data Augmentation ◽

Object Classification ◽

Training Data ◽

Gradient Descent Algorithm

Unmanned aerial vehicles (UAVs) are being widely utilized for various missions: in both civilian and military sectors. Many of these missions demand UAVs to acquire artificial intelligence about the environments they are navigating in. This perception can be realized by training a computing machine to classify objects in the environment. One of the well known machine training approaches is supervised deep learning, which enables a machine to classify objects. However, supervised deep learning comes with huge sacrifice in terms of time and computational resources. Collecting big input data, pre-training processes, such as labeling training data, and the need for a high performance computer for training are some of the challenges that supervised deep learning poses. To address these setbacks, this study proposes mission specific input data augmentation techniques and the design of light-weight deep neural network architecture that is capable of real-time object classification. Semi-direct visual odometry (SVO) data of augmented images are used to train the network for object classification. Ten classes of 10,000 different images in each class were used as input data where 80% were for training the network and the remaining 20% were used for network validation. For the optimization of the designed deep neural network, a sequential gradient descent algorithm was implemented. This algorithm has the advantage of handling redundancy in the data more efficiently than other algorithms.

Download Full-text

Rethinking the Random Cropping Data Augmentation Method Used in the Training of CNN-Based SAR Image Ship Detector

Remote Sensing ◽

10.3390/rs13010034 ◽

2020 ◽

Vol 13 (1) ◽

pp. 34

Author(s):

Rong Yang ◽

Robert Wang ◽

Yunkai Deng ◽

Xiaoxue Jia ◽

Heng Zhang

Keyword(s):

Neural Network ◽

Data Augmentation ◽

Back Propagation ◽

Detection Performance ◽

Training Data ◽

Sar Image ◽

Optical Images ◽

The Neural Network ◽

Effective Training ◽

Standard Configuration

The random cropping data augmentation method is widely used to train convolutional neural network (CNN)-based target detectors to detect targets in optical images (e.g., COCO datasets). It can expand the scale of the dataset dozens of times while consuming only a small amount of calculations when training the neural network detector. In addition, random cropping can also greatly enhance the spatial robustness of the model, because it can make the same target appear in different positions of the sample image. Nowadays, random cropping and random flipping have become the standard configuration for those tasks with limited training data, which makes it natural to introduce them into the training of CNN-based synthetic aperture radar (SAR) image ship detectors. However, in this paper, we show that the introduction of traditional random cropping methods directly in the training of the CNN-based SAR image ship detector may generate a lot of noise in the gradient during back propagation, which hurts the detection performance. In order to eliminate the noise in the training gradient, a simple and effective training method based on feature map mask is proposed. Experiments prove that the proposed method can effectively eliminate the gradient noise introduced by random cropping and significantly improve the detection performance under a variety of evaluation indicators without increasing inference cost.

Download Full-text

Oversampling Based on Data Augmentation in Convolutional Neural Network for Silicon Wafer Defect Classification

Knowledge Innovation Through Intelligent Software Methodologies, Tools and Techniques - Frontiers in Artificial Intelligence and Applications ◽

10.3233/faia200547 ◽

2020 ◽

Author(s):

Uzma Batool ◽

Mohd Ibrahim Shapiai ◽

Nordinah Ismail ◽

Hilman Fauzi ◽

Syahrizal Salleh

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Silicon Wafer ◽

Data Augmentation ◽

Imbalanced Data ◽

Training Data ◽

Defect Classification ◽

Learning Method ◽

Test Set

Silicon wafer defect data collected from fabrication facilities is intrinsically imbalanced because of the variable frequencies of defect types. Frequently occurring types will have more influence on the classification predictions if a model gets trained on such skewed data. A fair classifier for such imbalanced data requires a mechanism to deal with type imbalance in order to avoid biased results. This study has proposed a convolutional neural network for wafer map defect classification, employing oversampling as an imbalance addressing technique. To have an equal participation of all classes in the classifier’s training, data augmentation has been employed, generating more samples in minor classes. The proposed deep learning method has been evaluated on a real wafer map defect dataset and its classification results on the test set returned a 97.91% accuracy. The results were compared with another deep learning based auto-encoder model demonstrating the proposed method, a potential approach for silicon wafer defect classification that needs to be investigated further for its robustness.

Download Full-text

Attention-Based Residual Network with Scattering Transform Features for Hyperspectral Unmixing with Limited Training Samples

Remote Sensing ◽

10.3390/rs12030400 ◽

2020 ◽

Vol 12 (3) ◽

pp. 400 ◽

Cited By ~ 2

Author(s):

Zeng ◽

Ritz ◽

Zhao ◽

Lan

Keyword(s):

Neural Network ◽

Additive Noise ◽

Hyperspectral Data ◽

Training Data ◽

K Nearest Neighbors ◽

Practical Applications ◽

Hyperspectral Unmixing ◽

Scattering Transform ◽

Training Samples ◽

Limited Training Samples

This paper proposes a framework for unmixing of hyperspectral data that is based on utilizing the scattering transform to extract deep features that are then used within a neural network. Previous research has shown that using the scattering transform combined with a traditional K-nearest neighbors classifier (STFHU) is able to achieve more accurate unmixing results compared to a convolutional neural network (CNN) applied directly to the hyperspectral images. This paper further explores hyperspectral unmixing in limited training data scenarios, which are likely to occur in practical applications where the access to large amounts of labeled training data is not possible. Here, it is proposed to combine the scattering transform with the attention-based residual neural network (ResNet). Experimental results on three HSI datasets demonstrate that this approach provides at least 40% higher unmixing accuracy compared to the previous STFHU and CNN algorithms when using limited training data, ranging from 5% to 30%, are available. The use of the scattering transform for deriving features within the ResNet unmixing system also leads more than 25% improvement when unmixing hyperspectral data contaminated by additive noise.

Download Full-text

On the Effectiveness of Adversarial Training in Defending against Adversarial Example Attacks for Image Classification

Applied Sciences ◽

10.3390/app10228079 ◽

2020 ◽

Vol 10 (22) ◽

pp. 8079

Author(s):

Sanglee Park ◽

Jungmin So

Keyword(s):

Data Augmentation ◽

Black Box ◽

Training Data ◽

Model Parameters ◽

Neural Network Models ◽

Practical Applications ◽

Target Network ◽

Adversarial Examples ◽

Adversarial Training ◽

Adversarial Example

State-of-the-art neural network models are actively used in various fields, but it is well-known that they are vulnerable to adversarial example attacks. Throughout the efforts to make the models robust against adversarial example attacks, it has been found to be a very difficult task. While many defense approaches were shown to be not effective, adversarial training remains as one of the promising methods. In adversarial training, the training data are augmented by “adversarial” samples generated using an attack algorithm. If the attacker uses a similar attack algorithm to generate adversarial examples, the adversarially trained network can be quite robust to the attack. However, there are numerous ways of creating adversarial examples, and the defender does not know what algorithm the attacker may use. A natural question is: Can we use adversarial training to train a model robust to multiple types of attack? Previous work have shown that, when a network is trained with adversarial examples generated from multiple attack methods, the network is still vulnerable to white-box attacks where the attacker has complete access to the model parameters. In this paper, we study this question in the context of black-box attacks, which can be a more realistic assumption for practical applications. Experiments with the MNIST dataset show that adversarially training a network with an attack method helps defending against that particular attack method, but has limited effect for other attack methods. In addition, even if the defender trains a network with multiple types of adversarial examples and the attacker attacks with one of the methods, the network could lose accuracy to the attack if the attacker uses a different data augmentation strategy on the target network. These results show that it is very difficult to make a robust network using adversarial training, even for black-box settings where the attacker has restricted information on the target network.

Download Full-text

Segmentation Neural Network Incorporating Scale-Space in the Application of Cardiac MRI

Journal of Medical Imaging and Health Informatics ◽

10.1166/jmihi.2020.3041 ◽

2020 ◽

Vol 10 (7) ◽

pp. 1494-1505

Author(s):

Hyo-Hun Kim ◽

Byung-Woo Hong

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Cardiac Mri ◽

Network Architecture ◽

Data Augmentation ◽

Scale Space ◽

Segmentation Algorithm ◽

Training Data ◽

Training Procedure ◽

Whole Heart

In this work, we present an image segmentation algorithm based on the convolutional neural network framework where the scale space theory is incorporated in the course of training procedure. The construction of data augmentation is designed to apply the scale space to the training data in order to effectively deal with the variability of regions of interest in geometry and appearance such as shape and contrast. The proposed data augmentation algorithm via scale space is aimed to improve invariant features with respect to both geometry and appearance by taking into consideration of their diffusion process. We develop a segmentation algorithm based on the convolutional neural network framework where the network architecture consists of encoding and decoding substructures in combination with the data augmentation scheme via the scale space induced by the heat equation. The quantitative analysis using the cardiac MRI dataset indicates that the proposed algorithm achieves better accuracy in the delineation of the left ventricles, which demonstrates the potential of the algorithm in the application of the whole heart segmentation as a compute-aided diagnosis system for the cardiac diseases.

Download Full-text

Convolutional Neural Network-Based Discriminator for Outlier Detection

Computational Intelligence and Neuroscience ◽

10.1155/2021/8811147 ◽

2021 ◽

Vol 2021 ◽

pp. 1-13

Author(s):

Fahad Alharbi ◽

Khalil El Hindi ◽

Saad Al Ahmadi ◽

Hussien Alsalamn

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Systematic Approach ◽

Data Augmentation ◽

Random Noise ◽

Training Data ◽

Human Errors ◽

Machine Learning Methods ◽

Benchmark Datasets ◽

Trusted Data

Noise in training data increases the tendency of many machine learning methods to overfit the training data, which undermines the performance. Outliers occur in big data as a result of various factors, including human errors. In this work, we present a novel discriminator model for the identification of outliers in the training data. We propose a systematic approach for creating training datasets to train the discriminator based on a small number of genuine instances (trusted data). The noise discriminator is a convolutional neural network (CNN). We evaluate the discriminator’s performance using several benchmark datasets and with different noise ratios. We inserted random noise in each dataset and trained discriminators to clean them. Different discriminators were trained using different numbers of genuine instances with and without data augmentation. We compare the performance of the proposed noise-discriminator method with seven other methods proposed in the literature using several benchmark datasets. Our empirical results indicate that the proposed method is very competitive to the other methods. It actually outperforms them for pair noise.

Download Full-text

Fish Detection Using Deep Learning

Applied Computational Intelligence and Soft Computing ◽

10.1155/2020/3738108 ◽

2020 ◽

Vol 2020 ◽

pp. 1-13 ◽

Cited By ~ 3

Author(s):

Suxia Cui ◽

Yu Zhou ◽

Yonghui Wang ◽

Lujun Zhai

Keyword(s):

Neural Network ◽

Deep Learning ◽

Data Augmentation ◽

Autonomous Underwater Vehicle ◽

Training Data ◽

Training Process ◽

Data Set ◽

Speed Up ◽

Fish Detection ◽

Process Speed

Recently, human being’s curiosity has been expanded from the land to the sky and the sea. Besides sending people to explore the ocean and outer space, robots are designed for some tasks dangerous for living creatures. Take the ocean exploration for an example. There are many projects or competitions on the design of Autonomous Underwater Vehicle (AUV) which attracted many interests. Authors of this article have learned the necessity of platform upgrade from a previous AUV design project, and would like to share the experience of one task extension in the area of fish detection. Because most of the embedded systems have been improved by fast growing computing and sensing technologies, which makes them possible to incorporate more and more complicated algorithms. In an AUV, after acquiring surrounding information from sensors, how to perceive and analyse corresponding information for better judgement is one of the challenges. The processing procedure can mimic human being’s learning routines. An advanced system with more computing power can facilitate deep learning feature, which exploit many neural network algorithms to simulate human brains. In this paper, a convolutional neural network (CNN) based fish detection method was proposed. The training data set was collected from the Gulf of Mexico by a digital camera. To fit into this unique need, three optimization approaches were applied to the CNN: data augmentation, network simplification, and training process speed up. Data augmentation transformation provided more learning samples; the network was simplified to accommodate the artificial neural network; the training process speed up is introduced to make the training process more time efficient. Experimental results showed that the proposed model is promising, and has the potential to be extended to other underwear objects.

Download Full-text

A Comparison of Regularization Techniques in Deep Neural Networks

Symmetry ◽

10.3390/sym10110648 ◽

2018 ◽

Vol 10 (11) ◽

pp. 648 ◽

Cited By ~ 6

Author(s):

Ismoilov Nusrat ◽

Sung-Bong Jang

Keyword(s):

Neural Network ◽

Neural Networks ◽

Network Model ◽

Neural Network Model ◽

Data Augmentation ◽

Training Data ◽

Regularization Techniques ◽

Performance Results ◽

Normalization Scheme ◽

Significant Attention

Artificial neural networks (ANN) have attracted significant attention from researchers because many complex problems can be solved by training them. If enough data are provided during the training process, ANNs are capable of achieving good performance results. However, if training data are not enough, the predefined neural network model suffers from overfitting and underfitting problems. To solve these problems, several regularization techniques have been devised and widely applied to applications and data analysis. However, it is difficult for developers to choose the most suitable scheme for a developing application because there is no information regarding the performance of each scheme. This paper describes comparative research on regularization techniques by evaluating the training and validation errors in a deep neural network model, using a weather dataset. For comparisons, each algorithm was implemented using a recent neural network library of TensorFlow. The experiment results showed that an autoencoder had the worst performance among schemes. When the prediction accuracy was compared, data augmentation and the batch normalization scheme showed better performance than the others.

Download Full-text