A generative flow-based model for volumetric data augmentation in 3D deep learning for computed tomographic colonography

Abstract Purpose Deep learning can be used for improving the performance of computer-aided detection (CADe) in various medical imaging tasks. However, in computed tomographic (CT) colonography, the performance is limited by the relatively small size and the variety of the available training datasets. Our purpose in this study was to develop and evaluate a flow-based generative model for performing 3D data augmentation of colorectal polyps for effective training of deep learning in CADe for CT colonography. Methods We developed a 3D-convolutional neural network (3D CNN) based on a flow-based generative model (3D Glow) for generating synthetic volumes of interest (VOIs) that has characteristics similar to those of the VOIs of its training dataset. The 3D Glow was trained to generate synthetic VOIs of polyps by use of our clinical CT colonography case collection. The evaluation was performed by use of a human observer study with three observers and by use of a CADe-based polyp classification study with a 3D DenseNet. Results The area-under-the-curve values of the receiver operating characteristic analysis of the three observers were not statistically significantly different in distinguishing between real polyps and synthetic polyps. When trained with data augmentation by 3D Glow, the 3D DenseNet yielded a statistically significantly higher polyp classification performance than when it was trained with alternative augmentation methods. Conclusion The 3D Glow-generated synthetic polyps are visually indistinguishable from real colorectal polyps. Their application to data augmentation can substantially improve the performance of 3D CNNs in CADe for CT colonography. Thus, 3D Glow is a promising method for improving the performance of deep learning in CADe for CT colonography.

Download Full-text

Machine learning applied on chest x-ray can aid in the diagnosis of COVID-19: a first experience from Lombardy, Italy

European Radiology Experimental ◽

10.1186/s41747-020-00203-z ◽

2021 ◽

Vol 5 (1) ◽

Cited By ~ 1

Author(s):

Isabella Castiglioni ◽

Davide Ippolito ◽

Matteo Interlenghi ◽

Caterina Beatrice Monti ◽

Christian Salvatore ◽

...

Keyword(s):

Deep Learning ◽

Area Under The Curve ◽

Clinical Information ◽

Training Dataset ◽

Independent Dataset ◽

X Ray ◽

Learning Classifier ◽

Chest X Ray ◽

Polymerase Chain ◽

Deep Learning Model

Abstract Background We aimed to train and test a deep learning classifier to support the diagnosis of coronavirus disease 2019 (COVID-19) using chest x-ray (CXR) on a cohort of subjects from two hospitals in Lombardy, Italy. Methods We used for training and validation an ensemble of ten convolutional neural networks (CNNs) with mainly bedside CXRs of 250 COVID-19 and 250 non-COVID-19 subjects from two hospitals (Centres 1 and 2). We then tested such system on bedside CXRs of an independent group of 110 patients (74 COVID-19, 36 non-COVID-19) from one of the two hospitals. A retrospective reading was performed by two radiologists in the absence of any clinical information, with the aim to differentiate COVID-19 from non-COVID-19 patients. Real-time polymerase chain reaction served as the reference standard. Results At 10-fold cross-validation, our deep learning model classified COVID-19 and non-COVID-19 patients with 0.78 sensitivity (95% confidence interval [CI] 0.74–0.81), 0.82 specificity (95% CI 0.78–0.85), and 0.89 area under the curve (AUC) (95% CI 0.86–0.91). For the independent dataset, deep learning showed 0.80 sensitivity (95% CI 0.72–0.86) (59/74), 0.81 specificity (29/36) (95% CI 0.73–0.87), and 0.81 AUC (95% CI 0.73–0.87). Radiologists’ reading obtained 0.63 sensitivity (95% CI 0.52–0.74) and 0.78 specificity (95% CI 0.61–0.90) in Centre 1 and 0.64 sensitivity (95% CI 0.52–0.74) and 0.86 specificity (95% CI 0.71–0.95) in Centre 2. Conclusions This preliminary experience based on ten CNNs trained on a limited training dataset shows an interesting potential of deep learning for COVID-19 diagnosis. Such tool is in training with new CXRs to further increase its performance.

Download Full-text

Confronting Deep-Learning and Biodiversity Challenges for Automatic Video-Monitoring of Marine Ecosystems

Sensors ◽

10.3390/s22020497 ◽

2022 ◽

Vol 22 (2) ◽

pp. 497

Author(s):

Sébastien Villon ◽

Corina Iovan ◽

Morgan Mangeas ◽

Laurent Vigliola

Keyword(s):

Deep Learning ◽

Data Augmentation ◽

Low Cost ◽

Video Recording ◽

Species Abundance ◽

Automatic Monitoring ◽

Training Dataset ◽

Data Generation ◽

Automatic Assessment ◽

Open Set

With the availability of low-cost and efficient digital cameras, ecologists can now survey the world’s biodiversity through image sensors, especially in the previously rather inaccessible marine realm. However, the data rapidly accumulates, and ecologists face a data processing bottleneck. While computer vision has long been used as a tool to speed up image processing, it is only since the breakthrough of deep learning (DL) algorithms that the revolution in the automatic assessment of biodiversity by video recording can be considered. However, current applications of DL models to biodiversity monitoring do not consider some universal rules of biodiversity, especially rules on the distribution of species abundance, species rarity and ecosystem openness. Yet, these rules imply three issues for deep learning applications: the imbalance of long-tail datasets biases the training of DL models; scarce data greatly lessens the performances of DL models for classes with few data. Finally, the open-world issue implies that objects that are absent from the training dataset are incorrectly classified in the application dataset. Promising solutions to these issues are discussed, including data augmentation, data generation, cross-entropy modification, few-shot learning and open set recognition. At a time when biodiversity faces the immense challenges of climate change and the Anthropocene defaunation, stronger collaboration between computer scientists and ecologists is urgently needed to unlock the automatic monitoring of biodiversity.

Download Full-text

AUTOMATIC FISH DETECTION FROM DIFFERENT MARINE ENVIRONMENTS VIDEO USING DEEP LEARNING

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlvi-4-w3-2021-191-2022 ◽

2022 ◽

Vol XLVI-4/W3-2021 ◽

pp. 191-198

Author(s):

A. Loulidi ◽

R. Houssa ◽

L. Buhl-Mortensen ◽

H. Zidane ◽

H. Rhinane

Keyword(s):

Deep Learning ◽

Fish Species ◽

Data Augmentation ◽

Atlantic Coast ◽

Training Dataset ◽

Optimization Approach ◽

Marine Environments ◽

Low Light ◽

Fish Detection ◽

Performance Results

Abstract. The marine environment provides many ecosystems that support habitats biodiversity. Benthic habitats and fish species associations are investigated using underwater gears to secure and manage these marine ecosystems in a sustainable manner. The current study evaluates the possibility of using deep learning methods in particular the You Only Look Once version 3 algorithm to detect fish in different environments such as; different shading, low light, and high noise within images and by each frame within an underwater video, recorded in the Atlantic Coast of Morocco. The training dataset was collected from Open Images Dataset V6, a total of 1295 Fish images were captured and split into a training set and a test set. An optimization approach was applied to the YOLOv3 algorithm which is data augmentation transformation to provide more learning samples. The mean average precision (mAP) metric was applied to measure the YOLOv3 model’s performance. Results of this study revealed with a mAP of 91,3% the proposed method is proved to have the capability of detecting fish species in different natural marine environments also it has the potential to be applied to detect other underwater species and substratum.

Download Full-text

Fire Detection Method in Smart City Environments Using a Deep-Learning-Based Approach

Electronics ◽

10.3390/electronics11010073 ◽

2021 ◽

Vol 11 (1) ◽

pp. 73

Author(s):

Kuldoshbay Avazov ◽

Mukhriddin Mukhiddinov ◽

Fazliddin Makhmudov ◽

Young Im Cho

Keyword(s):

Deep Learning ◽

Urban Areas ◽

High Speed ◽

Data Augmentation ◽

Smart Cities ◽

Fire Detection ◽

Training Dataset ◽

Digital Cameras ◽

Augmentation Techniques ◽

Fire Accidents

In the construction of new smart cities, traditional fire-detection systems can be replaced with vision-based systems to establish fire safety in society using emerging technologies, such as digital cameras, computer vision, artificial intelligence, and deep learning. In this study, we developed a fire detector that accurately detects even small sparks and sounds an alarm within 8 s of a fire outbreak. A novel convolutional neural network was developed to detect fire regions using an enhanced You Only Look Once (YOLO) v4network. Based on the improved YOLOv4 algorithm, we adapted the network to operate on the Banana Pi M3 board using only three layers. Initially, we examined the originalYOLOv4 approach to determine the accuracy of predictions of candidate fire regions. However, the anticipated results were not observed after several experiments involving this approach to detect fire accidents. We improved the traditional YOLOv4 network by increasing the size of the training dataset based on data augmentation techniques for the real-time monitoring of fire disasters. By modifying the network structure through automatic color augmentation, reducing parameters, etc., the proposed method successfully detected and notified the incidence of disastrous fires with a high speed and accuracy in different weather environments—sunny or cloudy, day or night. Experimental results revealed that the proposed method can be used successfully for the protection of smart cities and in monitoring fires in urban areas. Finally, we compared the performance of our method with that of recently reported fire-detection approaches employing widely used performance matrices to test the fire classification results achieved.

Download Full-text

Few-Shot Wideband Tympanometry Classification in Otosclerosis via Domain Adaptation with Gaussian Processes

Applied Sciences ◽

10.3390/app112411839 ◽

2021 ◽

Vol 11 (24) ◽

pp. 11839

Author(s):

Leixin Nie ◽

Chao Li ◽

Alexis Bozorg Grayeli ◽

Franck Marzani

Keyword(s):

Transfer Learning ◽

Gaussian Processes ◽

Data Augmentation ◽

Domain Adaptation ◽

Area Under The Curve ◽

Characteristic Analysis ◽

Significant Discrepancy ◽

Non Invasive ◽

Middle Ear Disease ◽

Realistic Data

Otosclerosis is a common middle ear disease that requires a combination of examinations for its diagnosis in routine. In a previous study, we showed that this disease could be potentially diagnosed by wideband tympanometry (WBT) coupled with a convolutional neural network (CNN) in a rapid and non-invasive manner. We showed that deep transfer learning with data augmentation could be applied successfully on such a task. However, the involved synthetic and realistic data have a significant discrepancy that impedes the performance of transfer learning. To address this issue, a Gaussian processes-guided domain adaptation (GPGDA) algorithm was developed. It leveraged both the loss about the distribution distance calculated by the Gaussian processes and the loss of conventional cross entropy during the transferring. On a WBT dataset including 80 otosclerosis and 55 control samples, it achieved an area-under-the-curve of 97.9±1.1 percent after receiver operating characteristic analysis and an F1-score of 95.7±0.9 percent that were superior to the baseline methods (r=10, p<0.05, ANOVA). To understand the algorithm’s behavior, the role of each component in the GPGDA was experimentally explored on the dataset. In conclusion, our GPGDA algorithm appears to be an effective tool to enhance CNN-based WBT classification in otosclerosis using just a limited number of realistic data samples.

Download Full-text

A Survey on Deep Learning-Based Short/Zero-Calibration Approaches for EEG-Based Brain–Computer Interfaces

Frontiers in Human Neuroscience ◽

10.3389/fnhum.2021.643386 ◽

2021 ◽

Vol 15 ◽

Author(s):

Wonjun Ko ◽

Eunjin Jeon ◽

Seungwoo Jeong ◽

Jaeun Phyo ◽

Heung-Il Suk

Keyword(s):

Deep Learning ◽

Explicit Knowledge ◽

Data Augmentation ◽

Generative Model ◽

Machine Learning Techniques ◽

Brain Computer Interfaces ◽

Computer Interfaces ◽

Training Samples ◽

Calibration Methods ◽

Complex Patterns

Brain–computer interfaces (BCIs) utilizing machine learning techniques are an emerging technology that enables a communication pathway between a user and an external system, such as a computer. Owing to its practicality, electroencephalography (EEG) is one of the most widely used measurements for BCI. However, EEG has complex patterns and EEG-based BCIs mostly involve a cost/time-consuming calibration phase; thus, acquiring sufficient EEG data is rarely possible. Recently, deep learning (DL) has had a theoretical/practical impact on BCI research because of its use in learning representations of complex patterns inherent in EEG. Moreover, algorithmic advances in DL facilitate short/zero-calibration in BCI, thereby suppressing the data acquisition phase. Those advancements include data augmentation (DA), increasing the number of training samples without acquiring additional data, and transfer learning (TL), taking advantage of representative knowledge obtained from one dataset to address the so-called data insufficiency problem in other datasets. In this study, we review DL-based short/zero-calibration methods for BCI. Further, we elaborate methodological/algorithmic trends, highlight intriguing approaches in the literature, and discuss directions for further research. In particular, we search for generative model-based and geometric manipulation-based DA methods. Additionally, we categorize TL techniques in DL-based BCIs into explicit and implicit methods. Our systematization reveals advances in the DA and TL methods. Among the studies reviewed herein, ~45% of DA studies used generative model-based techniques, whereas ~45% of TL studies used explicit knowledge transferring strategy. Moreover, based on our literature review, we recommend an appropriate DA strategy for DL-based BCIs and discuss trends of TLs used in DL-based BCIs.

Download Full-text

Efficient Pneumonia Detection in Chest Xray Images Using Deep Transfer Learning

Diagnostics ◽

10.3390/diagnostics10060417 ◽

2020 ◽

Vol 10 (6) ◽

pp. 417 ◽

Cited By ~ 5

Author(s):

Mohammad Farukh Hashmi ◽

Satyarth Katiyar ◽

Avinash G Keskar ◽

Neeraj Dhanraj Bokde ◽

Zong Woo Geem

Keyword(s):

Deep Learning ◽

Transfer Learning ◽

Data Augmentation ◽

Medical Center ◽

Training Dataset ◽

Test Accuracy ◽

X Rays ◽

Learning Models ◽

Novel Approach ◽

Unseen Data

Pneumonia causes the death of around 700,000 children every year and affects 7% of the global population. Chest X-rays are primarily used for the diagnosis of this disease. However, even for a trained radiologist, it is a challenging task to examine chest X-rays. There is a need to improve the diagnosis accuracy. In this work, an efficient model for the detection of pneumonia trained on digital chest X-ray images is proposed, which could aid the radiologists in their decision making process. A novel approach based on a weighted classifier is introduced, which combines the weighted predictions from the state-of-the-art deep learning models such as ResNet18, Xception, InceptionV3, DenseNet121, and MobileNetV3 in an optimal way. This approach is a supervised learning approach in which the network predicts the result based on the quality of the dataset used. Transfer learning is used to fine-tune the deep learning models to obtain higher training and validation accuracy. Partial data augmentation techniques are employed to increase the training dataset in a balanced way. The proposed weighted classifier is able to outperform all the individual models. Finally, the model is evaluated, not only in terms of test accuracy, but also in the AUC score. The final proposed weighted classifier model is able to achieve a test accuracy of 98.43% and an AUC score of 99.76 on the unseen data from the Guangzhou Women and Children’s Medical Center pneumonia dataset. Hence, the proposed model can be used for a quick diagnosis of pneumonia and can aid the radiologists in the diagnosis process.

Download Full-text

Automatic cell counting from stimulated Raman imaging using deep learning

PLoS ONE ◽

10.1371/journal.pone.0254586 ◽

2021 ◽

Vol 16 (7) ◽

pp. e0254586

Author(s):

Qianqian Zhang ◽

Kyung Keun Yun ◽

Hao Wang ◽

Sang Won Yoon ◽

Fake Lu ◽

...

Keyword(s):

Deep Learning ◽

Stimulated Raman Scattering ◽

Area Under The Curve ◽

Semantic Segmentation ◽

Cell Counting ◽

Label Free ◽

Characteristic Analysis ◽

Human Brain Tumor ◽

Training Samples ◽

Stimulated Raman

In this paper, we propose an automatic cell counting framework for stimulated Raman scattering (SRS) images, which can assist tumor tissue characteristic analysis, cancer diagnosis, and surgery planning processes. SRS microscopy has promoted tumor diagnosis and surgery by mapping lipids and proteins from fresh specimens and conducting a fast disclose of fundamental diagnostic hallmarks of tumors with a high resolution. However, cell counting from label-free SRS images has been challenging due to the limited contrast of cells and tissue, along with the heterogeneity of tissue morphology and biochemical compositions. To this end, a deep learning-based cell counting scheme is proposed by modifying and applying U-Net, an effective medical image semantic segmentation model that uses a small number of training samples. The distance transform and watershed segmentation algorithms are also implemented to yield the cell instance segmentation and cell counting results. By performing cell counting on SRS images of real human brain tumor specimens, promising cell counting results are obtained with > 98% of area under the curve (AUC) and R = 0.97 in terms of cell counting correlation between SRS and histological images with hematoxylin and eosin (H&E) staining. The proposed cell counting scheme illustrates the possibility and potential of performing cell counting automatically in near real time and encourages the study of applying deep learning techniques in biomedical and pathological image analyses.

Download Full-text

The Effectiveness of Image Augmentation in Deep Learning Networks for Detecting COVID-19: A Geometric Transformation Perspective

Frontiers in Medicine ◽

10.3389/fmed.2021.629134 ◽

2021 ◽

Vol 8 ◽

Author(s):

Mohamed Elgendi ◽

Muhammad Umer Nasir ◽

Qunfeng Tang ◽

David Smith ◽

John-Paul Grenier ◽

...

Keyword(s):

Deep Learning ◽

Data Augmentation ◽

Learning Algorithms ◽

Training Dataset ◽

Geometric Transformation ◽

Detection Accuracy ◽

X Rays ◽

X Ray ◽

Testing Dataset ◽

The Impact

Chest X-ray imaging technology used for the early detection and screening of COVID-19 pneumonia is both accessible worldwide and affordable compared to other non-invasive technologies. Additionally, deep learning methods have recently shown remarkable results in detecting COVID-19 on chest X-rays, making it a promising screening technology for COVID-19. Deep learning relies on a large amount of data to avoid overfitting. While overfitting can result in perfect modeling on the original training dataset, on a new testing dataset it can fail to achieve high accuracy. In the image processing field, an image augmentation step (i.e., adding more training data) is often used to reduce overfitting on the training dataset, and improve prediction accuracy on the testing dataset. In this paper, we examined the impact of geometric augmentations as implemented in several recent publications for detecting COVID-19. We compared the performance of 17 deep learning algorithms with and without different geometric augmentations. We empirically examined the influence of augmentation with respect to detection accuracy, dataset diversity, augmentation methodology, and network size. Contrary to expectation, our results show that the removal of recently used geometrical augmentation steps actually improved the Matthews correlation coefficient (MCC) of 17 models. The MCC without augmentation (MCC = 0.51) outperformed four recent geometrical augmentations (MCC = 0.47 for Data Augmentation 1, MCC = 0.44 for Data Augmentation 2, MCC = 0.48 for Data Augmentation 3, and MCC = 0.49 for Data Augmentation 4). When we retrained a recently published deep learning without augmentation on the same dataset, the detection accuracy significantly increased, with a χMcNemar′s statistic2=163.2 and a p-value of 2.23 × 10−37. This is an interesting finding that may improve current deep learning algorithms using geometrical augmentations for detecting COVID-19. We also provide clinical perspectives on geometric augmentation to consider regarding the development of a robust COVID-19 X-ray-based detector.

Download Full-text

COVID-FACT: A Fully-Automated Capsule Network-Based Framework for Identification of COVID-19 Cases from Chest CT Scans

Frontiers in Artificial Intelligence ◽

10.3389/frai.2021.598932 ◽

2021 ◽

Vol 4 ◽

Author(s):

Shahin Heidarian ◽

Parnian Afshar ◽

Nastaran Enshaei ◽

Farnoosh Naderkhani ◽

Moezedin Javad Rafiee ◽

...

Keyword(s):

Health Care ◽

Deep Learning ◽

Early Diagnosis ◽

Data Augmentation ◽

Spatial Information ◽

Area Under The Curve ◽

Spatial Relations ◽

Building Blocks ◽

Ct Scans ◽

Rapid Separation

The newly discovered Coronavirus Disease 2019 (COVID-19) has been globally spreading and causing hundreds of thousands of deaths around the world as of its first emergence in late 2019. The rapid outbreak of this disease has overwhelmed health care infrastructures and arises the need to allocate medical equipment and resources more efficiently. The early diagnosis of this disease will lead to the rapid separation of COVID-19 and non-COVID cases, which will be helpful for health care authorities to optimize resource allocation plans and early prevention of the disease. In this regard, a growing number of studies are investigating the capability of deep learning for early diagnosis of COVID-19. Computed tomography (CT) scans have shown distinctive features and higher sensitivity compared to other diagnostic tests, in particular the current gold standard, i.e., the Reverse Transcription Polymerase Chain Reaction (RT-PCR) test. Current deep learning-based algorithms are mainly developed based on Convolutional Neural Networks (CNNs) to identify COVID-19 pneumonia cases. CNNs, however, require extensive data augmentation and large datasets to identify detailed spatial relations between image instances. Furthermore, existing algorithms utilizing CT scans, either extend slice-level predictions to patient-level ones using a simple thresholding mechanism or rely on a sophisticated infection segmentation to identify the disease. In this paper, we propose a two-stage fully automated CT-based framework for identification of COVID-19 positive cases referred to as the “COVID-FACT”. COVID-FACT utilizes Capsule Networks, as its main building blocks and is, therefore, capable of capturing spatial information. In particular, to make the proposed COVID-FACT independent from sophisticated segmentations of the area of infection, slices demonstrating infection are detected at the first stage and the second stage is responsible for classifying patients into COVID and non-COVID cases. COVID-FACT detects slices with infection, and identifies positive COVID-19 cases using an in-house CT scan dataset, containing COVID-19, community acquired pneumonia, and normal cases. Based on our experiments, COVID-FACT achieves an accuracy of 90.82%, a sensitivity of 94.55%, a specificity of 86.04%, and an Area Under the Curve (AUC) of 0.98, while depending on far less supervision and annotation, in comparison to its counterparts.

Download Full-text