Experimental Comparison of Stochastic Optimizers in Deep Learning

2019 ◽  
pp. 704-715 ◽  
Author(s):  
Emmanuel Okewu ◽  
Philip Adewole ◽  
Oladipupo Sennaike
Keyword(s):  
Deep Learning ◽  
Experimental Comparison

scholarly journals An Experimental Comparison between Deep Learning and Classical Machine Learning Approaches for Writer Identification in Medieval Documents

2020 ◽  
Vol 6 (9) ◽  
pp. 89
Author(s):  
Nicole Dalia Cilia ◽  
Claudio De Stefano ◽  
Francesco Fontanella ◽  
Claudio Marrocco ◽  
Mario Molinara ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
High Performance ◽  
Ad Hoc ◽  
Digital Images ◽  
Experimental Comparison ◽  
Learning Approaches ◽  
Test Bed ◽  
Ancient Manuscripts ◽  
Ancient Documents

In the framework of palaeography, the availability of both effective image analysis algorithms, and high-quality digital images has favored the development of new applications for the study of ancient manuscripts and has provided new tools for decision-making support systems. The quality of the results provided by such applications, however, is strongly influenced by the selection of effective features, which should be able to capture the distinctive aspects to which the paleography expert is interested in. This process is very difficult to generalize due to the enormous variability in the type of ancient documents, produced in different historical periods with different languages and styles. The effect is that it is very difficult to define standard techniques that are general enough to be effectively used in any case, and this is the reason why ad-hoc systems, generally designed according to paleographers’ suggestions, have been designed for the analysis of ancient manuscripts. In recent years, there has been a growing scientific interest in the use of techniques based on deep learning (DL) for the automatic processing of ancient documents. This interest is not only due to their capability of designing high-performance pattern recognition systems, but also to their ability of automatically extracting features from raw data, without using any a priori knowledge. Moving from these considerations, the aim of this study is to verify if DL-based approaches may actually represent a general methodology for automatically designing machine learning systems for palaeography applications. To this purpose, we compared the performance of a DL-based approach with that of a “classical” machine learning one, in a particularly unfavorable case for DL, namely that of highly standardized schools. The rationale of this choice is to compare the obtainable results even when context information is present and discriminating: this information is ignored by DL approaches, while it is used by machine learning methods, making the comparison more significant. The experimental results refer to the use of a large sets of digital images extracted from an entire 12th-century Bibles, the “Avila Bible”. This manuscript, produced by several scribes who worked in different periods and in different places, represents a severe test bed to evaluate the efficiency of scribe identification systems.

scholarly journals Computerized Ultrasonic Imaging Inspection: From Shallow to Deep Learning

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 3820 ◽  
Author(s):  
Jiaxing Ye ◽  
Shunya Ito ◽  
Nobuyuki Toyama
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Visual Information ◽  
Human Error ◽  
Ultrasonic Inspection ◽  
Ultrasonic Imaging ◽  
Representation Learning ◽  
Processing Technique ◽  
Experimental Comparison ◽  
Image Analysis System

For many decades, ultrasonic imaging inspection has been adopted as a principal method to detect multiple defects, e.g., void and corrosion. However, the data interpretation relies on an inspector’s subjective judgment, thus making the results vulnerable to human error. Nowadays, advanced computer vision techniques reveal new perspectives on the high-level visual understanding of universal tasks. This research aims to develop an efficient automatic ultrasonic image analysis system for nondestructive testing (NDT) using the latest visual information processing technique. To this end, we first established an ultrasonic inspection image dataset containing 6849 ultrasonic scan images with full defect/no-defect annotations. Using the dataset, we performed a comprehensive experimental comparison of various computer vision techniques, including both conventional methods using hand-crafted visual features and the most recent convolutional neural networks (CNN) which generate multiple-layer stacking for representation learning. In the computer vision community, the two groups are referred to as shallow and deep learning, respectively. Experimental results make it clear that the deep learning-enabled system outperformed conventional (shallow) learning schemes by a large margin. We believe this benchmarking could be used as a reference for similar research dealing with automatic defect detection in ultrasonic imaging inspection.

scholarly journals TraMiner: Vision-Based Analysis of Locomotion Traces for Cognitive Assessment in Smart-Homes

2021 ◽  
Author(s):  
Samaneh Zolfaghari ◽  
Elham Khodabandehloo ◽  
Daniele Riboni
Keyword(s):  
Deep Learning ◽  
Cognitive Impairment ◽  
Smart Homes ◽  
The Elderly ◽  
Extraction Methods ◽  
Cognitive Status ◽  
Sensor Data ◽  
Experimental Comparison ◽  
Test Bed ◽  
Segmentation Image

AbstractThe rapid increase in the senior population is posing serious challenges to national healthcare systems. Hence, innovative tools are needed to early detect health issues, including cognitive decline. Several clinical studies show that it is possible to identify cognitive impairment based on the locomotion patterns of the elderly. In this work, we investigate the use of sensor data and deep learning to recognize those patterns in instrumented smart-homes. In order to get rid of the noise introduced by indoor constraints and activity execution, we introduce novel visual feature extraction methods for locomotion data. Our solution relies on locomotion trace segmentation, image-based extraction of salient features from locomotion segments, and vision-based deep learning. We carried out extensive experiments with a large dataset acquired in a smart-home test bed from 153 seniors, including people with cognitive diseases. Results show that our system can accurately recognize the cognitive status of the senior, reaching a macro-$$F_1$$ F 1 score of 0.873 for the three categories that we target: cognitive health, mild cognitive impairment, and dementia. Moreover, an experimental comparison shows that our system outperforms state-of-the-art methods.

scholarly journals 6mAPred-MSFF: A Deep Learning Model for Predicting DNA N6-Methyladenine Sites across Species Based on a Multi-Scale Feature Fusion Mechanism

2021 ◽  
Vol 11 (16) ◽  
pp. 7731
Author(s):  
Rao Zeng ◽  
Minghong Liao
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Prediction Accuracy ◽  
Cross Validation ◽  
Feature Fusion ◽  
Experimental Comparison ◽  
Scale Feature ◽  
Learning Framework ◽  
Multi Scale ◽  
Fold Cross Validation

DNA methylation is one of the most extensive epigenetic modifications. DNA N6-methyladenine (6mA) plays a key role in many biology regulation processes. An accurate and reliable genome-wide identification of 6mA sites is crucial for systematically understanding its biological functions. Some machine learning tools can identify 6mA sites, but their limited prediction accuracy and lack of robustness limit their usability in epigenetic studies, which implies the great need of developing new computational methods for this problem. In this paper, we developed a novel computational predictor, namely the 6mAPred-MSFF, which is a deep learning framework based on a multi-scale feature fusion mechanism to identify 6mA sites across different species. In the predictor, we integrate the inverted residual block and multi-scale attention mechanism to build lightweight and deep neural networks. As compared to existing predictors using traditional machine learning, our deep learning framework needs no prior knowledge of 6mA or manually crafted sequence features and sufficiently capture better characteristics of 6mA sites. By benchmarking comparison, our deep learning method outperforms the state-of-the-art methods on the 5-fold cross-validation test on the seven datasets of six species, demonstrating that the proposed 6mAPred-MSFF is more effective and generic. Specifically, our proposed 6mAPred-MSFF gives the sensitivity and specificity of the 5-fold cross-validation on the 6mA-rice-Lv dataset as 97.88% and 94.64%, respectively. Our model trained with the rice data predicts well the 6mA sites of other five species: Arabidopsis thaliana, Fragaria vesca, Rosa chinensis, Homo sapiens, and Drosophila melanogaster with a prediction accuracy 98.51%, 93.02%, and 91.53%, respectively. Moreover, via experimental comparison, we explored performance impact by training and testing our proposed model under different encoding schemes and feature descriptors.

scholarly journals A deep learning-based computer-aided diagnosis method of X-ray images for bone age assessment

2021 ◽  
Author(s):  
Shaowei Li ◽  
Bowen Liu ◽  
Shulian Li ◽  
Xinyu Zhu ◽  
Yang Yan ◽  
...  
Keyword(s):  
Deep Learning ◽  
Bone Age ◽  
Computer Aided Diagnosis ◽  
Experimental Comparison ◽  
Age Assessment ◽  
X Ray ◽  
Computer Aided ◽  
Bone Age Assessment ◽  
Diagnosis Method ◽  
Aided Diagnosis

AbstractBone age assessment using hand-wrist X-ray images is fundamental when diagnosing growth disorders of a child or providing a more patient-specific treatment. However, as clinical procedures are a subjective assessment, the accuracy depends highly on the doctor’s experience. Motivated by this, a deep learning-based computer-aided diagnosis method was proposed for performing bone age assessment. Inspired by clinical approaches and aimed to reduce expensive manual annotations, informative regions localization based on a complete unsupervised learning method was firstly performed and an image-processing pipeline was proposed. Subsequently, an image model with pre-trained weights as a backbone was utilized to enhance the reliability of prediction. The prediction head was implemented by a Multiple Layer Perceptron with one hidden layer. In compliance with clinical studies, gender information was an additional input to the prediction head by embedded into the feature vector calculated from the backbone model. After the experimental comparison study, the best results showed a mean absolute error of 6.2 months on the public RSNA dataset and 5.1 months on the additional dataset using MobileNetV3 as the backbone.

scholarly journals Change Detection in Urban Point Clouds: An Experimental Comparison with Simulated 3D Datasets

2021 ◽  
Vol 13 (13) ◽  
pp. 2629
Author(s):  
Iris de Gélis ◽  
Sébastien Lefèvre ◽  
Thomas Corpetti
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Change Detection ◽  
Transfer Learning ◽  
Point Clouds ◽  
Experimental Comparison ◽  
Training Set ◽  
Learning Methods ◽  
Public Dataset ◽  
Different Characteristics

In the context of rapid urbanization, monitoring the evolution of cities is crucial. To do so, 3D change detection and characterization is of capital importance since, unlike 2D images, 3D data contain vertical information of utmost importance to monitoring city evolution (that occurs along both horizontal and vertical axes). Urban 3D change detection has thus received growing attention, and various methods have been published on the topic. Nevertheless, no quantitative comparison on a public dataset has been reported yet. This study presents an experimental comparison of six methods: three traditional (difference of DSMs, C2C and M3C2), one machine learning with hand-crafted features (a random forest model with a stability feature) and two deep learning (feed-forward and Siamese architectures). In order to compare these methods, we prepared five sub-datasets containing simulated pairs of 3D annotated point clouds with different characteristics: from high to low resolution, with various levels of noise. The methods have been tested on each sub-dataset for binary and multi-class segmentation. For supervised methods, we also assessed the transfer learning capacity and the influence of the training set size. The methods we used provide various kinds of results (2D pixels, 2D patches or 3D points), and each of them is impacted by the resolution of the PCs. However, while the performances of deep learning methods highly depend on the size of the training set, they seem to be less impacted by training on datasets with different characteristics. Oppositely, conventional machine learning methods exhibit stable results, even with smaller training sets, but embed low transfer learning capacities. While the main changes in our datasets were usually identified, there were still numerous instances of false detection, especially in dense urban areas, thereby calling for further development in this field. To assist such developments, we provide a public dataset composed of pairs of point clouds with different qualities together with their change-related annotations. This dataset was built with an original simulation tool which allows one to generate bi-temporal urban point clouds under various conditions.

Deep Learning

2009 ◽  
Author(s):  
Stellan Ohlsson
Keyword(s):  
Deep Learning
Sign in / Sign up

Export Citation Format

Share Document