scholarly journals Hyperparameter Optimization and Regularization on Fashion-MNIST Classification

2019 ◽  
Vol 8 (2) ◽  
pp. 3713-3719
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Data Augmentation ◽  
Network Models ◽  
Classification Problems ◽  
Regularization Techniques ◽  
Cropping Rotation ◽  
The Impact ◽  
Deep Learning Model ◽  
Learning In Computer Vision

Nowadays the most exciting technology breakthrough has been the rise of the deep learning. In computer vision Convolutional Neural Networks (CNN or ConvNet) are the default deep learning model used for image classification problems. In these deep network models, feature extraction is figure out by itself and these models tend to perform well with huge amount of samples. Herein we explore the impact of various Hyper-Parameter Optimization (HPO) methods and regularization techniques with deep neural networks on FashionMNIST (F-MNIST) dataset which is proposed by Zalando Research. We have proposed deep ConvNet architectures with Data Augmentation and explore the impact of this by configuring the hyperparameters and regularization methods. As deep learning requires a lots of data, the insufficiency of image samples can be expand through various data augmentation methods like Cropping, Rotation, Flipping, and Shifting. The experimental results show impressive results on this new benchmarking dataset F-MNIST

scholarly journals Data augmentation for computed tomography angiography via synthetic image generation and neural domain adaptation

2020 ◽  
Vol 6 (1) ◽  
Author(s):  
Malte Seemann ◽  
Lennart Bargsten ◽  
Alexander Schlaefer
Keyword(s):  
Computed Tomography ◽  
Neural Networks ◽  
Deep Learning ◽  
Medical Imaging ◽  
Computed Tomography Angiography ◽  
Data Augmentation ◽  
Domain Adaptation ◽  
Synthetic Image ◽  
Wide Range ◽  
The Impact

AbstractDeep learning methods produce promising results when applied to a wide range of medical imaging tasks, including segmentation of artery lumen in computed tomography angiography (CTA) data. However, to perform sufficiently, neural networks have to be trained on large amounts of high quality annotated data. In the realm of medical imaging, annotations are not only quite scarce but also often not entirely reliable. To tackle both challenges, we developed a two-step approach for generating realistic synthetic CTA data for the purpose of data augmentation. In the first step moderately realistic images are generated in a purely numerical fashion. In the second step these images are improved by applying neural domain adaptation. We evaluated the impact of synthetic data on lumen segmentation via convolutional neural networks (CNNs) by comparing resulting performances. Improvements of up to 5% in terms of Dice coefficient and 20% for Hausdorff distance represent a proof of concept that the proposed augmentation procedure can be used to enhance deep learning-based segmentation for artery lumen in CTA images.

scholarly journals Detection of COVID-19 and Other Pneumonia Cases Using Convolutional Neural Networks and X-ray Images

2021 ◽  
Vol 42 (1) ◽  
pp. e90289
Author(s):  
Carlos Eduardo Belman López
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Classification Accuracy ◽  
Network Capacity ◽  
Superior Performance ◽  
Data Set ◽  
X Ray ◽  
Regularization Techniques ◽  
The Impact

Given that it is fundamental to detect positive COVID-19 cases and treat affected patients quickly to mitigate the impact of the virus, X-ray images have been subjected to research regarding COVID-19, together with deep learning models, eliminating disadvantages such as the scarcity of RT-PCR test kits, their elevated costs, and the long wait for results. The contribution of this paper is to present new models for detecting COVID-19 and other cases of pneumonia using chest X-ray images and convolutional neural networks, thus providing accurate diagnostics in binary and 4-classes classification scenarios. Classification accuracy was improved, and overfitting was prevented by following 2 actions: (1) increasing the data set size while the classification scenarios were balanced; and (2) adding regularization techniques and performing hyperparameter optimization. Additionally, the network capacity and size in the models were reduced as much as possible, making the final models a perfect option to be deployed locally on devices with limited capacities and without the need for Internet access. The impact of key hyperparameters was tested using modern deep learning packages. The final models obtained a classification accuracy of 99,17 and 94,03% for the binary and categorical scenarios, respectively, achieving superior performance compared to other studies in the literature, and requiring a significantly lower number of parameters. The models can also be placed on a digital platform to provide instantaneous diagnostics and surpass the shortage of experts and radiologists.

scholarly journals Implementation of an incremental deep learning model for survival prediction of cardiovascular patients

2021 ◽  
Vol 10 (1) ◽  
pp. 101
Author(s):  
Sanaa Elyassami ◽  
Achraf Ait Kaddour
Keyword(s):  
Neural Networks ◽  
Heart Disease ◽  
Deep Learning ◽  
Learning Model ◽  
Classification Model ◽  
Stochastic Gradient Descent ◽  
Survival Prediction ◽  
Proposed Model ◽  
The Impact ◽  
Deep Learning Model

<span lang="EN-US">Cardiovascular diseases remain the leading cause of death, taking an estimated 17.9 million lives each year and representing 31% of all global deaths. The patient records including blood reports, cardiac echo reports, and physician’s notes can be used to perform feature analysis and to accurately classify heart disease patients. In this paper, an incremental deep learning model was developed and trained with stochastic gradient descent using feedforward neural networks. The chi-square test and the dropout regularization have been incorporated into the model to improve the generalization capabilities and the performance of the heart disease patients' classification model. The impact of the learning rate and the depth of neural networks on the performance were explored. The hyperbolic tangent, the rectifier linear unit, the Maxout, and the exponential rectifier linear unit were used as activation functions for the hidden and the output layer neurons. To avoid over-optimistic results, the performance of the proposed model was evaluated using balanced accuracy and the overall predictive value in addition to the accuracy, sensitivity, and specificity. The obtained results are promising, and the proposed model can be applied to a larger dataset and used by physicians to accurately classify heart disease patients.</span>

Levenshtein Augmentation Improves Performance of SMILES Based Deep-Learning Synthesis Prediction

2020 ◽  
Author(s):  
Dean Sumner ◽  
Jiazhen He ◽  
Amol Thakkar ◽  
Ola Engkvist ◽  
Esben Jannik Bjerrum
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Sequence Similarity ◽  
Learning Models ◽  
Underlying Network

<p>SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as <i>attentional gain </i>– an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.</p>

scholarly journals Comparative Analysis on Deep Learning Approaches for Heavy-Vehicle Detection based on Data Augmentation and Transfer-Learning techniques

2021 ◽  
Vol 13 (3) ◽  
pp. 809-820
Author(s):  
V. Sowmya ◽  
R. Radha
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Real Time ◽  
Transfer Learning ◽  
Convolutional Neural Networks ◽  
Traffic Management ◽  
Data Augmentation ◽  
Vehicle Detection ◽  
Heavy Vehicles ◽  
Detection And Recognition

Vehicle detection and recognition require demanding advanced computational intelligence and resources in a real-time traffic surveillance system for effective traffic management of all possible contingencies. One of the focus areas of deep intelligent systems is to facilitate vehicle detection and recognition techniques for robust traffic management of heavy vehicles. The following are such sophisticated mechanisms: Support Vector Machine (SVM), Convolutional Neural Networks (CNN), Regional Convolutional Neural Networks (R-CNN), You Only Look Once (YOLO) model, etcetera. Accordingly, it is pivotal to choose the precise algorithm for vehicle detection and recognition, which also addresses the real-time environment. In this study, a comparison of deep learning algorithms, such as the Faster R-CNN, YOLOv2, YOLOv3, and YOLOv4, are focused on diverse aspects of the features. Two entities for transport heavy vehicles, the buses and trucks, constitute detection and recognition elements in this proposed work. The mechanics of data augmentation and transfer-learning is implemented in the model; to build, execute, train, and test for detection and recognition to avoid over-fitting and improve speed and accuracy. Extensive empirical evaluation is conducted on two standard datasets such as COCO and PASCAL VOC 2007. Finally, comparative results and analyses are presented based on real-time.

Deep Learning Model for Digital Sales Increasing and Forecasting: Towards Smart E-Commerce

2021 ◽  
pp. 26-34
Author(s):  
admin admin ◽  
Keyword(s):  
Neural Networks ◽  
Decision Making ◽  
Deep Learning ◽  
Programming Languages ◽  
Digital Marketing ◽  
Deep Belief Networks ◽  
Marketing Manager ◽  
Management Level ◽  
Effective System ◽  
Deep Learning Model

In this paper, we have proposed a system that will be able to forecast the sales of the e-commerce systems by using the techniques of the deep learning, the main goal of this paper is to help the business and the top management level of the company in decision making in order to provide the workplace the effectiveness and the efficiency in the workplace and to provide an efficient and effective system that it is intelligence to forecast and increase the sales of an e-commerce system, this paper will start with building an e-commerce website using different programming languages which are HTML, CSS, Django, JavaScript Bootstrap, and it this e-commerce website will have a specific database that contains different tables for the product list, the orders, and for the user information and many other tables, then the deep learning algorithms such as Deep Belief Networks and Convolutional Neural Networks will be applied in order to provide an effective system for digital marketing usage, so, it will be able to function as a marketing manager.

scholarly journals The Impact of Emotional Music on Active ROI in Patients with Depression Based on Deep Learning: A Task-State fMRI Study

2019 ◽  
Vol 2019 ◽  
pp. 1-14
Author(s):  
Renzhou Gui ◽  
Tongjie Chen ◽  
Han Nie
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Correlation Matrix ◽  
Nearest Neighbor ◽  
Learning Model ◽  
Fmri Data ◽  
Support Vector ◽  
K Nearest Neighbor ◽  
The Impact ◽  
Deep Learning Model

With the continuous development of science, more and more research results have proved that machine learning is capable of diagnosing and studying the major depressive disorder (MDD) in the brain. We propose a deep learning network with multibranch and local residual feedback, for four different types of functional magnetic resonance imaging (fMRI) data produced by depressed patients and control people under the condition of listening to positive- and negative-emotions music. We use the large convolution kernel of the same size as the correlation matrix to match the features and obtain the results of feature matching of 264 regions of interest (ROIs). Firstly, four-dimensional fMRI data are used to generate the two-dimensional correlation matrix of one person’s brain based on ROIs and then processed by the threshold value which is selected according to the characteristics of complex network and small-world network. After that, the deep learning model in this paper is compared with support vector machine (SVM), logistic regression (LR), k-nearest neighbor (kNN), a common deep neural network (DNN), and a deep convolutional neural network (CNN) for classification. Finally, we further calculate the matched ROIs from the intermediate results of our deep learning model which can help related fields further explore the pathogeny of depression patients.

scholarly journals Rectified Wing Loss for Efficient and Robust Facial Landmark Localisation with Convolutional Neural Networks

2019 ◽  
Vol 128 (8-9) ◽  
pp. 2126-2145 ◽  
Author(s):  
Zhen-Hua Feng ◽  
Josef Kittler ◽  
Muhammad Awais ◽  
Xiao-Jun Wu
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Loss Function ◽  
Data Augmentation ◽  
Small Sample Size ◽  
Small Sample ◽  
Imbalance Problem ◽  
Facial Landmark ◽  
The Impact ◽  
Coarse To Fine

AbstractEfficient and robust facial landmark localisation is crucial for the deployment of real-time face analysis systems. This paper presents a new loss function, namely Rectified Wing (RWing) loss, for regression-based facial landmark localisation with Convolutional Neural Networks (CNNs). We first systemically analyse different loss functions, including L2, L1 and smooth L1. The analysis suggests that the training of a network should pay more attention to small-medium errors. Motivated by this finding, we design a piece-wise loss that amplifies the impact of the samples with small-medium errors. Besides, we rectify the loss function for very small errors to mitigate the impact of inaccuracy of manual annotation. The use of our RWing loss boosts the performance significantly for regression-based CNNs in facial landmarking, especially for lightweight network architectures. To address the problem of under-representation of samples with large pose variations, we propose a simple but effective boosting strategy, referred to as pose-based data balancing. In particular, we deal with the data imbalance problem by duplicating the minority training samples and perturbing them by injecting random image rotation, bounding box translation and other data augmentation strategies. Last, the proposed approach is extended to create a coarse-to-fine framework for robust and efficient landmark localisation. Moreover, the proposed coarse-to-fine framework is able to deal with the small sample size problem effectively. The experimental results obtained on several well-known benchmarking datasets demonstrate the merits of our RWing loss and prove the superiority of the proposed method over the state-of-the-art approaches.

Sign in / Sign up

Export Citation Format

Share Document