scholarly journals An Analysis of State-of-the-art Activation Functions For Supervised Deep Neural Network

2021 ◽  
Author(s):  
Anh Nguyen ◽  
Khoa Pham ◽  
Dat Ngo ◽  
Thanh Ngo ◽  
Lam Pham
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Activation Functions

scholarly journals An Analysis of State-of-the-art Activation Functions For Supervised Deep Neural Network

2021 ◽  
Author(s):  
Anh Nguyen ◽  
Khoa Pham ◽  
Dat Ngo ◽  
Thanh Ngo ◽  
Lam Pham
Keyword(s):  
Neural Network ◽  
Supervised Classification ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Network Architectures ◽  
Activation Functions ◽  
Scene Classification ◽  
Learning Network ◽  
Deep Learning Network

This paper provides an analysis of state-of-the-art activation functions with respect to supervised classification of deep neural network. These activation functions comprise of Rectified Linear Units (ReLU), Exponential Linear Unit (ELU), Scaled Exponential Linear Unit (SELU), Gaussian Error Linear Unit (GELU), and the Inverse Square Root Linear Unit (ISRLU). To evaluate, experiments over two deep learning network architectures integrating these activation functions are conducted. The first model, basing on Multilayer Perceptron (MLP), is evaluated with MNIST dataset to perform these activation functions.Meanwhile, the second model, likely VGGish-based architecture, is applied for Acoustic Scene Classification (ASC) Task 1A in DCASE 2018 challenge, thus evaluate whether these activation functions work well in different datasets as well as different network architectures.

scholarly journals Fast Accurate and Automatic Brushstroke Extraction

2021 ◽  
Vol 17 (2) ◽  
pp. 1-24
Author(s):  
Yunfei Fu ◽  
Hongchuan Yu ◽  
Chih-Kuo Yeh ◽  
Tong-Yee Lee ◽  
Jian J. Zhang
Keyword(s):  
Neural Network ◽  
Efficient Algorithm ◽  
Deep Neural Network ◽  
High Efficiency ◽  
State Of The Art ◽  
High Reliability ◽  
The Other ◽  
Manual Annotation ◽  
Stroke Extraction ◽  
Art Research

Brushstrokes are viewed as the artist’s “handwriting” in a painting. In many applications such as style learning and transfer, mimicking painting, and painting authentication, it is highly desired to quantitatively and accurately identify brushstroke characteristics from old masters’ pieces using computer programs. However, due to the nature of hundreds or thousands of intermingling brushstrokes in the painting, it still remains challenging. This article proposes an efficient algorithm for brush Stroke extraction based on a Deep neural network, i.e., DStroke. Compared to the state-of-the-art research, the main merit of the proposed DStroke is to automatically and rapidly extract brushstrokes from a painting without manual annotation, while accurately approximating the real brushstrokes with high reliability. Herein, recovering the faithful soft transitions between brushstrokes is often ignored by the other methods. In fact, the details of brushstrokes in a master piece of painting (e.g., shapes, colors, texture, overlaps) are highly desired by artists since they hold promise to enhance and extend the artists’ powers, just like microscopes extend biologists’ powers. To demonstrate the high efficiency of the proposed DStroke, we perform it on a set of real scans of paintings and a set of synthetic paintings, respectively. Experiments show that the proposed DStroke is noticeably faster and more accurate at identifying and extracting brushstrokes, outperforming the other methods.

scholarly journals An Efficient Method for Detection of DDoS Attacks on the Web Using Deep Learning Algorithms

2021 ◽  
Vol 10 (4) ◽  
pp. 2821-2829
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Ddos Attacks ◽  
Problem Statement ◽  
Neural Network Approach ◽  
Learning Techniques ◽  
Attack Data ◽  
Deep Learning Neural Network

Recently, DDoS attacks is the most significant threat in network security. Both industry and academia are currently debating how to detect and protect against DDoS attacks. Many studies are provided to detect these types of attacks. Deep learning techniques are the most suitable and efficient algorithm for categorizing normal and attack data. Hence, a deep neural network approach is proposed in this study to mitigate DDoS attacks effectively. We used a deep learning neural network to identify and classify traffic as benign or one of four different DDoS attacks. We will concentrate on four different DDoS types: Slowloris, Slowhttptest, DDoS Hulk, and GoldenEye. The rest of the paper is organized as follow: Firstly, we introduce the work, Section 2 defines the related works, Section 3 presents the problem statement, Section 4 describes the proposed methodology, Section 5 illustrate the results of the proposed methodology and shows how the proposed methodology outperforms state-of-the-art work and finally Section VI concludes the paper.

scholarly journals Regularized Training and Tight Certification for Randomized Smoothed Classifier with Provable Robustness

2020 ◽  
Vol 34 (04) ◽  
pp. 3858-3865
Author(s):  
Huijie Feng ◽  
Chunpeng Wu ◽  
Guoyang Chen ◽  
Weifeng Zhang ◽  
Yang Ning
Keyword(s):  
Neural Network ◽  
High Probability ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Computationally Efficient ◽  
Base Classifier ◽  
Training Scheme ◽  
Adversarial Training ◽  
Gaussian Perturbation ◽  
Probabilistic Robustness

Recently smoothing deep neural network based classifiers via isotropic Gaussian perturbation is shown to be an effective and scalable way to provide state-of-the-art probabilistic robustness guarantee against ℓ2 norm bounded adversarial perturbations. However, how to train a good base classifier that is accurate and robust when smoothed has not been fully investigated. In this work, we derive a new regularized risk, in which the regularizer can adaptively encourage the accuracy and robustness of the smoothed counterpart when training the base classifier. It is computationally efficient and can be implemented in parallel with other empirical defense methods. We discuss how to implement it under both standard (non-adversarial) and adversarial training scheme. At the same time, we also design a new certification algorithm, which can leverage the regularization effect to provide tighter robustness lower bound that holds with high probability. Our extensive experimentation demonstrates the effectiveness of the proposed training and certification approaches on CIFAR-10 and ImageNet datasets.

scholarly journals Covid-19 detection via deep neural network and occlusion sensitivity maps

2021 ◽  
Author(s):  
Noor Ahmad ◽  
Muhammad Aminu ◽  
Mohd Halim Mohd Noor
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Color Images ◽  
Fine Tuning ◽  
Training Dataset ◽  
Learning Approaches ◽  
Learning Models ◽  
Sensitivity Maps

Deep learning approaches have attracted a lot of attention in the automatic detection of Covid-19 and transfer learning is the most common approach. However, majority of the pre-trained models are trained on color images, which can cause inefficiencies when fine-tuning the models on Covid-19 images which are often grayscale. To address this issue, we propose a deep learning architecture called CovidNet which requires a relatively smaller number of parameters. CovidNet accepts grayscale images as inputs and is suitable for training with limited training dataset. Experimental results show that CovidNet outperforms other state-of-the-art deep learning models for Covid-19 detection.

CTL-DNNet: Effective Circular Traffic Light Recognition with a Deep Neural Network

2017 ◽  
Vol 31 (11) ◽  
pp. 1750037 ◽  
Author(s):  
Di Wang ◽  
Hong Bao ◽  
Feifei Zhang
Keyword(s):  
Neural Network ◽  
Autonomous Vehicles ◽  
Deep Neural Network ◽  
Road Traffic ◽  
High Accuracy ◽  
Activation Functions ◽  
Traffic Light ◽  
Traffic Lights ◽  
Learning Network ◽  
Deep Learning Network

This paper proposed an algorithm for a deep learning network for identifying circular traffic lights (CTL-DNNet). The sample labeling process uses translation to increase the number of positive samples, and the similarity is calculated to reduce the number of negative samples, thereby reducing overfitting. We use a dataset of approximately 370[Formula: see text]000 samples, with approximately 20[Formula: see text]000 positive samples and approximately 350[Formula: see text]000 negative samples. The datasets are generated from images taken at the Beijing Garden Expo. To obtain a very robust method for the detection of traffic lights, we use different layers, different cost functions and different activation functions of the depth neural network for training and comparison. Our algorithm has evaluated autonomous vehicles in varying illumination and gets the result with high accuracy and robustness. The experimental results show that CTL-DNNet is effective at recognizing road traffic lights in the Beijing Garden Expo area.

scholarly journals Covid-19 detection via deep neural network and occlusion sensitivity maps

2021 ◽  
Author(s):  
Noor Ahmad ◽  
Muhammad Aminu ◽  
Mohd Halim Mohd Noor
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Color Images ◽  
Fine Tuning ◽  
Training Dataset ◽  
Learning Approaches ◽  
Learning Models ◽  
Sensitivity Maps

Deep learning approaches have attracted a lot of attention in the automatic detection of Covid-19 and transfer learning is the most common approach. However, majority of the pre-trained models are trained on color images, which can cause inefficiencies when fine-tuning the models on Covid-19 images which are often grayscale. To address this issue, we propose a deep learning architecture called CovidNet which requires a relatively smaller number of parameters. CovidNet accepts grayscale images as inputs and is suitable for training with limited training dataset. Experimental results show that CovidNet outperforms other state-of-the-art deep learning models for Covid-19 detection.

scholarly journals Super resolution DOA estimation based on deep neural network

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Wanli Liu
Keyword(s):  
Neural Network ◽  
Potential Application ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Signal To Noise Ratio ◽  
Super Resolution ◽  
Doa Estimation ◽  
Angular Distance ◽  
Signal To Noise ◽  
Simulation Results

AbstractRecently, deep neural network (DNN) studies on direction-of-arrival (DOA) estimations have attracted more and more attention. This new method gives an alternative way to deal with DOA problem and has successfully shown its potential application. However, these works are often restricted to previously known signal number, same signal-to-noise ratio (SNR) or large intersignal angular distance, which will hinder their generalization in real application. In this paper, we present a novel DNN framework that realizes higher resolution and better generalization to random signal number and SNR. Simulation results outperform that of previous works and reach the state of the art.

Sign in / Sign up

Export Citation Format

Share Document