DPReLU: Dynamic Parametric Rectified Linear Unit

2020 ◽  
Author(s):  
Kien Mai Ngoc ◽  
Donghun Yang ◽  
Iksoo Shin ◽  
Hoyong Kim ◽  
Myunggwon Hwang
Information ◽  
2019 ◽  
Vol 10 (11) ◽  
pp. 356 ◽  
Author(s):  
Yuelei Xiao ◽  
Xing Xiao

Residual networks (ResNets) are prone to over-fitting for low-dimensional and small-scale datasets. And the existing intrusion detection systems (IDSs) fail to provide better performance, especially for remote-to-local (R2L) and user-to-root (U2R) attacks. To overcome these problems, a simplified residual network (S-ResNet) is proposed in this paper, which consists of several cascaded, simplified residual blocks. Compared with the original residual block, the simplified residual block deletes a weight layer and two batch normalization (BN) layers, adds a pooling layer, and replaces the rectified linear unit (ReLU) function with the parametric rectified linear unit (PReLU) function. Based on the S-ResNet, a novel IDS was proposed in this paper, which includes a data preprocessing module, a random oversampling module, a S-Resnet layer, a full connection layer and a Softmax layer. The experimental results on the NSL-KDD dataset show that the IDS based on the S-ResNet has a higher accuracy, recall and F1-score than the equal scale ResNet-based IDS, especially for R2L and U2R attacks. And the former has faster convergence velocity than the latter. It proves that the S-ResNet reduces the complexity of the network and effectively prevents over-fitting; thus, it is more suitable for low-dimensional and small-scale datasets than ResNet. Furthermore, the experimental results on the NSL-KDD datasets also show that the IDS based on the S-ResNet achieves better performance in terms of accuracy and recall compared to the existing IDSs, especially for R2L and U2R attacks.


2019 ◽  
Vol 20 (3) ◽  
pp. 251-259 ◽  
Author(s):  
Ilya Jackson ◽  
Jurijs Tolujevs ◽  
Sebastian Lang ◽  
Zhandos Kegenbekov

Abstract Inventory control problems arise in various industries, and each single real-world inventory is replete with non-standard factors and subtleties. Practical stochastic inventory control problems are often analytically intractable, because of their complexity. In this regard, simulation-optimization is becoming more and more popular tool for solving complicated business-driven problems. Unfortunately, simulation, especially detailed, is both time and memory consuming. In the light of this fact, it may be more reasonable to use an alternative cheaper-to-compute metamodel, which is specifically designed in order to approximate an original simulation. In this research we discus metamodelling of stochastic multiproduct inventory control system with perishable products using a multilayer perceptron with a rectified linear unit as an activation function.


2021 ◽  
Author(s):  
Alshimaa Hamdy ◽  
Tarek Abed Soliman ◽  
Mohamed Rihan ◽  
Moawad I. Dessouky

Abstract Beamforming design is a crucial stage in millimeter-wave systems with massive antenna arrays. We propose a deep learning network for the design of the precoder and combiner in hybrid architectures. The proposed network employs a parametric rectified linear unit (PReLU) activation function which improves model accuracy with almost no complexity cost compared to other functions. The proposed network accepts practical channel estimation input and can be trained to enhance spectral efficiency considering the hardware limitation of the hybrid design. Simulation shows that the proposed network achieves small performance improvement when compared to the same network with the ReLU activation function.


Author(s):  
Zhixian Chen ◽  
Jialin Tang ◽  
Xueyuan Gong ◽  
Qinglang Su

In order to improve the low accuracy of the face recognition methods in the case of e-health, this paper proposed a novel face recognition approach, which is based on convolutional neural network (CNN). In detail, through resolving the convolutional kernel, rectified linear unit (ReLU) activation function, dropout, and batch normalization, this novel approach reduces the number of parameters of the CNN model, improves the non-linearity of the CNN model, and alleviates overfitting of the CNN model. In these ways, the accuracy of face recognition is increased. In the experiments, the proposed approach is compared with principal component analysis (PCA) and support vector machine (SVM) on ORL, Cohn-Kanade, and extended Yale-B face recognition data set, and it proves that this approach is promising.


Sign in / Sign up

Export Citation Format

Share Document