scholarly journals Checkerboard artifacts free convolutional neural networks

2019 ◽  
Vol 8 ◽  
Author(s):  
Yusuke Sugawara ◽  
Sayaka Shiota ◽  
Hitoshi Kiya
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Experimental Results ◽  
The Novel ◽  
Multirate Systems ◽  
Conventional Condition ◽  
Novel Structure ◽  
Avoidance Condition

AbstractIt is well-known that a number of convolutional neural networks (CNNs) generate checkerboard artifacts in both of two processes: forward-propagation of upsampling layers and backpropagation of convolutional layers. A condition for avoiding the artifacts is proposed in this paper. So far, these artifacts have been studied mainly for linear multirate systems, but the conventional condition for avoiding them cannot be applied to CNNs due to the non-linearity of CNNs. We extend the avoidance condition for CNNs and apply the proposed structure to typical CNNs to confirm whether the novel structure is effective. Experimental results demonstrate that the proposed structure can perfectly avoid generating checkerboard artifacts while keeping the excellent properties that CNNs have.

scholarly journals Convolutional Neural Networks for Leaf Image-Based Plant Disease Classification

2019 ◽  
Vol 8 (4) ◽  
pp. 328
Author(s):  
Sachin B. Jadhav
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Plant Diseases ◽  
Experimental Results ◽  
Disease Classification ◽  
Soybean Leaves ◽  
Soybean Diseases ◽  
Validation Strategy

<span lang="EN-US">Plant pathologists desire soft computing technology for accurate and reliable diagnosis of plant diseases. In this study, we propose an efficient soybean disease identification method based on a transfer learning approach by using a pre-trained convolutional neural network (CNN’s) such as AlexNet, GoogleNet, VGG16, ResNet101, and DensNet201. The proposed convolutional neural networks were trained using 1200 plant village image dataset of diseased and healthy soybean leaves, to identify three soybean diseases out of healthy leaves. Pre-trained CNN used to enable a fast and easy system implementation in practice. We used the five-fold cross-validation strategy to analyze the performance of networks. In this study, we used a pre-trained convolutional neural network as feature extractors and classifiers. The experimental results based on the proposed approach using pre-trained AlexNet, GoogleNet, VGG16, ResNet101, and DensNet201 networks achieve an accuracy of 95%, 96.4 %, 96.4 %, 92.1%, 93.6% respectively. The experimental results for the identification of soybean diseases indicated that the proposed networks model achieves the highest accuracy</span>

scholarly journals CROSS-LANGUAGE TEXT CLASSIFICATION WITH CONVOLUTIONAL NEURAL NETWORKS FROM SCRATCH

2017 ◽  
Vol 2 ◽  
pp. 24-33 ◽  
Author(s):  
Musbah Zaid Enweiji ◽  
Taras Lehinevych ◽  
Аndrey Glybovets
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Current Method ◽  
Classification Model ◽  
The Novel ◽  
Novel Approach ◽  
Multilingual Learning ◽  
Cross Language ◽  
Language Text ◽  
Language Classification

Cross language classification is an important task in multilingual learning, where documents in different languages often share the same set of categories. The main goal is to reduce the labeling cost of training classification model for each individual language. The novel approach by using Convolutional Neural Networks for multilingual language classification is proposed in this article. It learns representation of knowledge gained from languages. Moreover, current method works for new individual language, which was not used in training. The results of empirical study on large dataset of 21 languages demonstrate robustness and competitiveness of the presented approach.

scholarly journals Regularizing Deep Neural Networks with an Ensemble-based Decorrelation Method

2018 ◽  
Author(s):  
Shuqin Gu ◽  
Yuexian Hou ◽  
Lipeng Zhang ◽  
Yazhou Zhang
Keyword(s):  
Neural Networks ◽  
Ensemble Learning ◽  
Convolutional Neural Networks ◽  
Deep Neural Networks ◽  
Experimental Results ◽  
Excellent Performance ◽  
Hidden Layer ◽  
Base Learner ◽  
Fully Connected

Although Deep Neural Networks (DNNs) have achieved excellent performance in many tasks, improving the generalization capacity of DNNs still remains a challenge. In this work, we propose a novel regularizer named Ensemble-based Decorrelation Method (EDM), which is motivated by the idea of the ensemble learning to improve generalization capacity of DNNs. EDM can be applied to hidden layers in fully connected neural networks or convolutional neural networks. We treat each hidden layer as an ensemble of several base learners through dividing all the hidden units into several non-overlap groups, and each group will be viewed as a base learner. EDM encourages DNNs to learn more diverse representations by minimizing the covariance between all base learners during the training step. Experimental results on MNIST and CIFAR datasets demonstrate that EDM can effectively reduce the overfitting and improve the generalization capacity of DNNs  

scholarly journals Multi-Branch-CNN: classification of ion channel interacting peptides using parallel convolutional neural networks

2021 ◽  
Author(s):  
Jielu Yan ◽  
Bob Zhang ◽  
Mingliang Zhou ◽  
Hang Fai Kwok ◽  
Shirley W.I. Siu
Keyword(s):  
Neural Networks ◽  
Ion Channels ◽  
Ion Channel ◽  
Convolutional Neural Networks ◽  
Data Sets ◽  
The Novel ◽  
Sodium Potassium ◽  
Test Set ◽  
Drug Candidates

Ligand peptides that have high affinity for ion channels are critical for regulating ion flux across the plasma membrane. These peptides are now being considered as potential drug candidates for many diseases, such as cardiovascular disease and cancers. There are several studies to identify ion channel interacting peptides computationally, but, to the best of our knowledge, none of them published available tools for prediction. To provide a solution, we present Multi-branch-CNN, a parallel convolutional neural networks (CNNs) method for identifying three types of ion channel peptide binders (sodium, potassium, and calcium). Our experiment shows that the Multi-Branch-CNN method performs comparably to thirteen traditional ML algorithms (TML13) on the test sets of three ion channels. To evaluate the predictive power of our method with respect to novel sequences, as is the case in real-world applications, we created an additional test set for each ion channel, called the novel-test set, which has little or no similarities to the sequences in either the sequences of the train set or the test set. In the novel-test experiment, Multi-Branch-CNN performs significantly better than TML13, showing an improvement in accuracy of 6%, 14%, and 15% for sodium, potassium, and calcium channels, respectively. We confirmed the effectiveness of Multi-Branch-CNN by comparing it to the standard CNN method with one input branch (Single-Branch-CNN) and an ensemble method (TML13-Stack). To facilitate applications, the data sets, script files to reproduce the experiments, and the final predictive models are freely available at https://github.com/jieluyan/Multi-Branch-CNN.

scholarly journals Self-paced Convolutional Neural Networks

2017 ◽  
Author(s):  
Hao Li ◽  
Maoguo Gong
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Stationary Solution ◽  
Convolutional Neural Networks ◽  
Reliable Data ◽  
Experimental Results ◽  
Theoretical Studies ◽  
Convolutional Network ◽  
Sample Weights ◽  
Learning Rates

Convolutional neural networks (CNNs) have achieved breakthrough performance in many pattern recognition tasks. In order to distinguish the reliable data from the noisy and confusing data, we improve CNNs with self-paced learning (SPL) for enhancing the learning robustness of CNNs. In the proposed self-paced convolutional network (SPCN), each sample is assigned to a weight to reflect the easiness of the sample. Then a dynamic self-paced function is incorporated into the leaning objective of CNN to jointly learn the parameters of CNN and the latent weight variable. SPCN learns the samples from easy to complex and the sample weights can dynamically control the learning rates for converging to better values. To gain more insights of SPCN, theoretical studies are conducted to show that SPCN converges to a stationary solution and is robust to the noisy and confusing data. Experimental results on MNIST and rectangles datasets demonstrate that the proposed method outperforms baseline methods.

A CONVblock for Convolutional Neural Networks

2021 ◽  
pp. 100-113
Author(s):  
Hmidi Alaeddine ◽  
Malek Jihene
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Convolutional Neural Networks ◽  
Experimental Results ◽  
Classification Models ◽  
Deep Architecture ◽  
Improved Performance ◽  
Convolution Filters ◽  
Classification Database

The reduction in the size of convolution filters has been shown to be effective in image classification models. They make it possible to reduce the calculation and the number of parameters used in the operations of the convolution layer while increasing the efficiency of the representation. The authors present a deep architecture for classification with improved performance. The main objective of this architecture is to improve the main performances of the network thanks to a new design based on CONVblock. The proposal is evaluated on a classification database: CIFAR-10 and MNIST. The experimental results demonstrate the effectiveness of the proposed method. This architecture offers an error of 1.4% on CIFAR-10 and 0.055% on MNIST.

Sanitizing hidden activations for improving adversarial robustness of convolutional neural networks

2021 ◽  
pp. 1-11
Author(s):  
Tianshi Mu ◽  
Kequan Lin ◽  
Huabing Zhang ◽  
Jian Wang
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Black Box ◽  
Experimental Results ◽  
Amplification Effect ◽  
Wide Range ◽  
Adversarial Examples

Deep learning is gaining significant traction in a wide range of areas. Whereas, recent studies have demonstrated that deep learning exhibits the fatal weakness on adversarial examples. Due to the black-box nature and un-transparency problem of deep learning, it is difficult to explain the reason for the existence of adversarial examples and also hard to defend against them. This study focuses on improving the adversarial robustness of convolutional neural networks. We first explore how adversarial examples behave inside the network through visualization. We find that adversarial examples produce perturbations in hidden activations, which forms an amplification effect to fool the network. Motivated by this observation, we propose an approach, termed as sanitizing hidden activations, to help the network correctly recognize adversarial examples by eliminating or reducing the perturbations in hidden activations. To demonstrate the effectiveness of our approach, we conduct experiments on three widely used datasets: MNIST, CIFAR-10 and ImageNet, and also compare with state-of-the-art defense techniques. The experimental results show that our sanitizing approach is more generalized to defend against different kinds of attacks and can effectively improve the adversarial robustness of convolutional neural networks.

scholarly journals Prediction for Chaotic Time Series-Based AE-CNN and Transfer Learning

Complexity ◽  
2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Baogui Xin ◽  
Wei Peng
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Transfer Learning ◽  
Convolutional Neural Networks ◽  
Prediction Performance ◽  
Chaotic Time Series ◽  
Experimental Results ◽  
Prediction Scheme

It has been a hot and challenging topic to predict the chaotic time series in the medium-to-long term. We combine autoencoders and convolutional neural networks (AE-CNN) to capture the intrinsic certainty of chaotic time series. We utilize the transfer learning (TL) theory to improve the prediction performance in medium-to-long term. Thus, we develop a prediction scheme for chaotic time series-based AE-CNN and TL named AE-CNN-TL. Our experimental results show that the proposed AE-CNN-TL has much better prediction performance than any one of the following: AE-CNN, ARMA, and LSTM.

scholarly journals High-Performance Tracking for Piezoelectric Actuators Using Super-Twisting Algorithm Based on Artificial Neural Networks

Mathematics ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. 244
Author(s):  
Cristian Napole ◽  
Oscar Barambones ◽  
Mohamed Derbeli ◽  
Isidro Calvo ◽  
Mohammed Yousri Silaa ◽  
...  
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
High Performance ◽  
Piezoelectric Actuators ◽  
The Novel ◽  
Tracking Accuracy ◽  
Novel Structure ◽  
Novel Approach ◽  
Artificial Neural ◽  
Twisting Algorithm

Piezoelectric actuators (PEA) are frequently employed in applications where nano-Micr-odisplacement is required because of their high-precision performance. However, the positioning is affected substantially by the hysteresis which resembles in an nonlinear effect. In addition, hysteresis mathematical models own deficiencies that can influence on the reference following performance. The objective of this study was to enhance the tracking accuracy of a commercial PEA stack actuator with the implementation of a novel approach which consists in the use of a Super-Twisting Algorithm (STA) combined with artificial neural networks (ANN). A Lyapunov stability proof is bestowed to explain the theoretical solution. Experimental results of the proposed method were compared with a proportional-integral-derivative (PID) controller. The outcomes in a real PEA reported that the novel structure is stable as it was proved theoretically, and the experiments provided a significant error reduction in contrast with the PID.

scholarly journals A Framework of Visual Checkout System Using Convolutional Neural Networks for Bento Buffet

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2627
Author(s):  
Mei-Yi Wu ◽  
Jia-Hong Lee ◽  
Chuan-Ying Hsueh
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Human Life ◽  
Experimental Results ◽  
Accuracy Rate ◽  
Desktop Computer ◽  
Recognition Time ◽  
Kinect Camera ◽  
Food Recognition

In recent years, the technology of artificial intelligence (AI) and robots is rapidly spreading to countries around the world. More and more scholars and industry experts have proposed AI deep learning models and methods to solve human life problems and improve work efficiency. Modern people’s lives are very busy, which led us to investigate whether the demand for Bento buffet cafeterias has gradually increased in Taiwan. However, when eating at a buffet in a cafeteria, people often encounter two problems. The first problem is that customers need to queue up to check out after they have selected and filled their dishes from the buffet. However, it always takes too much time waiting, especially at lunch or dinner time. The second problem is sometimes customers question the charges calculated by cafeteria staff, claiming they are too expensive at the checkout counter. Therefore, it is necessary to develop an AI-enabled checkout system. The AI-enabled self-checkout system will help the Bento buffet cafeterias reduce long lineups without the need to add additional workers. In this paper, we used computer vision and deep-learning technology to design and implement an AI-enabled checkout system for Bento buffet cafeterias. The prototype contains an angle steel shelf, a Kinect camera, a light source, and a desktop computer. Six baseline convolutional neural networks were applied for comparison on food recognition. In our experiments, there were 22 different food categories in a Bento buffet cafeteria employed. Experimental results show that the inception_v4 model can achieve the highest average validation accuracy of 99.11% on food recognition, but it requires the most training and recognition time. AlexNet model achieves a 94.5% accuracy and requires the least training time and recognition time. We propose a hierarchical approach with two stages to achieve good performance in both the recognition accuracy rate and the required training and recognition time. The approach is designed to perform the first step of identification and the second step of recognizing similar food images, respectively. Experimental results show that the proposed approach can achieve a 96.3% accuracy rate on our test dataset and required very little recognition time for input images. In addition, food volumes could be estimated using the depth images captured by the Kinect camera, and a framework of visual checkout system was successfully built.

Sign in / Sign up

Export Citation Format

Share Document