scholarly journals Laplacian networks: bounding indicator function smoothness for neural networks robustness

2021 ◽  
Vol 10 ◽  
Author(s):  
Carlos Lassance ◽  
Vincent Gripon ◽  
Antonio Ortega
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Supervised Learning ◽  
Indicator Function ◽  
Training Data ◽  
Theoretical Justification ◽  
The Past ◽  
Noisy Examples

For the past few years, deep learning (DL) robustness (i.e. the ability to maintain the same decision when inputs are subject to perturbations) has become a question of paramount importance, in particular in settings where misclassification can have dramatic consequences. To address this question, authors have proposed different approaches, such as adding regularizers or training using noisy examples. In this paper we introduce a regularizer based on the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DL architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness on classical supervised learning vision datasets for various types of perturbations. We also show it can be combined with existing methods to increase overall robustness.

scholarly journals Autonomous Tomato Harvesting Robotic System in Greenhouses: Deep Learning Classification

Mekatronika ◽  
2019 ◽  
Vol 1 (1) ◽  
pp. 80-86
Author(s):  
Ooi Peng Toon ◽  
Muhammad Aizzat Zakaria ◽  
Ahmad Fakhri Ab. Nasir ◽  
Anwar P.P. Abdul Majeed ◽  
Chung Young Tan ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
South America ◽  
Solanum Lycopersicum ◽  
Training Data ◽  
New Classification ◽  
The Past ◽  
Testing Data ◽  
Harvesting Robot

Solanum lycopersicum or generally known as tomato came from countries of South America and has been growing in many tropical countries and its healthy nutrients in tomato becomes one of the food demand by the locals in Malaysia when their lifestyle shifted to more concern for healthy food. Since export value and production has increased for the past few years, a vast amount of labours considered for the fruit-picking process. Hence, farmers are now preferring to look for automation to replace labour problems and high cost that they are facing. To pick a correct fruit within clusters, a harvesting robot requires guidance so that it can detect a fruit accurately. In this study, a new classification algorithm using deep learning specifically convolution neural network to classify the image is either a tomato or not tomato and next, the image is classified into either a ripe or unripe tomato. Furthermore, there are two classification neural networks which are tomato or not tomato and ripe and unripe tomato. Each network consists of 600 training data and 33 testing data. The accuracies that obtained from network 1 (tomato or not tomato) and network 2 (ripe or unripe tomato) are 76.366% and 98.788% respectively.

scholarly journals Improving Semi-Supervised Learning for Audio Classification with FixMatch

Electronics ◽  
2021 ◽  
Vol 10 (15) ◽  
pp. 1807
Author(s):  
Sascha Grollmisch ◽  
Estefanía Cano
Keyword(s):  
Neural Networks ◽  
Supervised Learning ◽  
Transfer Learning ◽  
Data Transfer ◽  
State Of The Art ◽  
Training Data ◽  
Audio Classification ◽  
Image Domain ◽  
Full Dataset ◽  
Audio Data

Including unlabeled data in the training process of neural networks using Semi-Supervised Learning (SSL) has shown impressive results in the image domain, where state-of-the-art results were obtained with only a fraction of the labeled data. The commonality between recent SSL methods is that they strongly rely on the augmentation of unannotated data. This is vastly unexplored for audio data. In this work, SSL using the state-of-the-art FixMatch approach is evaluated on three audio classification tasks, including music, industrial sounds, and acoustic scenes. The performance of FixMatch is compared to Convolutional Neural Networks (CNN) trained from scratch, Transfer Learning, and SSL using the Mean Teacher approach. Additionally, a simple yet effective approach for selecting suitable augmentation methods for FixMatch is introduced. FixMatch with the proposed modifications always outperformed Mean Teacher and the CNNs trained from scratch. For the industrial sounds and music datasets, the CNN baseline performance using the full dataset was reached with less than 5% of the initial training data, demonstrating the potential of recent SSL methods for audio data. Transfer Learning outperformed FixMatch only for the most challenging dataset from acoustic scene classification, showing that there is still room for improvement.

Beyond Deep Learning: An Econometric Example

2020 ◽  
Vol 28 (Supp01) ◽  
pp. 31-38 ◽  
Author(s):  
Ruofan Liao ◽  
Paravee Maneejuk ◽  
Songsak Sriboonchitta
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Prediction Models ◽  
Original Data ◽  
Parametric Model ◽  
Parametric Models ◽  
The Past ◽  
Currency Exchange ◽  
Currency Exchange Rate ◽  
Linear And Nonlinear

In the past, in many areas, the best prediction models were linear and nonlinear parametric models. In the last decade, in many application areas, deep learning has shown to lead to more accurate predictions than the parametric models. Deep learning-based predictions are reasonably accurate, but not perfect. How can we achieve better accuracy? To achieve this objective, we propose to combine neural networks with parametric model: namely, to train neural networks not on the original data, but on the differences between the actual data and the predictions of the parametric model. On the example of predicting currency exchange rate, we show that this idea indeed leads to more accurate predictions.

scholarly journals Patch-Based Change Detection Method for SAR Images with Label Updating Strategy

2021 ◽  
Vol 13 (7) ◽  
pp. 1236
Author(s):  
Yuanjun Shu ◽  
Wei Li ◽  
Menglong Yang ◽  
Peng Cheng ◽  
Songchen Han
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Change Detection ◽  
Detection Method ◽  
Training Data ◽  
Synthetic Aperture ◽  
Learning Method ◽  
Sar Images ◽  
Two Stage ◽  
Second Stage

Convolutional neural networks (CNNs) have been widely used in change detection of synthetic aperture radar (SAR) images and have been proven to have better precision than traditional methods. A two-stage patch-based deep learning method with a label updating strategy is proposed in this paper. The initial label and mask are generated at the pre-classification stage. Then a two-stage updating strategy is applied to gradually recover changed areas. At the first stage, diversity of training data is gradually restored. The output of the designed CNN network is further processed to generate a new label and a new mask for the following learning iteration. As the diversity of data is ensured after the first stage, pixels within uncertain areas can be easily classified at the second stage. Experiment results on several representative datasets show the effectiveness of our proposed method compared with several existing competitive methods.

Deep Learning based Tomato’s Ripe and Unripe Classification System

2022 ◽  
Vol 10 (1) ◽  
pp. 0-0
Keyword(s):  
Deep Learning ◽  
Classification Accuracy ◽  
Ccd Camera ◽  
Agricultural Products ◽  
Training Data ◽  
Maturity Level ◽  
Agriculture Sector ◽  
The Past ◽  
State Of Art

Effective productivity estimates of fresh produced crops are very essential for efficient farming, commercial planning, and logistical support. In the past ten years, machine learning (ML) algorithms have been widely used for grading and classification of agricultural products in agriculture sector. However, the precise and accurate assessment of the maturity level of tomatoes using ML algorithms is still a quite challenging to achieve due to these algorithms being reliant on hand crafted features. Hence, in this paper we propose a deep learning based tomato maturity grading system that helps to increase the accuracy and adaptability of maturity grading tasks with less amount of training data. The performance of proposed system is assessed on the real tomato datasets collected from the open fields using Nikon D3500 CCD camera. The proposed approach achieved an average maturity classification accuracy of 99.8 % which seems to be quite promising in comparison to the other state of art methods.

Understanding Memories of the Past in the Context of Different Complex Neural Network Architectures

2022 ◽  
pp. 1-27
Author(s):  
Clifford Bohm ◽  
Douglas Kirkpatrick ◽  
Arend Hintze
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Mental Representations ◽  
Network Architectures ◽  
Information Theoretic ◽  
Time Points ◽  
The Past ◽  
Black Boxes ◽  
Past Experiences ◽  
Computational Systems

Abstract Deep learning (primarily using backpropagation) and neuroevolution are the preeminent methods of optimizing artificial neural networks. However, they often create black boxes that are as hard to understand as the natural brains they seek to mimic. Previous work has identified an information-theoretic tool, referred to as R, which allows us to quantify and identify mental representations in artificial cognitive systems. The use of such measures has allowed us to make previous black boxes more transparent. Here we extend R to not only identify where complex computational systems store memory about their environment but also to differentiate between different time points in the past. We show how this extended measure can identify the location of memory related to past experiences in neural networks optimized by deep learning as well as a genetic algorithm.

scholarly journals Augmented Data Selector to Initiate Text-Based CAPTCHA Attack

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Aolin Che ◽  
Yalin Liu ◽  
Hong Xiao ◽  
Hao Wang ◽  
Ke Zhang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Training Data ◽  
Accuracy Rate ◽  
Data Set ◽  
Research Directions ◽  
Training Models ◽  
The Past ◽  
Future Work ◽  
Design Cost ◽  
Accuracy Rates

In the past decades, due to the low design cost and easy maintenance, text-based CAPTCHAs have been extensively used in constructing security mechanisms for user authentications. With the recent advances in machine/deep learning in recognizing CAPTCHA images, growing attack methods are presented to break text-based CAPTCHAs. These machine learning/deep learning-based attacks often rely on training models on massive volumes of training data. The poorly constructed CAPTCHA data also leads to low accuracy of attacks. To investigate this issue, we propose a simple, generic, and effective preprocessing approach to filter and enhance the original CAPTCHA data set so as to improve the accuracy of the previous attack methods. In particular, the proposed preprocessing approach consists of a data selector and a data augmentor. The data selector can automatically filter out a training data set with training significance. Meanwhile, the data augmentor uses four different image noises to generate different CAPTCHA images. The well-constructed CAPTCHA data set can better train deep learning models to further improve the accuracy rate. Extensive experiments demonstrate that the accuracy rates of five commonly used attack methods after combining our preprocessing approach are 2.62% to 8.31% higher than those without preprocessing approach. Moreover, we also discuss potential research directions for future work.

Uncertainty quantification in fault detection using convolutional neural networks

Geophysics ◽  
2021 ◽  
pp. 1-45
Author(s):  
Runhai Feng ◽  
Dario Grana ◽  
Niels Balling
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Reservoir Characterization ◽  
Epistemic Uncertainty ◽  
Confidence Regions ◽  
Training Data ◽  
Model Parameters ◽  
Sigmoid Function ◽  
Learning Tools

Segmentation of faults based on seismic images is an important step in reservoir characterization. With the recent developments of deep-learning methods and the availability of massive computing power, automatic interpretation of seismic faults has become possible. The likelihood of occurrence for a fault can be quantified using a sigmoid function. Our goal is to quantify the fault model uncertainty that is generally not captured by deep-learning tools. We propose to use the dropout approach, a regularization technique to prevent overfitting and co-adaptation in hidden units, to approximate the Bayesian inference and estimate the principled uncertainty over functions. Particularly, the variance of the learned model has been decomposed into aleatoric and epistemic parts. The proposed method is applied to a real dataset from the Netherlands F3 block with two different dropout ratios in convolutional neural networks. The aleatoric uncertainty is irreducible since it relates to the stochastic dependency within the input observations. As the number of Monte-Carlo realizations increases, the epistemic uncertainty asymptotically converges and the model standard deviation decreases, because the variability of model parameters is better simulated or explained with a larger sample size. This analysis can quantify the confidence to use fault predictions with less uncertainty. Additionally, the analysis suggests where more training data are needed to reduce the uncertainty in low confidence regions.

Convolutional Neural Network

2022 ◽  
pp. 1559-1575
Author(s):  
Mário Pereira Véstias
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Machine Learning Model ◽  
Artificial Neural

Machine learning is the study of algorithms and models for computing systems to do tasks based on pattern identification and inference. When it is difficult or infeasible to develop an algorithm to do a particular task, machine learning algorithms can provide an output based on previous training data. A well-known machine learning model is deep learning. The most recent deep learning models are based on artificial neural networks (ANN). There exist several types of artificial neural networks including the feedforward neural network, the Kohonen self-organizing neural network, the recurrent neural network, the convolutional neural network, the modular neural network, among others. This article focuses on convolutional neural networks with a description of the model, the training and inference processes and its applicability. It will also give an overview of the most used CNN models and what to expect from the next generation of CNN models.

scholarly journals Application of convolutional neural networks for stellar spectral classification

2019 ◽  
Vol 491 (2) ◽  
pp. 2280-2300 ◽  
Author(s):  
Kaushal Sharma ◽  
Ajit Kembhavi ◽  
Aniruddha Kembhavi ◽  
T Sivarani ◽  
Sheelu Abraham ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Spectral Classification ◽  
Stellar Spectra ◽  
The Past ◽  
Training Samples ◽  
Optical Region ◽  
Spectral Libraries ◽  
Improved Accuracy

ABSTRACT Due to the ever-expanding volume of observed spectroscopic data from surveys such as SDSS and LAMOST, it has become important to apply artificial intelligence (AI) techniques for analysing stellar spectra to solve spectral classification and regression problems like the determination of stellar atmospheric parameters Teff, $\rm {\log g}$, and [Fe/H]. We propose an automated approach for the classification of stellar spectra in the optical region using convolutional neural networks (CNNs). Traditional machine learning (ML) methods with ‘shallow’ architecture (usually up to two hidden layers) have been trained for these purposes in the past. However, deep learning methods with a larger number of hidden layers allow the use of finer details in the spectrum which results in improved accuracy and better generalization. Studying finer spectral signatures also enables us to determine accurate differential stellar parameters and find rare objects. We examine various machine and deep learning algorithms like artificial neural networks, Random Forest, and CNN to classify stellar spectra using the Jacoby Atlas, ELODIE, and MILES spectral libraries as training samples. We test the performance of the trained networks on the Indo-U.S. Library of Coudé Feed Stellar Spectra (CFLIB). We show that using CNNs, we are able to lower the error up to 1.23 spectral subclasses as compared to that of two subclasses achieved in the past studies with ML approach. We further apply the trained model to classify stellar spectra retrieved from the SDSS data base with SNR > 20.

Sign in / Sign up

Export Citation Format

Share Document