scholarly journals ALGORITHMIC STABILITY OF DEEP LEARNING NEURAL NETWORKS IN RECOGNIZING THE MICROSTRUCTURE OF MATERIALS

2021 ◽  
Vol 21 (1) ◽  
pp. 159-166
Author(s):  
R.A. Klestov ◽  
◽  
A.V. Klyuev ◽  
V.Yu. Stolbov ◽  
◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Learning Process ◽  
Functional Materials ◽  
Quality Of Data ◽  
Algorithmic Stability ◽  
The Stability

The division of data for training a neural network into training and test data in various proportions to each other is investigated. The question is raised about how the quality of data distribution and their correct annotation can affect the final result of constructing a neural network model. The paper investigates the algorithmic stability of training a deep neural network in problems of recognition of the microstructure of materials. The study of the stability of the learning process makes it possible to estimate the performance of a neural network model on incomplete data distorted by up to 10%. Purpose. Research of the stability of the learning process of a neural network in the classification of microstructures of functional materials. Materials and methods. Artificial neural network is the main instrument on the basis of which produced the study. Different subtypes of deep convolutional networks are used such as VGG and ResNet. Neural networks are trained using an improved backpropagation method. The studied model is the frozen state of the neural network after a certain number of learning epochs. The amount of data excluded from the study was randomly distributed for each class in five different distributions. Results. Investigated neural network learning process. Results of experiments conducted computing training with gradual decrease in the number of input data. Distortions of calculation results when changing data with a step of 2 percent are investigated. The percentage of deviation was revealed, equal to 10, at which the trained neural network model loses its stability. Conclusion. The results obtained mean that with an established quantitative or qualitative deviation in the training or test set, the results obtained by training the network can hardly be trusted. Although the results of this study are applicable to a particular case, i.e., microstructure recognition problems using ResNet-152, the authors propose a simpler technique for studying the stability of deep learning neural networks based on the analysis of a test, not a training set.

DYNAMIC PROPERTIES OF A BIOLOGICALLY MOTIVATED NEURAL NETWORK MODEL

1992 ◽  
Vol 03 (04) ◽  
pp. 371-378 ◽  
Author(s):  
FRANÇOIS CHAPEAU BLONDEAU ◽  
GILBERT CHAUVET
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Transfer Function ◽  
Network Model ◽  
Neural Network Model ◽  
Dynamic Properties ◽  
Biological Significance ◽  
Neural Architecture ◽  
The Stability ◽  
Basic Features

We develop a neural network model based on prominent basic features of biological neural networks. The description keeps a simple but coherent link between the subneuronal, neuronal and network levels. In addition, the variables of the model are endowed with realistic numerical values together with their physical units. This permits to reach quantitative significance for the results. To describe the operation of the neuron, a transfer function is used that is believed to convey more biological significance compared to the usual sigmoid transfer function. It is shown that the dynamic properties of the network, which can vary from stability to chaos, are significantly influenced by the choice of the neuron transfer function. Constraints on the synaptic efficacies, as imposed by Dale’s rule, are also shown to modify the dynamic properties by increasing the stability of the network. A simple neural architecture is presented that leads to a controllable time evolution of the network activities.

scholarly journals Neural Reversible Steganography with Long Short-Term Memory

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Ching-Chun Chang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Low Level ◽  
Long Short Term Memory

Deep learning has brought about a phenomenal paradigm shift in digital steganography. However, there is as yet no consensus on the use of deep neural networks in reversible steganography, a class of steganographic methods that permits the distortion caused by message embedding to be removed. The underdevelopment of the field of reversible steganography with deep learning can be attributed to the perception that perfect reversal of steganographic distortion seems scarcely achievable, due to the lack of transparency and interpretability of neural networks. Rather than employing neural networks in the coding module of a reversible steganographic scheme, we instead apply them to an analytics module that exploits data redundancy to maximise steganographic capacity. State-of-the-art reversible steganographic schemes for digital images are based primarily on a histogram-shifting method in which the analytics module is often modelled as a pixel intensity predictor. In this paper, we propose to refine the prior estimation from a conventional linear predictor through a neural network model. The refinement can be to some extent viewed as a low-level vision task (e.g., noise reduction and super-resolution imaging). In this way, we explore a leading-edge neuroscience-inspired low-level vision model based on long short-term memory with a brief discussion of its biological plausibility. Experimental results demonstrated a significant boost contributed by the neural network model in terms of prediction accuracy and steganographic rate-distortion performance.

scholarly journals ENNGene: an Easy Neural Network model building tool for Genomics

2021 ◽  
Author(s):  
Eliska Chalupova ◽  
Ondrej Vaculik ◽  
Filip Jozefov ◽  
Jakub Polacek ◽  
Tomas Majtner ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Big Data ◽  
Deep Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Model Building ◽  
Evolutionary Conservation ◽  
Learning Methods

Background: The recent big data revolution in Genomics, coupled with the emergence of Deep Learning as a set of powerful machine learning methods, has shifted the standard practices of machine learning for Genomics. Even though Deep Learning methods such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are becoming widespread in Genomics, developing and training such models is outside the ability of most researchers in the field. Results: Here we present ENNGene - Easy Neural Network model building tool for Genomics. This tool simplifies training of custom CNN or hybrid CNN-RNN models on genomic data via an easy-to-use Graphical User Interface. ENNGene allows multiple input branches, including sequence, evolutionary conservation, and secondary structure, and performs all the necessary preprocessing steps, allowing simple input such as genomic coordinates. The network architecture is selected and fully customized by the user, from the number and types of the layers to each layer's precise set-up. ENNGene then deals with all steps of training and evaluation of the model, exporting valuable metrics such as multi-class ROC and precision-recall curve plots or TensorBoard log files. To facilitate interpretation of the predicted results, we deploy Integrated Gradients, providing the user with a graphical representation of an attribution level of each input position. To showcase the usage of ENNGene, we train multiple models on the RBP24 dataset, quickly reaching the state of the art while improving the performance on more than half of the proteins by including the evolutionary conservation score and tuning the network per protein. Conclusions: As the role of DL in big data analysis in the near future is indisputable, it is important to make it available for a broader range of researchers. We believe that an easy-to-use tool such as ENNGene can allow Genomics researchers without a background in Computational Sciences to harness the power of DL to gain better insights into and extract important information from the large amounts of data available in the field.

scholarly journals RISC-V Virtual Platform-Based Convolutional Neural Network Accelerator Implemented in SystemC

Electronics ◽  
2021 ◽  
Vol 10 (13) ◽  
pp. 1514
Author(s):  
Seung-Ho Lim ◽  
WoonSik William Suh ◽  
Jin-Young Kim ◽  
Sang-Young Cho
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Deep Neural Network ◽  
System Level ◽  
Neural Network Models ◽  
Data Set ◽  
Embedded Device ◽  
Virtual Platform

The optimization for hardware processor and system for performing deep learning operations such as Convolutional Neural Networks (CNN) in resource limited embedded devices are recent active research area. In order to perform an optimized deep neural network model using the limited computational unit and memory of an embedded device, it is necessary to quickly apply various configurations of hardware modules to various deep neural network models and find the optimal combination. The Electronic System Level (ESL) Simulator based on SystemC is very useful for rapid hardware modeling and verification. In this paper, we designed and implemented a Deep Learning Accelerator (DLA) that performs Deep Neural Network (DNN) operation based on the RISC-V Virtual Platform implemented in SystemC in order to enable rapid and diverse analysis of deep learning operations in an embedded device based on the RISC-V processor, which is a recently emerging embedded processor. The developed RISC-V based DLA prototype can analyze the hardware requirements according to the CNN data set through the configuration of the CNN DLA architecture, and it is possible to run RISC-V compiled software on the platform, can perform a real neural network model like Darknet. We performed the Darknet CNN model on the developed DLA prototype, and confirmed that computational overhead and inference errors can be analyzed with the DLA prototype developed by analyzing the DLA architecture for various data sets.

Deep learning model for protein disease classification

2021 ◽  
Vol 16 ◽  
Author(s):  
Farida Alaaeldin Mostafa ◽  
Yasmine Mohamed Afify ◽  
Rasha Mohamed Ismail ◽  
Nagwa Lotfy Badr
Keyword(s):  
Neural Network ◽  
Support Vector Machine ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Protein Analysis ◽  
Kernel Functions ◽  
Support Vector ◽  
Feature Descriptors

Background: Protein sequence analysis helps in the prediction of protein functions. As the number of proteins increases, it gives the bioinformaticians a challenge to analyze and study the similarity between them. Most of the existing protein analysis methods use Support Vector Machine. Deep learning did not receive much attention regarding protein analysis as it is noted that little work focused on studying the protein diseases classification. Objective: The contribution of this paper is to present a deep learning approach that classifies protein diseases based on protein descriptors. Methods: Different protein descriptors are used and decomposed into modified feature descriptors. Uniquely, we introduce using Convolutional Neural Network model to learn and classify protein diseases. The modified feature descriptors are fed to the Convolutional Neural Network model on a dataset of 1563 protein sequences classified into 3 different disease classes: Aids, Tumor suppressor, and Proto oncogene. Results: The usage of the modified feature descriptors shows a significant increase in the performance of the Convolutional Neural Network model over Support Vector Machine using different kernel functions. One modified feature descriptor improved by 19.8%, 27.9%, 17.6%, 21.5%, 17.3%, and 22% for evaluation metrics: Area Under the Curve, Matthews Correlation Coefficient, Accuracy, F1-score, Recall, and Precision, respectively. Conclusion: Results show that the prediction of the proposed modified feature descriptors significantly surpasses that of Support Vector Machine model.

scholarly journals Dynamic versus static neural network model for rainfall forecasting at Klang River Basin, Malaysia

2012 ◽  
Vol 16 (4) ◽  
pp. 1151-1169 ◽  
Author(s):  
A. El-Shafie ◽  
A. Noureldin ◽  
M. Taha ◽  
A. Hussain ◽  
M. Mukhlisin
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Rainfall Time Series ◽  
Hydrological Process ◽  
Multi Layer Perceptron ◽  
Rainfall Forecasting ◽  
Unseen Data

Abstract. Rainfall is considered as one of the major components of the hydrological process; it takes significant part in evaluating drought and flooding events. Therefore, it is important to have an accurate model for rainfall forecasting. Recently, several data-driven modeling approaches have been investigated to perform such forecasting tasks as multi-layer perceptron neural networks (MLP-NN). In fact, the rainfall time series modeling involves an important temporal dimension. On the other hand, the classical MLP-NN is a static and has a memoryless network architecture that is effective for complex nonlinear static mapping. This research focuses on investigating the potential of introducing a neural network that could address the temporal relationships of the rainfall series. Two different static neural networks and one dynamic neural network, namely the multi-layer perceptron neural network (MLP-NN), radial basis function neural network (RBFNN) and input delay neural network (IDNN), respectively, have been examined in this study. Those models had been developed for the two time horizons for monthly and weekly rainfall forecasting at Klang River, Malaysia. Data collected over 12 yr (1997–2008) on a weekly basis and 22 yr (1987–2008) on a monthly basis were used to develop and examine the performance of the proposed models. Comprehensive comparison analyses were carried out to evaluate the performance of the proposed static and dynamic neural networks. Results showed that the MLP-NN neural network model is able to follow trends of the actual rainfall, however, not very accurately. RBFNN model achieved better accuracy than the MLP-NN model. Moreover, the forecasting accuracy of the IDNN model was better than that of static network during both training and testing stages, which proves a consistent level of accuracy with seen and unseen data.

scholarly journals Nowcasting of Lumber Futures Price with Google Trends Index Using Machine Learning and Deep Learning Models

2021 ◽  
Vol 72 (1) ◽  
pp. 11-20
Author(s):  
Mingtao He ◽  
Wenying Li ◽  
Brian K. Via ◽  
Yaoqi Zhang
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Price Volatility ◽  
Google Trends ◽  
Learning Models ◽  
Fresh Perspective ◽  
Futures Price

Abstract Firms engaged in producing, processing, marketing, or using lumber and lumber products always invest in futures markets to reduce the risk of lumber price volatility. The accurate prediction of real-time prices can help companies and investors hedge risks and make correct market decisions. This paper explores whether Internet browsing habits can accurately nowcast the lumber futures price. The predictors are Google Trends index data related to lumber prices. This study offers a fresh perspective on nowcasting the lumber price accurately. The novel outlook of employing both machine learning and deep learning methods shows that despite the high predictive power of both the methods, on average, deep learning models can better capture trends and provide more accurate predictions than machine learning models. The artificial neural network model is the most competitive, followed by the recurrent neural network model.

scholarly journals Image Classification Using Transfer Learning and Deep Learning

2021 ◽  
Vol 10 (9) ◽  
pp. 25394-25398
Author(s):  
Chitra Desai
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Image Classification ◽  
Transfer Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Large Scale ◽  
Deep Neural Network ◽  
Visual Recognition ◽  
Classification Of Images

Deep learning models have demonstrated improved efficacy in image classification since the ImageNet Large Scale Visual Recognition Challenge started since 2010. Classification of images has further augmented in the field of computer vision with the dawn of transfer learning. To train a model on huge dataset demands huge computational resources and add a lot of cost to learning. Transfer learning allows to reduce on cost of learning and also help avoid reinventing the wheel. There are several pretrained models like VGG16, VGG19, ResNet50, Inceptionv3, EfficientNet etc which are widely used.   This paper demonstrates image classification using pretrained deep neural network model VGG16 which is trained on images from ImageNet dataset. After obtaining the convolutional base model, a new deep neural network model is built on top of it for image classification based on fully connected network. This classifier will use features extracted from the convolutional base model.

Sign in / Sign up

Export Citation Format

Share Document