scholarly journals Research on Network Layer Recursive Reduction Model Compression for Image Recognition

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Hongfei Ling ◽  
Weiwei Zhang ◽  
Yingjie Tao ◽  
Mi Zhou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Model ◽  
Computational Cost ◽  
Network Size ◽  
Compression Technique ◽  
Network Layer ◽  
Network Layers ◽  
Model Compression ◽  
The Neural Network

ResNet has been widely used in the field of machine learning since it was proposed. This network model is successful in extracting features from input data by superimposing multiple layers of neural networks and thus achieves high accuracy in many applications. However, the superposition of multilayer neural networks increases their computational cost. For this reason, we propose a network model compression technique that removes multiple neural network layers from ResNet without decreasing the accuracy rate. The key idea is to provide a priority term to identify the importance of each neural network layer, and then select the unimportant layers to be removed during the training process based on the priority of the neural network layers. In addition, this paper also retrains the network model to avoid the accuracy degradation caused by the deletion of network layers. Experiments demonstrate that the network size can be reduced by 24.00%–42.86% of the number of layers without reducing the classification accuracy when classification is performed on CIFAR-10/100 and ImageNet.

Credit Scoring Using Supervised and Unsupervised Neural Networks

2002 ◽  
pp. 154-166 ◽  
Author(s):  
David West ◽  
Cornelius Muchineuta
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Decision Support ◽  
Network Model ◽  
Neural Network Model ◽  
Credit Scoring ◽  
Self Organizing Map ◽  
Subprime Lending ◽  
The Neural Network ◽  
Combined Use

Some of the concerns that plague developers of neural network decision support systems include: (a) How do I understand the underlying structure of the problem domain; (b) How can I discover unknown imperfections in the data which might detract from the generalization accuracy of the neural network model; and (c) What variables should I include to obtain the best generalization properties in the neural network model? In this paper we explore the combined use of unsupervised and supervised neural networks to address these concerns. We develop and test a credit-scoring application using a self-organizing map and a multilayered feedforward neural network. The final product is a neural network decision support system that facilitates subprime lending and is flexible and adaptive to the needs of e-commerce applications.

APPLYING NEURAL NETWORKS TO SOFTWARE RELIABILITY ASSESSMENT

2010 ◽  
Vol 17 (04) ◽  
pp. 313-329 ◽  
Author(s):  
NORMAN SCHNEIDEWIND
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Model ◽  
Neural Network Model ◽  
Software Reliability ◽  
Likelihood Estimation ◽  
Time To Failure ◽  
The Neural Network ◽  
Method Of Maximum Likelihood ◽  
Network Method

We adapt concepts from the field of neural networks to assess the reliability of software, employing cumulative failures, reliability, remaining failures, and time to failure metrics. In addition, the risk of not achieving reliability, remaining failure, and time to failure goals are assessed. The purpose of the assessment is to compare a criterion, derived from a neural network model, for estimating the parameters of software reliability metrics, with the method of maximum likelihood estimation. To our surprise the neural network method proved superior for all the reliability metrics that were assessed by virtue of yielding lower prediction error and risk. We also found that considerable adaptation of the neural network model was necessary to be meaningful for our application – only inputs, functions, neurons, weights, activation units, and outputs were required to characterize our application.

scholarly journals Neural network method as means of processing experimental data on grain crop yields

2020 ◽  
Vol 161 ◽  
pp. 01031
Author(s):  
Aleksandr Nikiforov ◽  
Aleksei Kuchumov ◽  
Sergei Terentev ◽  
Inessa Karamulina ◽  
Iraida Romanova ◽  
...  
Keyword(s):  
Neural Network ◽  
Experimental Data ◽  
Neural Networks ◽  
Network Model ◽  
Neural Network Model ◽  
Crop Yields ◽  
Winter Triticale ◽  
The Neural Network ◽  
Grain Crop ◽  
Scientific Results

In the work based on agroecological and technological testing of varieties of grain crops of domestic and foreign breeding, winter triticale in particular, conducted on the experimental field of the Smolensk State Agricultural Academy between 2015 and 2019, we present the methodology and results of processing the experimental data used for constructing the neural network model. Neural networks are applicable for solving tasks that are difficult for computers of traditional design and humans alike. Those are processing large volumes of experimental data, automation of image recognition, approximation of functions and prognosis. Neural networks include analyzing subject areas and weight coefficients of neurons, detecting conflict samples and outliers, normalizing data, determining the number of samples required for teaching a neural network and increasing the learning quality when their number is insufficient, as well as selecting the neural network type and decomposition based on the number of input neurons. We consider the technology of initial data processing and selecting the optimal neural network structure that allows to significantly reduce modeling errors in comparison with neural networks created with unprepared source data. Our accumulated experience of working with neural networks has demonstrated encouraging results, which indicates the prospects of this area, especially when describing processes with large amounts of variables. In order to verify the resulting neural network model, we have carried out a computational experiment, which showed the possibility of applying scientific results in practice.

scholarly journals Modeling Exchange Rates With Neural Networks

2011 ◽  
Vol 14 (1) ◽  
pp. 1 ◽  
Author(s):  
A. M. M. Jamal ◽  
Cuddalore Sundar
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Exchange Rates ◽  
Network Model ◽  
Neural Network Model ◽  
Econometric Model ◽  
Short Term ◽  
Bilateral Exchange ◽  
The Neural Network ◽  
The U.S

<span>This paper applies the neural network model to forecast bilateral exchange rates between the U.S. and Germany and U.S. and France. The predictions from the neural network model were compared to those based on a standard econometric model. The results suggest that the neural network model may have some advantages when frequent short term forecasts are needed.</span>

scholarly journals Nuclear mass systematics by complementing the Finite Range Droplet Model with neural networks

2019 ◽  
Vol 14 ◽  
pp. 65
Author(s):  
S. Athanassopoulos ◽  
E. Mavrommatis ◽  
K. A. Gernoth ◽  
J. W. Clark
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Model ◽  
Neural Network Model ◽  
Predictive Performance ◽  
Finite Range ◽  
Mass Excess ◽  
Nuclear Mass ◽  
Droplet Model ◽  
The Neural Network

A neural-network model is developed to reproduce the differences between experimental nuclear mass-excess values and the theoretical values given by the Finite Range Droplet Model. The results point to the existence of subtle regularities of nuclear structure not yet contained in the best microscopic/phenomenological models of atomic masses. Combining the FRDM and the neural-network model, we create a hybrid model with improved predictive performance on nuclear-mass systematics and related quantities.

scholarly journals A Novel Low-Bit Quantization Strategy for Compressing Deep Neural Networks

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Xin Long ◽  
XiangRong Zeng ◽  
Zongcheng Ben ◽  
Dianle Zhou ◽  
Maojun Zhang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Network Models ◽  
Memory Consumption ◽  
Neural Network Models ◽  
The Neural Network ◽  
The Neural Networks ◽  
Novel Strategy

The increase in sophistication of neural network models in recent years has exponentially expanded memory consumption and computational cost, thereby hindering their applications on ASIC, FPGA, and other mobile devices. Therefore, compressing and accelerating the neural networks are necessary. In this study, we introduce a novel strategy to train low-bit networks with weights and activations quantized by several bits and address two corresponding fundamental issues. One is to approximate activations through low-bit discretization for decreasing network computational cost and dot-product memory. The other is to specify weight quantization and update mechanism for discrete weights to avoid gradient mismatch. With quantized low-bit weights and activations, the costly full-precision operation will be replaced by shift operation. We evaluate the proposed method on common datasets, and results show that this method can dramatically compress the neural network with slight accuracy loss.

scholarly journals Neural network model for recognition of driving strategies and interaction of drivers in traffic conditions

2018 ◽  
Vol 9 (4) ◽  
pp. 153-166
Author(s):  
S.B. Efremov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Traffic Flow ◽  
Network Model ◽  
Neural Network Model ◽  
Self Organizing Map ◽  
Flow Conditions ◽  
Traffic Conditions ◽  
Dangerous Driving ◽  
The Neural Network

The paper presents a neural network model for recognizing driving strategies based on the interaction of drivers in traffic flow conditions. The architecture of the model, based on self-organizing map (SOM), consisting of various neural networks based on RBF (Radial Basis Function). The purpose of this work is to describe the architecture and structure of the neural network model, which allows to recognize the strategic features of driving. Our neural network is able to identify the interaction strategies of cars (drivers) in traffic flow conditions, as well as to identify such behavioral patterns of movement that can be correlated with different types of dangerous driving. From the results of the study, it follows that neural networks of the SOM RBF type are able to recognize and classify the types of interactions in traffic conditions based on modeling the analysis of the trajectories of cars. This neural network showed a high percentage of recognition and clear clustering of similar driving strategies.

scholarly journals Improving prediction of neural networks: a study of tow financial prediction tasks

2004 ◽  
Vol 8 (4) ◽  
pp. 219-233
Author(s):  
Tarun K. Sen ◽  
Parviz Ghandforoush ◽  
Charles T. Stivason
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Model ◽  
Neural Network Model ◽  
Prediction Models ◽  
Prediction Performance ◽  
Financial Prediction ◽  
The Neural Network ◽  
Input Variables ◽  
Relationship Of

Neural networks are excellent mapping tools for complex financial data. Their mapping capabilities however do not always result in good generalizability for financial prediction models. Increasing the number of nodes and hidden layers in a neural network model produces better mapping of the data since the number of parameters available to the model increases. This is determinal to generalizabilitiy of the model since the model memorizes idiosyncratic patterns in the data. A neural network model can be expected to be more generalizable if the model architecture is made less complex by using fewer input nodes. In this study we simplify the neural network by eliminating input nodes that have the least contribution to the prediction of a desired outcome. We also provide a theoretical relationship of the sensitivity of output variables to the input variables under certain conditions. This research initiates an effort in identifying methods that would improve the generalizability of neural networks in financial prediction tasks by using mergers and bankruptcy models. The result indicates that incorporating more variables that appear relevant in a model does not necessarily improve prediction performance.

COMBINING REGRESSION ESTIMATORS: GA-BASED SELECTIVE NEURAL NETWORK ENSEMBLE

2001 ◽  
Vol 01 (04) ◽  
pp. 341-356 ◽  
Author(s):  
ZHI-HUA ZHOU ◽  
JIAN-XIN WU ◽  
WEI TANG ◽  
ZHAO-QIAN CHEN
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Computational Cost ◽  
Neural Network Ensemble ◽  
Generalization Ability ◽  
Ensemble Approach ◽  
The Neural Network ◽  
Random Weights ◽  
Regression Estimators ◽  
The Individual

Neural network ensemble is a learning paradigm where a collection of neural networks is trained for the same task. In this paper, the relationship between the generalization ability of the neural network ensemble and the correlation of the individual neural networks constituting the ensemble is analyzed in the context of combining neural regression estimators, which reveals that ensembling a selective subset of trained networks is superior to ensembling all the trained networks in some cases. Based on such recognition, an approach named GASEN is proposed. GASEN trains a number of individual neural networks at first. Then it assigns random weights to the individual networks and employs a genetic algorithm to evolve those weights so that they can characterize to some extent the importance of the individual networks in constituting an ensemble. Finally it selects an optimum subset of individual networks based on the evolved weights to make up the ensemble. Experimental results show that, comparing with a popular ensemble approach, i.e., averaging all, and a theoretically optimum selective ensemble approach, i.e. enumerating, GASEN has preferable performance in generating ensembles with strong generalization ability in relatively small computational cost. This paper also analyzes the working mechanism of GASEN from the view of error-ambiguity decomposition, which reveals that GASEN improves generalization ability mainly through reducing the average generalization error of the individual neural networks constituting the ensemble.

scholarly journals Regularity Normalization: Neuroscience-Inspired Unsupervised Attention across Neural Network Layers

Entropy ◽  
2021 ◽  
Vol 24 (1) ◽  
pp. 59
Author(s):  
Baihan Lin
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Question Answering ◽  
Minimum Description Length ◽  
Neuronal Firing ◽  
Universal Code ◽  
Network Layers ◽  
Generative Modeling ◽  
The Neural Network ◽  
Partially Observable

Inspired by the adaptation phenomenon of neuronal firing, we propose the regularity normalization (RN) as an unsupervised attention mechanism (UAM) which computes the statistical regularity in the implicit space of neural networks under the Minimum Description Length (MDL) principle. Treating the neural network optimization process as a partially observable model selection problem, the regularity normalization constrains the implicit space by a normalization factor, the universal code length. We compute this universal code incrementally across neural network layers and demonstrate the flexibility to include data priors such as top-down attention and other oracle information. Empirically, our approach outperforms existing normalization methods in tackling limited, imbalanced and non-stationary input distribution in image classification, classic control, procedurally-generated reinforcement learning, generative modeling, handwriting generation and question answering tasks with various neural network architectures. Lastly, the unsupervised attention mechanisms is a useful probing tool for neural networks by tracking the dependency and critical learning stages across layers and recurrent time steps of deep networks.

Sign in / Sign up

Export Citation Format

Share Document