scholarly journals Smooth Non-increasing Square Spatial Extents of Filters in Convolutional Layers of CNNs for Image Classification Problems

2018 ◽  
Vol 23 (1) ◽  
pp. 52-62
Author(s):  
Vadim V. Romanuke
Keyword(s):  
Neural Network ◽  
Image Classification ◽  
Network Architecture ◽  
Network Performance ◽  
Medium Size ◽  
Classification Problems ◽  
Layered Architecture ◽  
The Neural Network ◽  
Neural Network Classifiers ◽  
Secondary Condition

Abstract The present paper considers an open problem of setting hyperparameters for convolutional neural networks aimed at image classification. Since selecting filter spatial extents for convolutional layers is a topical problem, it is approximately solved by accumulating statistics of the neural network performance. The network architecture is taken on the basis of the MNIST database experience. The eight-layered architecture having four convolutional layers is nearly best suitable for classifying small and medium size images. Image databases are formed of grayscale images whose size range is 28 × 28 to 64 × 64 by step 2. Except for the filter spatial extents, the rest of those eight layer hyperparameters are unalterable, and they are chosen scrupulously based on rules of thumb. A sequence of possible filter spatial extents is generated for each size. Then sets of four filter spatial extents producing the best performance are extracted. The rule of this extraction that allows selecting the best filter spatial extents is formalized with two conditions. Mainly, difference between maximal and minimal extents must be as minimal as possible. No unit filter spatial extent is recommended. The secondary condition is that the filter spatial extents should constitute a non-increasing set. Validation on MNIST and CIFAR- 10 databases justifies such a solution, which can be extended for building convolutional neural network classifiers of colour and larger images.

scholarly journals DuFeNet: Improve the Accuracy and Increase Shape Bias of Neural Network Models

2022 ◽  
Author(s):  
Zecong Ye ◽  
Zhiqiang Gao ◽  
Xiaolong Cui ◽  
Yaojie Wang ◽  
Nanliang Shan
Keyword(s):  
Neural Network ◽  
Image Classification ◽  
Network Structure ◽  
Network Performance ◽  
Human Visual Perception ◽  
Neural Network Models ◽  
Bias Effect ◽  
Shape Information ◽  
Shape Bias ◽  
The Neural Network

AbstractIn image classification field, existing work tends to modify the network structure to obtain higher accuracy or faster speed. However, some studies have found that the neural network usually has texture bias effect, which means that the neural network is more sensitive to the texture information than the shape information. Based on such phenomenon, we propose a new way to improve network performance by making full use of gradient information. The dual features network (DuFeNet) is proposed in this paper. In DuFeNet, one sub-network is used to learn the information of gradient features, and the other is a traditional neural network with texture bias. The structure of DuFeNet is easy to implement in the original neural network structure. The experimental results clearly show that DuFeNet can achieve better accuracy in image classification and detection. It can increase the shape bias of the network adapted to human visual perception. Besides, DuFeNet can be used without modifying the structure of the original network at lower additional parameters cost.

SCORING MODELING BASED ON NEURAL NETWORKS FOR DETERMINING A BANK BORROWER'S RATING

2020 ◽  
Vol 2020 (10) ◽  
pp. 54-62
Author(s):  
Oleksii VASYLIEV ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Statistical Data ◽  
Activation Function ◽  
Decision Making Process ◽  
Neural Network Architecture ◽  
Acceptable Accuracy ◽  
The Neural Network ◽  
Sigmoid Activation Function

The problem of applying neural networks to calculate ratings used in banking in the decision-making process on granting or not granting loans to borrowers is considered. The task is to determine the rating function of the borrower based on a set of statistical data on the effectiveness of loans provided by the bank. When constructing a regression model to calculate the rating function, it is necessary to know its general form. If so, the task is to calculate the parameters that are included in the expression for the rating function. In contrast to this approach, in the case of using neural networks, there is no need to specify the general form for the rating function. Instead, certain neural network architecture is chosen and parameters are calculated for it on the basis of statistical data. Importantly, the same neural network architecture can be used to process different sets of statistical data. The disadvantages of using neural networks include the need to calculate a large number of parameters. There is also no universal algorithm that would determine the optimal neural network architecture. As an example of the use of neural networks to determine the borrower's rating, a model system is considered, in which the borrower's rating is determined by a known non-analytical rating function. A neural network with two inner layers, which contain, respectively, three and two neurons and have a sigmoid activation function, is used for modeling. It is shown that the use of the neural network allows restoring the borrower's rating function with quite acceptable accuracy.

Spatial Variability Aware Deep Neural Networks (SVANN): A General Approach

2021 ◽  
Vol 12 (6) ◽  
pp. 1-21
Author(s):  
Jayant Gupta ◽  
Carl Molnar ◽  
Yiqun Xie ◽  
Joe Knight ◽  
Shashi Shekhar
Keyword(s):  
Neural Network ◽  
Spatial Variability ◽  
Network Architecture ◽  
Network Models ◽  
Neural Network Architecture ◽  
Neural Network Models ◽  
Climatic Zones ◽  
The Neural Network ◽  
Plant Hardiness ◽  
Interpretation Model

Spatial variability is a prominent feature of various geographic phenomena such as climatic zones, USDA plant hardiness zones, and terrestrial habitat types (e.g., forest, grasslands, wetlands, and deserts). However, current deep learning methods follow a spatial-one-size-fits-all (OSFA) approach to train single deep neural network models that do not account for spatial variability. Quantification of spatial variability can be challenging due to the influence of many geophysical factors. In preliminary work, we proposed a spatial variability aware neural network (SVANN-I, formerly called SVANN ) approach where weights are a function of location but the neural network architecture is location independent. In this work, we explore a more flexible SVANN-E approach where neural network architecture varies across geographic locations. In addition, we provide a taxonomy of SVANN types and a physics inspired interpretation model. Experiments with aerial imagery based wetland mapping show that SVANN-I outperforms OSFA and SVANN-E performs the best of all.

An artificial neural network approach to discrete-event simulation

1995 ◽  
Vol 9 (1) ◽  
pp. 37-49 ◽  
Author(s):  
Ian Flood ◽  
Kenneth Worley
Keyword(s):  
Neural Network ◽  
Discrete Event Simulation ◽  
Network Architecture ◽  
Discrete Event ◽  
Network Approach ◽  
Neural Network Approach ◽  
Event Simulation ◽  
The Neural Network ◽  
Component Processes ◽  
And Performance

AbstractThis paper proposes and evaluates a neural network-based method for simulating manufacturing processes that exhibit both noncontinuous and stochastic behavior processes more conventionally modeled, using discrete-event simulation algorithms. The incentive for developing the technique is its potential for rapid execution of a simulation through parallel processing, and facilitation of the development and improvement of models particularly where there is limited theory describing the dependence between component processes. A brief introduction is provided to a radial-Gaussian neural network architecture and training process, the system adopted for the work presented in this paper. A description of the basic approach proposed for applying this technology to simulation is then described. This involves the use of a modularized neural network approach to model construction and the prediction of the occurrence of events using information retained from several previous states of the simulation. A class of earth-moving systems, comprising a push-dozer and a fleet of scrapers, is used as the basis for assessing the viability and performance of the proposed approach. A series of experiments show the neural network to be capable of both capturing the characteristic behavior and making an accurate prediction of production rates of scraper-based earth-moving systems. The paper concludes with an indication of some areas for further development and evaluation of the technique.

scholarly journals ON SOME BOUNDS OF THE MINIMUM EDGE DOMINATING ENERGY OF A GRAPH

2021 ◽  
Vol 12 (4) ◽  
pp. 1285-1296
Author(s):  
Dr. Gauri Ghule , Et. al.
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Fuzzy Controller ◽  
Back Propagation ◽  
Back Propagation Neural Network ◽  
Superior Performance ◽  
Fuzzy Classifier ◽  
Manual Intervention ◽  
The Neural Network ◽  
Hidden Neurons

Number of hidden neurons is necessary constant for tuning the neural network to achieve superior performance. These parameters are set manually through experimentation. The performance of the network is evaluated repeatedly to choose the best input parameters.Random selection of hidden neurons may cause underfitting or overfitting of the network. We propose a novel fuzzy controller for finding the optimal value of hidden neurons automatically. The hybrid classifier helps to design competent neural network architecture, eliminating manual intervention for setting the input parameters. The effectiveness of tuning the number of hidden neurons automatically on the convergence of a back-propagation neural network, is verified on speech data. The experimental outcomes demonstrate that the proposed Neuro-Fuzzy classifier can be viably utilized for speech recognition with maximum classification accuracy.

Enhancing Vibration-Based Structural Health Monitoring via Edge Computing: A Tiny Machine Learning Perspective

2021 ◽  
Author(s):  
Federica Zonzini ◽  
Francesca Romano ◽  
Antonio Carbone ◽  
Matteo Zauli ◽  
Luca De Marchi
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Structural Health Monitoring ◽  
Health Monitoring ◽  
Network Architecture ◽  
Binary Classification ◽  
Low Cost ◽  
Classification Problems ◽  
Computationally Efficient ◽  
Structural Health

Abstract Despite the outstanding improvements achieved by artificial intelligence in the Structural Health Monitoring (SHM) field, some challenges need to be coped with. Among them, the necessity to reduce the complexity of the models and the data-to-user latency time which are still affecting state-of-the-art solutions. This is due to the continuous forwarding of a huge amount of data to centralized servers, where the inference process is usually executed in a bulky manner. Conversely, the emerging field of Tiny Machine Learning (TinyML), promoted by the recent advancements by the electronic and information engineering community, made sensor-near data inference a tangible, low-cost and computationally efficient alternative. In line with this observation, this work explored the embodiment of the One Class Classifier Neural Network, i.e., a neural network architecture solving binary classification problems for vibration-based SHM scenarios, into a resource-constrained device. To this end, OCCNN has been ported on the Arduino Nano 33 BLE Sense platform and validated with experimental data from the Z24 bridge use case, reaching an average accuracy and precision of 95% and 94%, respectively.

Signal processing algorithm for neural networks with integrodifferential splines as an activation function and its particular case of image classification

2021 ◽  
Author(s):  
T.K. Biryukova
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Classification ◽  
Activation Function ◽  
Experimental Comparison ◽  
Training Time ◽  
Operation Speed ◽  
The Neural Network ◽  
Linear Algebraic Equations ◽  
Network Operation

Classic neural networks suppose trainable parameters to include just weights of neurons. This paper proposes parabolic integrodifferential splines (ID-splines), developed by author, as a new kind of activation function (AF) for neural networks, where ID-splines coefficients are also trainable parameters. Parameters of ID-spline AF together with weights of neurons are vary during the training in order to minimize the loss function thus reducing the training time and increasing the operation speed of the neural network. The newly developed algorithm enables software implementation of the ID-spline AF as a tool for neural networks construction, training and operation. It is proposed to use the same ID-spline AF for neurons in the same layer, but different for different layers. In this case, the parameters of the ID-spline AF for a particular layer change during the training process independently of the activation functions (AFs) of other network layers. In order to comply with the continuity condition for the derivative of the parabolic ID-spline on the interval (x x0, n) , its parameters fi (i= 0,...,n) should be calculated using the tridiagonal system of linear algebraic equations: To solve the system it is necessary to use two more equations arising from the boundary conditions for specific problems. For exam- ple the values of the grid function (if they are known) in the points (x x0, n) may be used for solving the system above: f f x0 = ( 0) , f f xn = ( n) . The parameters Iii+1 (i= 0,...,n−1 ) are used as trainable parameters of neural networks. The grid boundaries and spacing of the nodes of ID-spline AF are best chosen experimentally. The optimal selection of grid nodes allows improving the quality of results produced by the neural network. The formula for a parabolic ID-spline is such that the complexity of the calculations does not depend on whether the grid of nodes is uniform or non-uniform. An experimental comparison of the results of image classification from the popular FashionMNIST dataset by convolutional neural 0, x< 0 networks with the ID-spline AFs and the well-known ReLUx( ) =AF was carried out. The results reveal that the usage x x, ≥ 0 of the ID-spline AFs provides better accuracy of neural network operation than the ReLU AF. The training time for two convolutional layers network with two ID-spline AFs is just about 2 times longer than with two instances of ReLU AF. Doubling of the training time due to complexity of the ID-spline formula is the acceptable price for significantly better accuracy of the network. Wherein the difference of an operation speed of the networks with ID-spline and ReLU AFs will be negligible. The use of trainable ID-spline AFs makes it possible to simplify the architecture of neural networks without losing their efficiency. The modification of the well-known neural networks (ResNet etc.) by replacing traditional AFs with ID-spline AFs is a promising approach to increase the neural network operation accuracy. In a majority of cases, such a substitution does not require to train the network from scratch because it allows to use pre-trained on large datasets neuron weights supplied by standard software libraries for neural network construction thus substantially shortening training time.

Layer-Wise Tumor Segmentation of Breast Images Using Convolutional Neural Networks

2021 ◽  
pp. 70-84
Author(s):  
Nishanth Krishnaraj ◽  
A. Mary Mekala ◽  
Bhaskar M. ◽  
Ruban Nersisson ◽  
Alex Noel Joseph Raj
Keyword(s):  
Breast Cancer ◽  
Neural Network ◽  
Network Architecture ◽  
Breast Imaging ◽  
Breast Tissue ◽  
Imaging Techniques ◽  
Tumor Segmentation ◽  
Cancer Type ◽  
The Neural Network ◽  
Life Threatening

Early prediction of cancer type has become very crucial. Breast cancer is common to women and it leads to life threatening. Several imaging techniques have been suggested for timely detection and treatment of breast cancer. More research findings have been done to accurately detect the breast cancer. Automated whole breast ultrasound (AWBUS) is a new breast imaging technology that can render the entire breast anatomy in 3-D volume. The tissue layers in the breast are segmented and the type of lesion in the breast tissue can be identified which is essential for cancer detection. In this chapter, a u-net convolutional neural network architecture is used to implement the segmentation of breast tissues from AWBUS images into the different layers, that is, epidermis, subcutaneous, and muscular layer. The architecture was trained and tested with the AWBUS dataset images. The performance of the proposed scheme was based on accuracy, loss and the F1 score of the neural network that was calculated for each layer of the breast tissue.

Sign in / Sign up

Export Citation Format

Share Document