scholarly journals Utilization of the fractal dimension metric in training of dense Neural Networks

2021 ◽  
Author(s):  
Kalin Stoyanov ◽  
Jordan Hristov
Keyword(s):  
Neural Networks ◽  
Fractal Dimension ◽  
Regression Line ◽  
Training Data ◽  
Training Process ◽  
Fractal Structures ◽  
Data Set ◽  
Rigorous Approach ◽  
Recursive Procedures

The process of training of Artificial Neural Networks essentially is optimization of the values of the weights $ w_{pq} $ associated with the arcs, connecting the nodes of the layers. This is a process of minimization of the Loss function (maximization of Accuracy function). During the training, the training data set recursively is utilized at subsequent stages, called \textit{Epochs}. The training continues until a satisfactory values of the Loss, Accuracy etc. parameters are reached. The matrices $ W^{UV} $ comprising the weights of the arcs connecting the layers $ U $ and $ V $, can be regarded as gray-scale images of a surface. Starting as random matrices, processed by recursive procedures, they gradually become fractal structures, characterized with respective fractal dimension $ D_f $. In the presented article we have made an attempt to utilize the correspondence of $ D_f $ with the Loss/Accuracy values, in order to forecast the optimal ending point of the NN training process. Similar conclusions were made for the correspondence between the number of layer’s nodes and $ D_f $. An attempt to apply statistically more rigorous approach in the determination of the slope of the regression line in Richardson-Mandelbrot plot, was made.

Operational Determination of the Activated Sludge Process Using Neural Networks

1992 ◽  
Vol 26 (9-11) ◽  
pp. 2461-2464 ◽  
Author(s):  
R. D. Tyagi ◽  
Y. G. Du
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Steady State ◽  
Activated Sludge ◽  
Feedforward Neural Network ◽  
Training Data ◽  
Activated Sludge Process

A steady-statemathematical model of an activated sludgeprocess with a secondary settler was developed. With a limited number of training data samples obtained from the simulation at steady state, a feedforward neural network was established which exhibits an excellent capability for the operational prediction and determination.

scholarly journals Localization of Scattering Objects Using Neural Networks

Sensors ◽  
2020 ◽  
Vol 21 (1) ◽  
pp. 11
Author(s):  
Domonkos Haffner ◽  
Ferenc Izsák
Keyword(s):  
Neural Networks ◽  
Plane Waves ◽  
Measurement Data ◽  
Scattered Wave ◽  
Training Data ◽  
Data Set ◽  
Main Compound ◽  
Incident Plane ◽  
The Neural Networks ◽  
Simulation Package

The localization of multiple scattering objects is performed while using scattered waves. An up-to-date approach: neural networks are used to estimate the corresponding locations. In the scattering phenomenon under investigation, we assume known incident plane waves, fully reflecting balls with known diameters and measurement data of the scattered wave on one fixed segment. The training data are constructed while using the simulation package μ-diff in Matlab. The structure of the neural networks, which are widely used for similar purposes, is further developed. A complex locally connected layer is the main compound of the proposed setup. With this and an appropriate preprocessing of the training data set, the number of parameters can be kept at a relatively low level. As a result, using a relatively large training data set, the unknown locations of the objects can be estimated effectively.

scholarly journals Determination of Body Parts in Holstein Friesian Cows Comparing Neural Networks and k Nearest Neighbour Classification

Animals ◽  
2020 ◽  
Vol 11 (1) ◽  
pp. 50
Author(s):  
Jennifer Salau ◽  
Jan Henning Haas ◽  
Wolfgang Junge ◽  
Georg Thaller
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Body Parts ◽  
Nearest Neighbour ◽  
Data Set ◽  
3D Data ◽  
Holstein Friesian ◽  
Knn Classification ◽  
Friesian Cows

Machine learning methods have become increasingly important in animal science, and the success of an automated application using machine learning often depends on the right choice of method for the respective problem and data set. The recognition of objects in 3D data is still a widely studied topic and especially challenging when it comes to the partition of objects into predefined segments. In this study, two machine learning approaches were utilized for the recognition of body parts of dairy cows from 3D point clouds, i.e., sets of data points in space. The low cost off-the-shelf depth sensor Microsoft Kinect V1 has been used in various studies related to dairy cows. The 3D data were gathered from a multi-Kinect recording unit which was designed to record Holstein Friesian cows from both sides in free walking from three different camera positions. For the determination of the body parts head, rump, back, legs and udder, five properties of the pixels in the depth maps (row index, column index, depth value, variance, mean curvature) were used as features in the training data set. For each camera positions, a k nearest neighbour classifier and a neural network were trained and compared afterwards. Both methods showed small Hamming losses (between 0.007 and 0.027 for k nearest neighbour (kNN) classification and between 0.045 and 0.079 for neural networks) and could be considered successful regarding the classification of pixel to body parts. However, the kNN classifier was superior, reaching overall accuracies 0.888 to 0.976 varying with the camera position. Precision and recall values associated with individual body parts ranged from 0.84 to 1 and from 0.83 to 1, respectively. Once trained, kNN classification is at runtime prone to higher costs in terms of computational time and memory compared to the neural networks. The cost vs. accuracy ratio for each methodology needs to be taken into account in the decision of which method should be implemented in the application.

Prediction Models for Truck Accidents at Freeway Ramps in Washington State Using Regression and Artificial Intelligence Techniques

1998 ◽  
Vol 1635 (1) ◽  
pp. 30-36 ◽  
Author(s):  
Wael H. Awad ◽  
Bruce N. Janson
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Linear Regression ◽  
Prediction Models ◽  
Washington State ◽  
Training Data ◽  
Coefficient Of Determination ◽  
Training Process ◽  
Truck Accidents ◽  
High Level

Three different modeling approaches were applied to explain truck accidents at interchanges in Washington State during a 27-month period. Three models were developed for each ramp type including linear regression, neural networks, and a hybrid system using fuzzy logic and neural networks. The study showed that linear regression was able to predict accident frequencies that fell within one standard deviation from the overall mean of the dependent variable. However, the coefficient of determination was very low in all cases. The other two artificial intelligence (AI) approaches showed a high level of performance in identifying different patterns of accidents in the training data and presented a better fit when compared to the regression model. However, the ability of these AI models to predict test data that were not included in the training process showed unsatisfactory results.

scholarly journals Opportunities and Constraints in Applying Artificial Neural Networks (ANNs) in Food Authentication. Honey—A Case Study

2021 ◽  
Vol 11 (15) ◽  
pp. 6723
Author(s):  
Ariana Raluca Hategan ◽  
Romulus Puscas ◽  
Gabriela Cristea ◽  
Adriana Dehelean ◽  
Francois Guyon ◽  
...  
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Training Sample ◽  
Training Data ◽  
Elemental Content ◽  
Neural Structure ◽  
Food Authentication ◽  
Data Set ◽  
Artificial Neural

The present work aims to test the potential of the application of Artificial Neural Networks (ANNs) for food authentication. For this purpose, honey was chosen as the working matrix. The samples were originated from two countries: Romania (50) and France (53), having as floral origins: acacia, linden, honeydew, colza, galium verum, coriander, sunflower, thyme, raspberry, lavender and chestnut. The ANNs were built on the isotope and elemental content of the investigated honey samples. This approach conducted to the development of a prediction model for geographical recognition with an accuracy of 96%. Alongside this work, distinct models were developed and tested, with the aim of identifying the most suitable configurations for this application. In this regard, improvements have been continuously performed; the most important of them consisted in overcoming the unwanted phenomenon of over-fitting, observed for the training data set. This was achieved by identifying appropriate values for the number of iterations over the training data and for the size and number of the hidden layers and by introducing of a dropout layer in the configuration of the neural structure. As a conclusion, ANNs can be successfully applied in food authenticity control, but with a degree of caution with respect to the “over optimization” of the correct classification percentage for the training sample set, which can lead to an over-fitted model.

scholarly journals Modeling of daily runoff from a small agricultural watershed using artificial neural network with resampling techniques

2014 ◽  
Vol 17 (1) ◽  
pp. 56-74 ◽  
Author(s):  
Gurjeet Singh ◽  
Rabindra K. Panda ◽  
Marc Lamers
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Agricultural Watershed ◽  
Training Data ◽  
Optimum Number ◽  
Training Process ◽  
Data Set ◽  
Reported Study ◽  
Artificial Neural ◽  
Artificial Neural Network Ann

The reported study was undertaken in a small agricultural watershed, namely, Kapgari in Eastern India having a drainage area of 973 ha. The watershed was subdivided into three sub-watersheds on the basis of drainage network and land topography. An attempt was made to relate the continuously monitored runoff data from the sub-watersheds and the whole-watershed with the rainfall and temperature data using the artificial neural network (ANN) technique. The reported study also evaluated the bias in the prediction of daily runoff with shorter length of training data set using different resampling techniques with the ANN modeling. A 10-fold cross-validation (CV) technique was used to find the optimum number of hidden neurons in the hidden layer and to avoid neural network over-fitting during the training process for shorter length of data. The results illustrated that the ANN models developed with shorter length of training data set avoid neural network over-fitting during the training process, using a 10-fold CV method. Moreover, the biasness was investigated using the bootstrap resampling technique based ANN (BANN) for short length of training data set. In comparison with the 10-fold CV technique, the BANN is more efficient in solving the problems of the over-fitting and under-fitting during training of models for shorter length of data set.

Simple Convolutional-Based Models: Are They Learning the Task or the Data?

2021 ◽  
pp. 1-17
Author(s):  
Luis Sa-Couto ◽  
Andreas Wichert
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Training Data ◽  
Model Complexity ◽  
Data Sets ◽  
Simple Task ◽  
Data Set ◽  
Knowing That ◽  
Handwritten Digit ◽  
End To End

Abstract Convolutional neural networks (CNNs) evolved from Fukushima's neocognitron model, which is based on the ideas of Hubel and Wiesel about the early stages of the visual cortex. Unlike other branches of neocognitron-based models, the typical CNN is based on end-to-end supervised learning by backpropagation and removes the focus from built-in invariance mechanisms, using pooling not as a way to tolerate small shifts but as a regularization tool that decreases model complexity. These properties of end-to-end supervision and flexibility of structure allow the typical CNN to become highly tuned to the training data, leading to extremely high accuracies on typical visual pattern recognition data sets. However, in this work, we hypothesize that there is a flip side to this capability, a hidden overfitting. More concretely, a supervised, backpropagation based CNN will outperform a neocognitron/map transformation cascade (MTCCXC) when trained and tested inside the same data set. Yet if we take both models trained and test them on the same task but on another data set (without retraining), the overfitting appears. Other neocognitron descendants like the What-Where model go in a different direction. In these models, learning remains unsupervised, but more structure is added to capture invariance to typical changes. Knowing that, we further hypothesize that if we repeat the same experiments with this model, the lack of supervision may make it worse than the typical CNN inside the same data set, but the added structure will make it generalize even better to another one. To put our hypothesis to the test, we choose the simple task of handwritten digit classification and take two well-known data sets of it: MNIST and ETL-1. To try to make the two data sets as similar as possible, we experiment with several types of preprocessing. However, regardless of the type in question, the results align exactly with expectation.

scholarly journals Automatic Gully Detection: Neural Networks and Computer Vision

2020 ◽  
Vol 12 (11) ◽  
pp. 1743
Author(s):  
Artur M. Gafurov ◽  
Oleg P. Yermolayev
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Large Scale ◽  
Satellite Images ◽  
Gully Erosion ◽  
Training Data ◽  
Russian Plain ◽  
Data Set ◽  
High Resolution Satellite Images

Transition from manual (visual) interpretation to fully automated gully detection is an important task for quantitative assessment of modern gully erosion, especially when it comes to large mapping areas. Existing approaches to semi-automated gully detection are based on either object-oriented selection based on multispectral images or gully selection based on a probabilistic model obtained using digital elevation models (DEMs). These approaches cannot be used for the assessment of gully erosion on the territory of the European part of Russia most affected by gully erosion due to the lack of national large-scale DEM and limited resolution of open source multispectral satellite images. An approach based on the use of convolutional neural networks for automated gully detection on the RGB-synthesis of ultra-high resolution satellite images publicly available for the test region of the east of the Russian Plain with intensive basin erosion has been proposed and developed. The Keras library and U-Net architecture of convolutional neural networks were used for training. Preliminary results of application of the trained gully erosion convolutional neural network (GECNN) allow asserting that the algorithm performs well in detecting active gullies, well differentiates gullies from other linear forms of slope erosion — rills and balkas, but so far has errors in detecting complex gully systems. Also, GECNN does not identify a gully in 10% of cases and in another 10% of cases it identifies not a gully. To solve these problems, it is necessary to additionally train the neural network on the enlarged training data set.

Artificial neural networks in diagnosis of thyroid function from in vitro laboratory tests

1993 ◽  
Vol 39 (11) ◽  
pp. 2248-2253 ◽  
Author(s):  
P K Sharpe ◽  
H E Solberg ◽  
K Rootwelt ◽  
M Yearworth
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Thyroid Function ◽  
Laboratory Tests ◽  
Extreme Values ◽  
Real Life ◽  
Training Data ◽  
Data Set ◽  
Artificial Neural

Abstract We studied the potential benefit of using artificial neural networks (ANNs) for the diagnosis of thyroid function. We examined two types of ANN architecture and assessed their robustness in the face of diagnostic noise. The thyroid function data we used had previously been studied by multivariate statistical methods and a variety of pattern-recognition techniques. The total data set comprised 392 cases that had been classified according to both thyroid function and 19 clinical categories. All cases had a complete set of results of six laboratory tests (total thyroxine, free thyroxine, triiodothyronine, triiodothyronine uptake test, thyrotropin, and thyroxine-binding globulin). This data set was divided into subsets used for training the networks and for testing their performance; the test subsets contained various proportions of cases with diagnostic noise to mimic real-life diagnostic situations. The networks studied were a multilayer perceptron trained by back-propagation, and a learning vector quantization network. The training data subsets were selected according to two strategies: either training data based on cases with extreme values for the laboratory tests with randomly selected nonextreme cases added, or training cases from very pure functional groups. Both network architectures were efficient irrespective of the type of training data. The correct allocation of cases in test data subsets was 96.4-99.7% when extreme values were used for training and 92.7-98.8% when only pure cases were used.

scholarly journals Fish Detection Using Deep Learning

2020 ◽  
Vol 2020 ◽  
pp. 1-13 ◽  
Author(s):  
Suxia Cui ◽  
Yu Zhou ◽  
Yonghui Wang ◽  
Lujun Zhai
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Data Augmentation ◽  
Autonomous Underwater Vehicle ◽  
Training Data ◽  
Training Process ◽  
Data Set ◽  
Speed Up ◽  
Fish Detection ◽  
Process Speed

Recently, human being’s curiosity has been expanded from the land to the sky and the sea. Besides sending people to explore the ocean and outer space, robots are designed for some tasks dangerous for living creatures. Take the ocean exploration for an example. There are many projects or competitions on the design of Autonomous Underwater Vehicle (AUV) which attracted many interests. Authors of this article have learned the necessity of platform upgrade from a previous AUV design project, and would like to share the experience of one task extension in the area of fish detection. Because most of the embedded systems have been improved by fast growing computing and sensing technologies, which makes them possible to incorporate more and more complicated algorithms. In an AUV, after acquiring surrounding information from sensors, how to perceive and analyse corresponding information for better judgement is one of the challenges. The processing procedure can mimic human being’s learning routines. An advanced system with more computing power can facilitate deep learning feature, which exploit many neural network algorithms to simulate human brains. In this paper, a convolutional neural network (CNN) based fish detection method was proposed. The training data set was collected from the Gulf of Mexico by a digital camera. To fit into this unique need, three optimization approaches were applied to the CNN: data augmentation, network simplification, and training process speed up. Data augmentation transformation provided more learning samples; the network was simplified to accommodate the artificial neural network; the training process speed up is introduced to make the training process more time efficient. Experimental results showed that the proposed model is promising, and has the potential to be extended to other underwear objects.

Sign in / Sign up

Export Citation Format

Share Document