scholarly journals Random Compressed Coding with Neurons

2022 ◽  
Author(s):  
Simone Blanco Malerba ◽  
Mirko Pieropan ◽  
Yoram Burak ◽  
Rava Azeredo da Silveira
Keyword(s):  
Neural Network ◽  
Fine Tuning ◽  
Neural Population ◽  
Dimensional Representation ◽  
Tuning Curves ◽  
Efficient Coding ◽  
Monotonic Functions ◽  
The Neural Network ◽  
Classical Models ◽  
Low Dimensional

Classical models of efficient coding in neurons assume simple mean responses--'tuning curves'--such as bell shaped or monotonic functions of a stimulus feature. Real neurons, however, can be more complex: grid cells, for example, exhibit periodic responses which impart the neural population code with high accuracy. But do highly accurate codes require fine tuning of the response properties? We address this question with the use of a benchmark model: a neural network with random synaptic weights which result in output cells with irregular tuning curves. Irregularity enhances the local resolution of the code but gives rise to catastrophic, global errors. For optimal smoothness of the tuning curves, when local and global errors balance out, the neural network compresses information from a high-dimensional representation to a low-dimensional one, and the resulting distributed code achieves exponential accuracy. An analysis of recordings from monkey motor cortex points to such 'compressed efficient coding'. Efficient codes do not require a finely tuned design--they emerge robustly from irregularity or randomness.

scholarly journals Implementing a Bayes Filter in a Neural Circuit: The Case of Unknown Stimulus Dynamics

2017 ◽  
Vol 29 (9) ◽  
pp. 2450-2490 ◽  
Author(s):  
Sacha Sokoloski
Keyword(s):  
Neural Network ◽  
Internal Model ◽  
Nonlinear Filtering ◽  
Neural Circuit ◽  
Stochastic Gradient Descent ◽  
Neural Population ◽  
Bayes Rule ◽  
Tuning Curves ◽  
Population Responses ◽  
The Neural Network

In order to interact intelligently with objects in the world, animals must first transform neural population responses into estimates of the dynamic, unknown stimuli that caused them. The Bayesian solution to this problem is known as a Bayes filter, which applies Bayes' rule to combine population responses with the predictions of an internal model. The internal model of the Bayes filter is based on the true stimulus dynamics, and in this note, we present a method for training a theoretical neural circuit to approximately implement a Bayes filter when the stimulus dynamics are unknown. To do this we use the inferential properties of linear probabilistic population codes to compute Bayes' rule and train a neural network to compute approximate predictions by the method of maximum likelihood. In particular, we perform stochastic gradient descent on the negative log-likelihood of the neural network parameters with a novel approximation of the gradient. We demonstrate our methods on a finite-state, a linear, and a nonlinear filtering problem and show how the hidden layer of the neural network develops tuning curves consistent with findings in experimental neuroscience.

scholarly journals Isobaric vapour-liquid equilibrium calculations of binary systems using neural network

2004 ◽  
Vol 69 (8-9) ◽  
pp. 669-674 ◽  
Author(s):  
Mehmet Bilgin
Keyword(s):  
Neural Network ◽  
Binary Systems ◽  
Back Propagation ◽  
Back Propagation Neural Network ◽  
Liquid Equilibrium ◽  
Chemical Structures ◽  
Feed Forward Back Propagation ◽  
The Neural Network ◽  
Classical Models ◽  
Vle Data

A model on a feed forward back propagation neural network was employed to calculate the isobaric vapour?liquid equilibrium (VLE) data at 40, 66.67, and 101.32 ??0.02 kPa for the methylcyclohexane ? toluene and isopropanol ? methyl isobutyl ketone binary systems, which are composed of different chemical structures (cyclic, aromatic, alcohol and ketone) and do not show azeotrope behaviour. Half of the experimental VLE data only were assigned into the designed framework as training patterns in order to estimate the VLE data over the whole composition range at the mentioned pressures. The results were compared with the data calculated by the two classical models used in this field, the UNIFAC and Margules models. In all cases the deviations the experimental activity coefficients and those calculated by the neural network model (NNET) were lower than those obtained using the Margules and UNIFAC models.

scholarly journals Bias in Deep Neural Networks in Land Use Characterization for International Development

2021 ◽  
Vol 13 (15) ◽  
pp. 2908
Author(s):  
Do-Hyung Kim ◽  
Guzmán López ◽  
Diego Kiedanski ◽  
Iyke Maduako ◽  
Braulio Ríos ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Land Use ◽  
International Development ◽  
Deep Neural Networks ◽  
Satellite Image ◽  
Fine Tuning ◽  
Training Dataset ◽  
Model Accuracy ◽  
The Neural Network

Understanding the biases in Deep Neural Networks (DNN) based algorithms is gaining paramount importance due to its increased applications on many real-world problems. A known problem of DNN penalizing the underrepresented population could undermine the efficacy of development projects dependent on data produced using DNN-based models. In spite of this, the problems of biases in DNN for Land Use and Land Cover Classification (LULCC) have not been a subject of many studies. In this study, we explore ways to quantify biases in DNN for land use with an example of identifying school buildings in Colombia from satellite imagery. We implement a DNN-based model by fine-tuning an existing, pre-trained model for school building identification. The model achieved overall 84% accuracy. Then, we used socioeconomic covariates to analyze possible biases in the learned representation. The retrained deep neural network was used to extract visual features (embeddings) from satellite image tiles. The embeddings were clustered into four subtypes of schools, and the accuracy of the neural network model was assessed for each cluster. The distributions of various socioeconomic covariates by clusters were analyzed to identify the links between the model accuracy and the aforementioned covariates. Our results indicate that the model accuracy is lowest (57%) where the characteristics of the landscape are predominantly related to poverty and remoteness, which confirms our original assumption on the heterogeneous performances of Artificial Intelligence (AI) algorithms and their biases. Based on our findings, we identify possible sources of bias and present suggestions on how to prepare a balanced training dataset that would result in less biased AI algorithms. The framework used in our study to better understand biases in DNN models would be useful when Machine Learning (ML) techniques are adopted in lieu of ground-based data collection for international development programs. Because such programs aim to solve issues of social inequality, MLs are only applicable when they are transparent and accountable.

scholarly journals Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network

2017 ◽  
Vol 110 ◽  
pp. 387-405 ◽  
Author(s):  
Eric Laloy ◽  
Romain Hérault ◽  
John Lee ◽  
Diederik Jacques ◽  
Niklas Linde
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Dimensional Representation ◽  
Low Dimensional

scholarly journals Visualizing Clusters in Artificial Neural Networks Using Morse Theory

2013 ◽  
Vol 2013 ◽  
pp. 1-8 ◽  
Author(s):  
Paul T. Pearson
Keyword(s):  
Neural Network ◽  
Iterative Process ◽  
Principal Component ◽  
Topological Data Analysis ◽  
High Dimensional ◽  
Clustering Problem ◽  
The Neural Network ◽  
Low Dimensional ◽  
High Dimensional Clustering

This paper develops a process whereby a high-dimensional clustering problem is solved using a neural network and a low-dimensional cluster diagram of the results is produced using the Mapper method from topological data analysis. The low-dimensional cluster diagram makes the neural network's solution to the high-dimensional clustering problem easy to visualize, interpret, and understand. As a case study, a clustering problem from a diabetes study is solved using a neural network. The clusters in this neural network are visualized using the Mapper method during several stages of the iterative process used to construct the neural network. The neural network and Mapper clustering diagram results for the diabetes study are validated by comparison to principal component analysis.

scholarly journals A novel method for peanut variety identification and classification by Improved VGG16

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Haoyan Yang ◽  
Jiangong Ni ◽  
Jiyue Gao ◽  
Zhongzhi Han ◽  
Tao Luan
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Fine Tuning ◽  
Variety Identification ◽  
Learning Technology ◽  
Data Set ◽  
Average Accuracy ◽  
Classical Models ◽  
Crop Variety ◽  
Fully Connected

AbstractCrop variety identification is an essential link in seed detection, phenotype collection and scientific breeding. This paper takes peanut as an example to explore a new method for crop variety identification. Peanut is a crucial oil crop and cash crop. The yield and quality of different peanut varieties are different, so it is necessary to identify and classify different peanut varieties. The traditional image processing method of peanut variety identification needs to extract many features, which has defects such as intense subjectivity and insufficient generalization ability. Based on the deep learning technology, this paper improved the deep convolutional neural network VGG16 and applied the improved VGG16 to the identification and classification task of 12 varieties of peanuts. Firstly, the peanut pod images of 12 varieties obtained by the scanner were preprocessed with gray-scale, binarization, and ROI extraction to form a peanut pod data set with a total of 3365 images of 12 varieties. A series of improvements have been made to VGG16. Remove the F6 and F7 fully connected layers of VGG16. Add Conv6 and Global Average Pooling Layer. The three convolutional layers of conv5 have changed into Depth Concatenation and add the Batch Normalization(BN) layers to the model. Besides, fine-tuning is carried out based on the improved VGG16. We adjusted the location of the BN layers. Adjust the number of filters for Conv6. Finally, the improved VGG16 model's training test results were compared with the other classic models, AlexNet, VGG16, GoogLeNet, ResNet18, ResNet50, SqueezeNet, DenseNet201 and MobileNetv2 verify its superiority. The average accuracy of the improved VGG16 model on the peanut pods test set was 96.7%, which was 8.9% higher than that of VGG16, and 1.6–12.3% higher than that of other classical models. Besides, supplementary experiments were carried out to prove the robustness and generality of the improved VGG16. The improved VGG16 was applied to the identification and classification of seven corn grain varieties with the same method and an average accuracy of 90.1% was achieved. The experimental results show that the improved VGG16 proposed in this paper can identify and classify peanut pods of different varieties, proving the feasibility of a convolutional neural network in variety identification and classification. The model proposed in this experiment has a positive significance for exploring other Crop variety identification and classification.

scholarly journals Design of a Food Recommendation System using ADNet algorithm on a Hybrid Data Mining Process

2022 ◽  
Vol 3 (4) ◽  
pp. 272-282
Author(s):  
Haoxiang Wang
Keyword(s):  
Neural Network ◽  
Data Mining ◽  
Architectural Design ◽  
Recommendation System ◽  
Numerical Data ◽  
Fine Tuning ◽  
Classification Rate ◽  
The Neural Network ◽  
Hybrid Data ◽  
Neural Network Algorithm

Hybrid data mining processes are employed in recent days on several applications to achieve a better prediction and classification rate along with customer satisfaction. Hybrid data mining processes are the combination of different form of data considered for a neural network decision. In some cases, the different form of data represents image along with numerical data. In the proposed work, a food recommendation system is developed with respect to the flavour taste of the customer and considering the review comments of previous customers. The suggestions given by the users are taken into account as a feedback layer in the neural network for fine tuning the accuracy of the prediction process. The architectural design of the proposed model is employed with an ADNet (Adaptively Dense Convolutional Neural Network) algorithm to enable the usage of low range features in an efficient way. To verify the performance of the developed model, a pizza flavour recommender dataset is employed in the work for analysis. The experimental work analysis indicates that the ADNet algorithm works in a better way on a hybrid data analysis than the traditional DenseNet and ResNet algorithms.

Study on PCA-Based BP Neural Network Algorithm

2014 ◽  
Vol 530-531 ◽  
pp. 517-521
Author(s):  
Jian Qing Hong ◽  
De'an Zhao ◽  
Wei Kuan Jia
Keyword(s):  
Neural Network ◽  
Bp Neural Network ◽  
Computing Time ◽  
Principal Component ◽  
Bp Algorithm ◽  
Complex Data ◽  
Network Algorithm ◽  
The Neural Network ◽  
Neural Network Algorithm ◽  
Low Dimensional

Using the neural network to deal with complex data, because the pending sample with many variables, aiming at this nature of the pending sample and the structure properties of the BP neural network, in this paper, we propose the new BP neural network algorithm base on principal component analysis (PCA-BP algorithm). The new algorithm through PCA dimension reduction for complex data, got the low-dimensional data as the BP neural networks input, it will be beneficial to design the hidden layer of neural network, save a lot of storage space and computing time, and conductive to the convergence of the neural network. In order to verify the validity of the new algorithm, compared with the traditional BP algorithm, through the case analysis, the result show that the new algorithm improve the efficiency and recognition precise, worthy of further promotion.

Understanding Image Classification Using TensorFlow Deep Learning - Convolution Neural Network

2019 ◽  
Vol 3 (2) ◽  
pp. 19-37
Author(s):  
Vinit Kumar Gunjan ◽  
Rashmi Pathak ◽  
Omveer Singh
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolution Neural Network ◽  
Fine Tuning ◽  
Learning Technology ◽  
The Neural Network ◽  
Learning Technique ◽  
Classification Of Images ◽  
Initial Classification

This article describes how to establish the neural network technique for various image groupings in a convolution neural network (CNN) training. In addition, it also suggests initial classification results using CNN learning characteristics and classification of images from different categories. To determine the correct architecture, we explore a transfer learning technique, called Fine-Tuning of Deep Learning Technology, a dataset used to provide solutions for individually classified image-classes.

Sign in / Sign up

Export Citation Format

Share Document