Information-Geometric Measures as Robust Estimators of Connection Strengths and External Inputs

2009 ◽  
Vol 21 (8) ◽  
pp. 2309-2335 ◽  
Author(s):  
Masami Tatsuno ◽  
Jean-Marc Fellous ◽  
Shun-ichi Amari
Keyword(s):  
Neural Network ◽  
Linear Model ◽  
Information Geometry ◽  
Second Order ◽  
System Level ◽  
Biophysical Model ◽  
Order Information ◽  
Close Link ◽  
The Neural Network ◽  
Log Linear

Information geometry has been suggested to provide a powerful tool for analyzing multineuronal spike trains. Among several advantages of this approach, a significant property is the close link between information-geometric measures and neural network architectures. Previous modeling studies established that the first- and second-order information-geometric measures corresponded to the number of external inputs and the connection strengths of the network, respectively. This relationship was, however, limited to a symmetrically connected network, and the number of neurons used in the parameter estimation of the log-linear model needed to be known. Recently, simulation studies of biophysical model neurons have suggested that information geometry can estimate the relative change of connection strengths and external inputs even with asymmetric connections. Inspired by these studies, we analytically investigated the link between the information-geometric measures and the neural network structure with asymmetrically connected networks of N neurons. We focused on the information-geometric measures of orders one and two, which can be derived from the two-neuron log-linear model, because unlike higher-order measures, they can be easily estimated experimentally. Considering the equilibrium state of a network of binary model neurons that obey stochastic dynamics, we analytically showed that the corrected first- and second-order information-geometric measures provided robust and consistent approximation of the external inputs and connection strengths, respectively. These results suggest that information-geometric measures provide useful insights into the neural network architecture and that they will contribute to the study of system-level neuroscience.

ARTIFICIAL NEURAL NETWORK BASED CLASSIFICATION OF MAMMOGRAPHIC MICROCALCIFICATIONS USING IMAGE STRUCTURE FEATURES

1993 ◽  
Vol 07 (06) ◽  
pp. 1377-1401 ◽  
Author(s):  
YATEEN CHITRE ◽  
ATAM P. DHAWAN ◽  
MYRON MOSKOWITZ
Keyword(s):  
Neural Network ◽  
Correlation Study ◽  
Second Order ◽  
Breast Cancers ◽  
Grey Level ◽  
K Nearest Neighbor ◽  
Image Structure ◽  
The Neural Network ◽  
Grey Level Histogram

Mammography associated with clinical breast examination and breast self-examination is the only effective and viable method for mass breast screening. Most of the minimal breast cancers are detected by the presence of microcalcifications. It is however difficult to distinguish between benign and malignant microcalcifications associated with breast cancer. Most of the techniques used in the computerized analysis of mammographic microcalcifications segment the digitized grey-level image into regions representing microcalcifications. Since mammographic images usually suffer from poorly defined microcalcification features, the extraction of microcalcification features based on segmentation process is not reliable and accurate. We present a second-order grey-level histogram based feature extraction approach which does not require the segmentation of microcalcifications into binary regions to extract features to be used in classification. The image structure features, computed from the second-order grey-level histogram statistics, are used for classification of microcalcifications. Several image structure features were computed for 100 cases of “difficult to diagnose” microcalcification cases with known biopsy results. These features were analyzed in a correlation study which provided a set of five best image structure features. A feedforward backpropagation neural network was used to classify mammographic microcalcifications using the image structure features. Four networks were trained for different combinations of training and test cases, and number of nodes in hidden layers. False Positive (FP) and True Positive (TP) rates for microcalcification classification were computed to compare the performance of the trained networks. The results of the neural network based classification were compared with those obtained using multivariate Baye’s classifiers, and the k-nearest neighbor classifier. The neural network yielded good results for classification of “difficult-to-diagnose” micro-calcifications into benign and malignant categories using the selected image structure features.

scholarly journals Neural Network for Solving SOCQP and SOCCVI Based on Two Discrete-Type Classes of SOC Complementarity Functions

2019 ◽  
Vol 2019 ◽  
pp. 1-18 ◽  
Author(s):  
Juhe Sun ◽  
Xiao-Ren Wu ◽  
B. Saheya ◽  
Jein-Shan Chen ◽  
Chun-Hsu Ko
Keyword(s):  
Neural Network ◽  
Second Order ◽  
Merit Functions ◽  
Second Order Cone ◽  
The Neural Network ◽  
Complementarity Functions ◽  
Type Classes ◽  
Quadratic Programming Problems ◽  
Numerical Performance ◽  
Discrete Type

This paper focuses on solving the quadratic programming problems with second-order cone constraints (SOCQP) and the second-order cone constrained variational inequality (SOCCVI) by using the neural network. More specifically, a neural network model based on two discrete-type families of SOC complementarity functions associated with second-order cone is proposed to deal with the Karush-Kuhn-Tucker (KKT) conditions of SOCQP and SOCCVI. The two discrete-type SOC complementarity functions are newly explored. The neural network uses the two discrete-type families of SOC complementarity functions to achieve two unconstrained minimizations which are the merit functions of the Karuch-Kuhn-Tucker equations for SOCQP and SOCCVI. We show that the merit functions for SOCQP and SOCCVI are Lyapunov functions and this neural network is asymptotically stable. The main contribution of this paper lies on its simulation part because we observe a different numerical performance from the existing one. In other words, for our two target problems, more effective SOC complementarity functions, which work well along with the proposed neural network, are discovered.

Investigation of Possible Neural Architectures Underlying Information-Geometric Measures

2004 ◽  
Vol 16 (4) ◽  
pp. 737-765 ◽  
Author(s):  
Masami Tatsuno ◽  
Masato Okada
Keyword(s):  
Parameter Estimation ◽  
Linear Model ◽  
Information Geometry ◽  
External Input ◽  
Geometric Parameters ◽  
Neural Structure ◽  
Asymmetric Networks ◽  
Neural Interactions ◽  
Ill Posed ◽  
Log Linear

A novel analytical method based on information geometry was recently proposed, and this method may provide useful insights into the statistical interactions within neural groups. The link between information-geometric measures and the structure of neural interactions has not yet been elucidated, however, because of the ill-posed nature of the problem. Here, possible neural architectures underlying information-geometric measures are investigated using an isolated pair and an isolated triplet of model neurons. By assuming the existence of equilibrium states, we derive analytically the relationship between the information-geometric parameters and these simple neural architectures. For symmetric networks, the first- and second-order information-geometric parameters represent, respectively, the external input and the underlying connections between the neurons provided that the number of neurons used in the parameter estimation in the log-linear model and the number of neurons in the network are the same. For asymmetric networks, however, these parameters are dependent on both the intrinsic connections and the external inputs to each neuron. In addition, we derive the relation between the information-geometric parameter corresponding to the two-neuron interaction and a conventional cross-correlation measure. We also show that the information-geometric parameters vary depending on the number of neurons assumed for parameter estimation in the log-linear model. This finding suggests a need to examine the information-geometric method carefully. A possible criterion for choosing an appropriate orthogonal coordinate is also discussed. This article points out the importance of a model-based approach and sheds light on the possible neural structure underlying the application of information geometry to neural network analysis.

Efficient neural network learning using second order information with fuzzy control

2002 ◽  
Vol 43 (1-4) ◽  
pp. 197-217 ◽  
Author(s):  
Peitsang Wu ◽  
Shu-Cherng Fang ◽  
Henry L.W. Nuttle
Keyword(s):  
Neural Network ◽  
Fuzzy Control ◽  
Second Order ◽  
Order Information ◽  
Neural Network Learning ◽  
Network Learning

Study on the Relation between Waxberry Color and its Nutrition Composition Based on BP Neural Network

2011 ◽  
Vol 304 ◽  
pp. 268-273
Author(s):  
Hong Xia Zhao ◽  
Zhi Xia Liu ◽  
Zhi Yang Luo ◽  
Guan Yun Xiao
Keyword(s):  
Neural Network ◽  
Transfer Function ◽  
Linear Model ◽  
Bp Neural Network ◽  
Bp Network ◽  
The Neural Network ◽  
Farm Produce ◽  
Training Error ◽  
Hidden Layer ◽  
Function Transfer

The color of farm produce is a very important index of quality, its nutrition is correlative with itself color. At present, most of the analyses for pigment and nutrient composition still depend on chemical method; therefore the relation is studied between waxberry color and its nutrition composition based on BP neural network. The conversion relation is expressed by three-layer BP network, which hidden layer has 11 node numbers and its transfer function adopts tansig function; transfer function of output layer selects purelin function. The neural network and linear model of nutrition composition is compared respectively. The MSE value of linear model is 0.300892, and that training error of neural network is 0.0219585. From this result，we can find that the conversion relation between waxberry color and its nutrition composition is a complex non-linear relation, so neural network is adopted to complete this conversion.

scholarly journals Hybrid order characteristics in car-following behavior

2020 ◽  
Vol 20 (1) ◽  
pp. 158
Author(s):  
Chunling Tu ◽  
Shengzhi Du
Keyword(s):  
Neural Network ◽  
Interesting Property ◽  
Second Order ◽  
Traffic Data ◽  
Car Following ◽  
The Neural Network ◽  
Multiple Vehicles ◽  
The Subject ◽  
Theoretical Simulations ◽  
Real Traffic

<span>This paper addresses the discovery of an interesting property in car-following processes, which was not reported in the existing literatures. A hybrid order behavior is supported by both experimental data and theoretical simulations. To demonstrate this behavior, the first order and the second order car-following behaviors are defined. Then, by comparing the first and the second order car-following behaviors in the existing analystic models and the real traffic context, this paper finds that a significant amount of the second order car-following processes in real traffic context do not match the existing models and structural mismatches are observed. The popularity and significance of such cases suggest the existence of unmodelled dynamics in the existing methods, that is, the car following behavior should be determined by more factors than the immediate proceeding vehicle. Therefore, the existing car-following models must be improved to accommodate these factors. This forms one of the main values of this paper. This paper then defines the hybrid order car-following behavior and prompts to associate this behavior with the concerned unmodelled dynamics (mismatches between the actual traffic data and the simulation from models). The neural network is employed to model such dynamics. The idea of the proposed hybrid order behavior matches the fact that the car-following behavior is determined by multiple vehicles driving in front of the subject car instead of only the immediate proceeding one. This is valuable because it provides guidance on the improvement of existing car-following models. The neural network model validates that the consideration of multiple vehicles improves the accuracy of car-following modelling.</span>

Sign in / Sign up

Export Citation Format

Share Document