scholarly journals A brain-inspired network architecture for cost-efficient object recognition in shallow hierarchical neural networks

2021 ◽  
Vol 134 ◽  
pp. 76-85
Author(s):  
Youngjin Park ◽  
Seungdae Baek ◽  
Se-Bum Paik
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Network Architecture ◽  
Hierarchical Neural Networks ◽  
Cost Efficient

scholarly journals Diverse Deep Neural Networks All Predict Human IT Well, After Training and Fitting

2021 ◽  
pp. 1-21
Author(s):  
Katherine R. Storrs ◽  
Tim C. Kietzmann ◽  
Alexander Walther ◽  
Johannes Mehrer ◽  
Nikolaus Kriegeskorte
Keyword(s):  
Neural Networks ◽  
Visual Cortex ◽  
Object Recognition ◽  
Network Architecture ◽  
High Performance ◽  
Deep Neural Networks ◽  
Model Fitting ◽  
Recognition Task ◽  
Dissimilarity Matrix ◽  
Weighted Combination

Abstract Deep neural networks (DNNs) trained on object recognition provide the best current models of high-level visual cortex. What remains unclear is how strongly experimental choices, such as network architecture, training, and fitting to brain data, contribute to the observed similarities. Here, we compare a diverse set of nine DNN architectures on their ability to explain the representational geometry of 62 object images in human inferior temporal (hIT) cortex, as measured with fMRI. We compare untrained networks to their task-trained counterparts and assess the effect of cross-validated fitting to hIT, by taking a weighted combination of the principal components of features within each layer and, subsequently, a weighted combination of layers. For each combination of training and fitting, we test all models for their correlation with the hIT representational dissimilarity matrix, using independent images and subjects. Trained models outperform untrained models (accounting for 57% more of the explainable variance), suggesting that structured visual features are important for explaining hIT. Model fitting further improves the alignment of DNN and hIT representations (by 124%), suggesting that the relative prevalence of different features in hIT does not readily emerge from the Imagenet object-recognition task used to train the networks. The same models can also explain the disparate representations in primary visual cortex (V1), where stronger weights are given to earlier layers. In each region, all architectures achieved equivalently high performance once trained and fitted. The models' shared properties—deep feedforward hierarchies of spatially restricted nonlinear filters—seem more important than their differences, when modeling human visual representations.

scholarly journals Sparse long-range connections in visual cortex for cost-efficient small-world networks

2020 ◽  
Author(s):  
Seungdae Baek ◽  
Youngjin Park ◽  
Se-Bum Paik
Keyword(s):  
Neural Networks ◽  
Visual Cortex ◽  
Object Recognition ◽  
Long Range ◽  
Deep Neural Networks ◽  
Small World ◽  
List Type ◽  
Cost Efficient ◽  
Local Circuits ◽  
The Brain

AbstractThe brain performs visual object recognition using much shallower hierarchical stages than artificial deep neural networks employ. However, the mechanism underlying this cost-efficient function is elusive. Here, we show that cortical long-range connectivity(LRC) may enable this parsimonious organization of circuits for balancing cost and performance. Using model network simulations based on data in tree shrews, we found that sparse LRCs, when added to local connections, organize a small-world network that dramatically enhances object recognition of shallow feedforward networks. We found that optimization of the ratio between LRCs and local connections maximizes the small-worldness and task performance of the network, by minimizing the total length of wiring needed for integration of the global information. We also found that the effect of LRCs varies by network size, which explains the existence of species-specific LRCs in mammalian visual cortex of various sizes. Our results demonstrate a biological strategy to achieve cost-efficient brain circuits.HighlightsLong-range connections (LRCs) enhance the object recognition of shallow networksSparse LRCs added to dense local connections organize a small-world type networkSmall-worldness of networks modulates the balance between performance and wiring costDistinct LRCs in various species are due to the size-dependent effect of LRCsSignificance statementThe hierarchical depth of the visual pathway in the brain is constrained by biological factors, whereas artificial deep neural networks consist of super-deep structures (i.e., as deep as computational power allows). Here, we show that long-range horizontal connections (LRCs) observed in mammalian visual cortex may enable shallow biological networks to perform cognitive tasks that require deeper artificial structures, by implementing cost-efficient organization of circuitry. Using model simulations based on anatomical data, we found that sparse LRCs, when added to dense local circuits, organize “small-world” type networks and that this dramatically enhances image classification performance by integrating both local and global components of visual stimulus. Our findings show a biological strategy of brain circuitry to balance sensory performance and wiring cost in the networks.One sentence summaryCortical long-range connections organize a small-world type network to achieve cost-efficient functional circuits under biological constraints

SCORING MODELING BASED ON NEURAL NETWORKS FOR DETERMINING A BANK BORROWER'S RATING

2020 ◽  
Vol 2020 (10) ◽  
pp. 54-62
Author(s):  
Oleksii VASYLIEV ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Statistical Data ◽  
Activation Function ◽  
Decision Making Process ◽  
Neural Network Architecture ◽  
Acceptable Accuracy ◽  
The Neural Network ◽  
Sigmoid Activation Function

The problem of applying neural networks to calculate ratings used in banking in the decision-making process on granting or not granting loans to borrowers is considered. The task is to determine the rating function of the borrower based on a set of statistical data on the effectiveness of loans provided by the bank. When constructing a regression model to calculate the rating function, it is necessary to know its general form. If so, the task is to calculate the parameters that are included in the expression for the rating function. In contrast to this approach, in the case of using neural networks, there is no need to specify the general form for the rating function. Instead, certain neural network architecture is chosen and parameters are calculated for it on the basis of statistical data. Importantly, the same neural network architecture can be used to process different sets of statistical data. The disadvantages of using neural networks include the need to calculate a large number of parameters. There is also no universal algorithm that would determine the optimal neural network architecture. As an example of the use of neural networks to determine the borrower's rating, a model system is considered, in which the borrower's rating is determined by a known non-analytical rating function. A neural network with two inner layers, which contain, respectively, three and two neurons and have a sigmoid activation function, is used for modeling. It is shown that the use of the neural network allows restoring the borrower's rating function with quite acceptable accuracy.

MeliusNet: An Improved Network Architecture for Binary Neural Networks

2021 ◽  
Author(s):  
Joseph Bethge ◽  
Christian Bartz ◽  
Haojin Yang ◽  
Ying Chen ◽  
Christoph Meinel
Keyword(s):  
Neural Networks ◽  
Network Architecture

scholarly journals SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

2021 ◽  
Vol 40 (3) ◽  
pp. 1-13
Author(s):  
Lumin Yang ◽  
Jiajie Zhuang ◽  
Hongbo Fu ◽  
Xiangzhi Wei ◽  
Kun Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Large Scale ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Structure Information ◽  
Graph Neural Networks ◽  
Node Labels ◽  
Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

scholarly journals Human and bird detection and classification based on Doppler radar spectrograms and vision images using convolutional neural networks

2021 ◽  
Vol 18 (3) ◽  
pp. 172988142110105
Author(s):  
Jnana Sai Abhishek Varma Gokaraju ◽  
Weon Keun Song ◽  
Min-Ho Ka ◽  
Somyot Kaitwanidvilai
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Convolutional Neural Networks ◽  
Doppler Radar ◽  
Autonomous Vehicle ◽  
Sampling Point ◽  
Body Segment ◽  
Kinematic Modeling ◽  
Deep Convolutional Neural Networks ◽  
Kinematic Models

The study investigated object detection and classification based on both Doppler radar spectrograms and vision images using two deep convolutional neural networks. The kinematic models for a walking human and a bird flapping its wings were incorporated into MATLAB simulations to create data sets. The dynamic simulator identified the final position of each ellipsoidal body segment taking its rotational motion into consideration in addition to its bulk motion at each sampling point to describe its specific motion naturally. The total motion induced a micro-Doppler effect and created a micro-Doppler signature that varied in response to changes in the input parameters, such as varying body segment size, velocity, and radar location. Micro-Doppler signature identification of the radar signals returned from the target objects that were animated by the simulator required kinematic modeling based on a short-time Fourier transform analysis of the signals. Both You Only Look Once V3 and Inception V3 were used for the detection and classification of the objects with different red, green, blue colors on black or white backgrounds. The results suggested that clear micro-Doppler signature image-based object recognition could be achieved in low-visibility conditions. This feasibility study demonstrated the application possibility of Doppler radar to autonomous vehicle driving as a backup sensor for cameras in darkness. In this study, the first successful attempt of animated kinematic models and their synchronized radar spectrograms to object recognition was made.

Sign in / Sign up

Export Citation Format

Share Document