scholarly journals A Hierarchical Neural-Network-Based Document Representation Approach for Text Classification

2018 ◽  
Vol 2018 ◽  
pp. 1-10 ◽  
Author(s):  
Jianming Zheng ◽  
Yupu Guo ◽  
Chong Feng ◽  
Honghui Chen
Keyword(s):  
Neural Network ◽  
Text Classification ◽  
Network Models ◽  
Document Classification ◽  
Neural Representation ◽  
Hierarchical Architecture ◽  
Document Representation ◽  
Neural Network Models ◽  
Hierarchical Neural Network ◽  
Public Datasets

Document representation is widely used in practical application, for example, sentiment classification, text retrieval, and text classification. Previous work is mainly based on the statistics and the neural networks, which suffer from data sparsity and model interpretability, respectively. In this paper, we propose a general framework for document representation with a hierarchical architecture. In particular, we incorporate the hierarchical architecture into three traditional neural-network models for document representation, resulting in three hierarchical neural representation models for document classification, that is, TextHFT, TextHRNN, and TextHCNN. Our comprehensive experimental results on two public datasets, that is, Yelp 2016 and Amazon Reviews (Electronics), show that our proposals with hierarchical architecture outperform the corresponding neural-network models for document classification, resulting in a significant improvement ranging from 4.65% to 35.08% in terms of accuracy with a comparable (or substantially less) expense of time consumption. In addition, we find that the long documents benefit more from the hierarchical architecture than the short ones as the improvement in terms of accuracy on long documents is greater than that on short documents.

scholarly journals Feedback-dependent generalization

2013 ◽  
Vol 109 (1) ◽  
pp. 202-215 ◽  
Author(s):  
Jordan A. Taylor ◽  
Laura L. Hieber ◽  
Richard B. Ivry
Keyword(s):  
Neural Network ◽  
Learning Rule ◽  
Network Models ◽  
Visuomotor Adaptation ◽  
Neural Representation ◽  
Neural Network Models ◽  
Feedback Processing ◽  
Feedback Information ◽  
Specific Effects ◽  
Sensorimotor Map

Generalization provides a window into the representational changes that occur during motor learning. Neural network models have been integral in revealing how the neural representation constrains the extent of generalization. Specifically, two key features are thought to define the pattern of generalization. First, generalization is constrained by the properties of the underlying neural units; with directionally tuned units, the extent of generalization is limited by the width of the tuning functions. Second, error signals are used to update a sensorimotor map to align the desired and actual output, with a gradient-descent learning rule ensuring that the error produces changes in those units responsible for the error. In prior studies, task-specific effects in generalization have been attributed to differences in neural tuning functions. Here we ask whether differences in generalization functions may arise from task-specific error signals. We systematically varied visual error information in a visuomotor adaptation task and found that this manipulation led to qualitative differences in generalization. A neural network model suggests that these differences are the result of error feedback processing operating on a homogeneous and invariant set of tuning functions. Consistent with novel predictions derived from the model, increasing the number of training directions led to specific distortions of the generalization function. Taken together, the behavioral and modeling results offer a parsimonious account of generalization that is based on the utilization of feedback information to update a sensorimotor map with stable tuning functions.

scholarly journals A Multibranch of Convolutional Neural Network Models for Electroencephalogram-Based Motor Imagery Classification

Biosensors ◽  
2022 ◽  
Vol 12 (1) ◽  
pp. 22
Author(s):  
Ghadir Ali Altuwaijri ◽  
Ghulam Muhammad
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Motor Imagery ◽  
Classification Accuracy ◽  
Network Models ◽  
Classification Methods ◽  
Neural Network Models ◽  
High Gamma ◽  
Public Datasets ◽  
High Level

Automatic high-level feature extraction has become a possibility with the advancement of deep learning, and it has been used to optimize efficiency. Recently, classification methods for convolutional neural network (CNN)-based electroencephalography (EEG) motor imagery have been proposed, and have achieved reasonably high classification accuracy. These approaches, however, use the CNN single convolution scale, whereas the best convolution scale varies from subject to subject. This limits the precision of classification. This paper proposes multibranch CNN models to address this issue by effectively extracting the spatial and temporal features from raw EEG data, where the branches correspond to different filter kernel sizes. The proposed method’s promising performance is demonstrated by experimental results on two public datasets, the BCI Competition IV 2a dataset and the High Gamma Dataset (HGD). The results of the technique show a 9.61% improvement in the classification accuracy of multibranch EEGNet (MBEEGNet) from the fixed one-branch EEGNet model, and 2.95% from the variable EEGNet model. In addition, the multibranch ShallowConvNet (MBShallowConvNet) improved the accuracy of a single-scale network by 6.84%. The proposed models outperformed other state-of-the-art EEG motor imagery classification methods.

scholarly journals Multicenter Validation of Convolutional Neural Networks for Automated Detection of Cardiomegaly on Chest Radiographs

2020 ◽  
Author(s):  
Diego Cardenas ◽  
José Ferreira Junior ◽  
Ramon Moreno ◽  
Marina Rebelo ◽  
José Krieger ◽  
...  
Keyword(s):  
Neural Network ◽  
External Validation ◽  
Area Under The Curve ◽  
Network Models ◽  
Automated Detection ◽  
Neural Network Models ◽  
Test Set ◽  
Chest Cavity ◽  
Public Datasets ◽  
Multicenter Validation

This work focused on validating five convolutional neural network models to detect automatically cardiomegaly, a health complication that causes heart enlargement, which may lead to cardiac arrest. To do that, we trained the models with a customized multilayer perceptron. Radiographs from two public datasets were used in experiments, one of them only for external validation. Images were pre-processed to contain just the chest cavity. The EfficientNet model yielded the highest area under the curve (AUC) of 0.91 on the test set. However, the Inception-based model obtained the best generalization performance with AUC of 0.88 on the independent multicentric dataset. Therefore, this work accurately validated radiographic models to identify patients with cardiomegaly.

scholarly journals Hierarchical Inter-Attention Network for Document Classification with Multi-Task Learning

2019 ◽  
Author(s):  
Bing Tian ◽  
Yong Zhang ◽  
Jin Wang ◽  
Chunxiao Xing
Keyword(s):  
Document Classification ◽  
Hierarchical Architecture ◽  
Shared Knowledge ◽  
Document Representation ◽  
Document Structure ◽  
Learning Framework ◽  
Task Learning ◽  
Proposed Model ◽  
Public Datasets ◽  
Specific Modeling

Document classification is an essential task in many real world applications. Existing approaches adopt both text semantics and document structure to obtain the document representation. However, these models usually require a large collection of annotated training instances, which are not always feasible, especially in low-resource settings. In this paper, we propose a multi-task learning framework to jointly train multiple related document classification tasks. We devise a hierarchical architecture to make use of the shared knowledge from all tasks to enhance the document representation of each task. We further propose an inter-attention approach to improve the task-specific modeling of documents with global information. Experimental results on 15 public datasets demonstrate the benefits of our proposed model.

scholarly journals Time-evolving Text Classification with Deep Neural Networks

2018 ◽  
Author(s):  
Yu He ◽  
Jianxin Li ◽  
Yangqiu Song ◽  
Mutian He ◽  
Hao Peng
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Text Classification ◽  
Network Models ◽  
Short Term ◽  
Neural Network Models ◽  
Evolutionary Neural Network ◽  
Learned Features ◽  
Term Fluctuation

Traditional text classification algorithms are based on the assumption that data are independent and identically distributed. However, in most non-stationary scenarios, data may change smoothly due to long-term evolution and short-term fluctuation, which raises new challenges to traditional methods. In this paper, we present the first attempt to explore evolutionary neural network models for time-evolving text classification. We first introduce a simple way to extend arbitrary neural networks to evolutionary learning by using a temporal smoothness framework, and then propose a diachronic propagation framework to incorporate the historical impact into currently learned features through diachronic connections. Experiments on real-world news data demonstrate that our approaches greatly and consistently outperform traditional neural network models in both accuracy and stability.

scholarly journals Localist plasticity identified by mutual information

2019 ◽  
Author(s):  
Gabriele Scheler ◽  
Johann Schumann
Keyword(s):  
Neural Network ◽  
Gaussian Model ◽  
Network Models ◽  
Human Memory ◽  
Neural Representation ◽  
Neural Network Models ◽  
Pattern Recall ◽  
Intrinsic Plasticity ◽  
Pattern Information ◽  
Gaussian Network

AbstractThe issue of memory is difficult for standard neural network models. Ubiquitous synaptic plasticity introduces the problem of interference, which limits pattern recall and introduces conflation errors. We present a lognormal recurrent neural network, load patterns into it (MNIST), and test the resulting neural representation for information content by an output classifier. We identify neurons, which ‘compress’ the pattern information into their own adjacency network, and by stimulating these achieve recall. Learning is limited to intrinsic plasticity and output synapses of these pattern neurons (localist plasticity), which prevents interference.Our first experiments show that this form of storage and recall is possible, with the caveat of a ‘lossy’ recall similar to human memory. Comparing our results with a standard Gaussian network model, we notice that this effect breaks down for the Gaussian model.

scholarly journals Multi-scale hierarchical neural network models that bridge from single neurons in the primate primary visual cortex to object recognition behavior

2021 ◽  
Author(s):  
Tiago Marques ◽  
Martin Schrimpf ◽  
James J. DiCarlo
Keyword(s):  
Neural Network ◽  
Object Recognition ◽  
Single Neuron ◽  
Network Models ◽  
Neural Population ◽  
Single Neurons ◽  
Neural Network Models ◽  
V1 Neurons ◽  
Multi Stage ◽  
Hierarchical Neural Network

AbstractObject recognition relies on inferior temporal (IT) cortical neural population representations that are themselves computed by a hierarchical network of feedforward and recurrently connected neural population called the ventral visual stream (areas V1, V2, V4 and IT). While recent work has created some reasonably accurate image-computable hierarchical neural network models of those neural stages, those models do not yet bridge between the properties of individual neurons and the overall emergent behavior of the ventral stream. For example, current leading ventral stream models do not allow us to ask questions such as: How does the surround suppression behavior of individual V1 neurons ultimately relate to IT neural representation and to behavior?; or How would deactivation of a particular sub-population of V1 neurons specifically alter object recognition behavior? One reason we cannot yet do this is that individual V1 artificial neurons in multi-stage models have not been shown to be functionally similar with individual biological V1 neurons. Here, we took an important first step towards this direction by building and evaluating hundreds of hierarchical neural network models in how well their artificial single neurons approximate macaque primary visual cortical (V1) neurons. We found that single neurons in some models are surprisingly similar to their biological counterparts and that the distributions of single neuron properties, such as those related to orientation and spatial frequency tuning, approximately match those in macaque V1. Crucially, we also observed that hierarchical models with V1-layers that better match macaque V1 at the single neuron level are also more aligned with human object recognition behavior. These results provide the first multi-stage, multi-scale models that allow our field to ask precisely how the specific properties of individual V1 neurons relate to recognition behavior. Finally, we here show that an optimized classical neuroscientific model of V1 is still more functionally similar to primate V1 than all of the tested multi-stage models, suggesting that further model improvements are possible, and that those improvements would likely have tangible payoffs in terms of behavioral prediction accuracy and behavioral robustness.HighlightsSingle neurons in some image-computable hierarchical neural network models are functionally similar to single neurons in macaque primate visual cortex (V1)Some hierarchical neural networks models have V1 layers that better match the biological distributions of macaque V1 single neuron response propertiesMulti-stage hierarchical neural network models with V1 stages that better match macaque V1 are also more aligned with human object recognition behavior at their output stage

Sign in / Sign up

Export Citation Format

Share Document