scholarly journals A Normative Bayesian Model of Classification for Agents with Bounded Memory

2019 ◽  
Author(s):  
Heeseung Lee ◽  
Hyang-Jung Lee ◽  
Kyoung Whan Choe ◽  
Sang-Hun Lee

AbstractClassification, one of the key ingredients for human cognition, entails establishing a criterion that splits a given feature space into mutually exclusive subspaces. In classification tasks performed in daily life, however, a criterion is often not provided explicitly but instead needs to be guessed from past samples of a feature space. For example, we judge today’s temperature to be “cold” or “warm” by implicitly comparing it against a “typical” seasonal temperature. In such situations, establishing an optimal criterion is challenging for cognitive agents with bounded memory because it requires retrieving an entire set of past episodes with precision. As a computational account for how humans carry out this challenging operation, we developed a normative Bayesian model of classification (NBMC), in which Bayesian agents, whose working-memory precision decays as episodes elapse, continuously update their criterion as they perform a binary perceptual classification task on sequentially presented stimuli. We drew a set of specific implications regarding key properties of classification from the NBMC, and demonstrated the correspondence between the NBMC and human observers in classification behavior for each of those implications. Furthermore, in the functional magnetic resonance imaging responses acquired concurrently with behavioral data, we identified an ensemble of brain activities that coherently represent the latent variables, including the inferred criterion, of the NBMC. Given these results, we believe that the NBMC is worth being considered as a useful computational model that guides behavioral and neural studies on perceptual classification, especially for agents with bounded memory representation of past sensory events.Significant StatementAlthough classification—assigning events into mutually exclusive classes—requires a criterion, people often have to perform various classification tasks without explicit criteria. In such situations, forming a criterion based on past experience is quite challenging because people’s memory of past events deteriorates quickly over time. Here, we provided a computational model for how a memory-bounded yet normative agent infers the criterion from past episodes to maximally perform a binary perceptual classification task. This model successfully captured several key properties of human classification behavior, and the neural signals representing its latent variables were identified in the classifying human brains. By offering a rational account for memory-bonded agents’ classification, our model can guide future behavioral and neural studies on perceptual classification.

Author(s):  
Cunxiao Du ◽  
Zhaozheng Chen ◽  
Fuli Feng ◽  
Lei Zhu ◽  
Tian Gan ◽  
...  

Text classification is one of the fundamental tasks in natural language processing. Recently, deep neural networks have achieved promising performance in the text classification task compared to shallow models. Despite of the significance of deep models, they ignore the fine-grained (matching signals between words and classes) classification clues since their classifications mainly rely on the text-level representations. To address this problem, we introduce the interaction mechanism to incorporate word-level matching signals into the text classification task. In particular, we design a novel framework, EXplicit interAction Model (dubbed as EXAM), equipped with the interaction mechanism. We justified the proposed approach on several benchmark datasets including both multilabel and multi-class text classification tasks. Extensive experimental results demonstrate the superiority of the proposed method. As a byproduct, we have released the codes and parameter settings to facilitate other researches.


Author(s):  
Le Hui ◽  
Xiang Li ◽  
Chen Gong ◽  
Meng Fang ◽  
Joey Tianyi Zhou ◽  
...  

Convolutional Neural Networks (CNNs) have shown great power in various classification tasks and have achieved remarkable results in practical applications. However, the distinct learning difficulties in discriminating different pairs of classes are largely ignored by the existing networks. For instance, in CIFAR-10 dataset, distinguishing cats from dogs is usually harder than distinguishing horses from ships. By carefully studying the behavior of CNN models in the training process, we observe that the confusion level of two classes is strongly correlated with their angular separability in the feature space. That is, the larger the inter-class angle is, the lower the confusion will be. Based on this observation, we propose a novel loss function dubbed “Inter-Class Angular Loss” (ICAL), which explicitly models the class correlation and can be directly applied to many existing deep networks. By minimizing the proposed ICAL, the networks can effectively discriminate the examples in similar classes by enlarging the angle between their corresponding class vectors. Thorough experimental results on a series of vision and nonvision datasets confirm that ICAL critically improves the discriminative ability of various representative deep neural networks and generates superior performance to the original networks with conventional softmax loss.


2014 ◽  
Vol 556-562 ◽  
pp. 4901-4905
Author(s):  
Yang Yan ◽  
Wen Bo Huang ◽  
Yun Ji Wang

We use Conditional Random Fields (CRFs) to classify regions in an image. CRFs provide a discriminative framework to incorporate spatial dependencies in an image, which is more appropriate for classification tasks as opposed to a generative framework. In this paper we apply CRFs to the image multi-classification task, we focus on three aspects of the classification task: feature extraction, the Original feature clustering based on K-means, and feature vector modeling base on CRF to obtain multiclass classification. We present classification results on sample images from Cambridge (MSRC) database, and the experimental results show that the method we present can classify the images accurately.


2005 ◽  
Vol 17 (2) ◽  
pp. 397-423 ◽  
Author(s):  
Yan Karklin ◽  
Michael S. Lewicki

Capturing statistical regularities in complex, high-dimensional data is an important problem in machine learning and signal processing. Models such as principal component analysis (PCA) and independent component analysis (ICA) make few assumptions about the structure in the data and have good scaling properties, but they are limited to representing linear statistical regularities and assume that the distribution of the data is stationary. For many natural, complex signals, the latent variables often exhibit residual dependencies as well as nonstationary statistics. Here we present a hierarchical Bayesian model that is able to capture higher-order nonlinear structure and represent nonstationary data distributions. The model is a generalization of ICA in which the basis function coefficients are no longer assumed to be independent; instead, the dependencies in their magnitudes are captured by a set of density components. Each density component describes a common pattern of deviation from the marginal density of the pattern ensemble; in different combinations, they can describe nonstationary distributions. Adapting the model to image or audio data yields a nonlinear, distributed code for higher-order statistical regularities that reflect more abstract, invariant properties of the signal.


Processes ◽  
2019 ◽  
Vol 7 (4) ◽  
pp. 222 ◽  
Author(s):  
Bodur ◽  
Atsa’am

This research developed and tested a filter algorithm that serves to reduce the feature space in healthcare datasets. The algorithm binarizes the dataset, and then separately evaluates the risk ratio of each predictor with the response, and outputs ratios that represent the association between a predictor and the class attribute. The value of the association translates to the importance rank of the corresponding predictor in determining the outcome. Using Random Forest and Logistic regression classification, the performance of the developed algorithm was compared against the regsubsets and varImp functions, which are unsupervised methods of variable selection. Equally, the proposed algorithm was compared with the supervised Fisher score and Pearson’s correlation feature selection methods. Different datasets were used for the experiment, and, in the majority of the cases, the predictors selected by the new algorithm outperformed those selected by the existing algorithms. The proposed filter algorithm is therefore a reliable alternative for variable ranking in data mining classification tasks with a dichotomous response.


2011 ◽  
Vol 60 (4) ◽  
pp. 485-496
Author(s):  
Jun Ouyang ◽  
David Lowther

Towards a case-based computational model for the creative design of electromagnetic devicesIn order to explore creativity in design, a computational model based on Case-Based Reasoning (CBR) (an approach to employing old experiences to solve new problems) and other soft computing techniques from machine learning, is proposed in this paper. The new model is able to address the four challenging issues: generation of a design prototype from incomplete requirements, judgment and improvement of system performance given a sparse initial case base library, extraction of critical features from a given feature space, adaptation of retrieved previous solutions to similar problems for deriving a solution to a given design task. The core principle within this model is that different knowledge from various level cases can be explicitly explored and integrated into a practical design process. In order to demonstrate the practical significance of our presented computational model, a case-based design system for EM devices, which is capable of deriving a new design prototype from a real-world device case base with high dimensionality, has been developed.


2021 ◽  
pp. 003151252110440
Author(s):  
Qiangqiang Wang ◽  
Lina Ma ◽  
Weidong Tao ◽  
Zhiwei Wang ◽  
Guichun Jin

How people encode numbers in the context of multiple overlapping encoded cues remains unclear. In this study, we explored Chinese finger numbers, which contain both a numerical magnitude cue and a left-right hand cue offered by the expressing hand, to investigate the number encoding mechanism in the context of multiple overlapping cues. Chinese finger numbers expressed by the left or right hand were randomly and centrally presented on a computer screen to participants who were asked to perform a hand classification task (Experiment 1), a magnitude classification task (Experiment 2), a parity classification task (Experiment 3) and a magnitude classification or ring classification task (Experiment 4). We discovered (a) only an association effect between the pressed key and the expressing hand in hand classification and parity classification tasks, (b) the SNARC effect only on the magnitude classification task, (c) the association effect between the pressed key and the expressing hand on the larger, Chinese finger number, magnitude classification task in Experiment 2, and (d) the SNARC effect and the association between the pressed key and the expressing hand were reversed on the ring classification task. From these results, we concluded that people can flexibly choose appropriate number encoding cues and how numbers are encoded in the context of multiple overlapping cues depending on (a) which cognition task individuals perform and (b) the character of the numbers involved.


2021 ◽  
Vol 11 (12) ◽  
pp. 5409
Author(s):  
Julián Gil-González ◽  
Andrés Valencia-Duque ◽  
Andrés Álvarez-Meza ◽  
Álvaro Orozco-Gutiérrez ◽  
Andrea García-Moreno

The increasing popularity of crowdsourcing platforms, i.e., Amazon Mechanical Turk, changes how datasets for supervised learning are built. In these cases, instead of having datasets labeled by one source (which is supposed to be an expert who provided the absolute gold standard), databases holding multiple annotators are provided. However, most state-of-the-art methods devoted to learning from multiple experts assume that the labeler’s behavior is homogeneous across the input feature space. Besides, independence constraints are imposed on annotators’ outputs. This paper presents a regularized chained deep neural network to deal with classification tasks from multiple annotators. The introduced method, termed RCDNN, jointly predicts the ground truth label and the annotators’ performance from input space samples. In turn, RCDNN codes interdependencies among the experts by analyzing the layers’ weights and includes l1, l2, and Monte-Carlo Dropout-based regularizers to deal with the over-fitting issue in deep learning models. Obtained results (using both simulated and real-world annotators) demonstrate that RCDNN can deal with multi-labelers scenarios for classification tasks, defeating state-of-the-art techniques.


2010 ◽  
Vol 7 (9) ◽  
pp. 313-313
Author(s):  
R. Rosenholtz ◽  
N. Twarog ◽  
M. Wattenberg

Sign in / Sign up

Export Citation Format

Share Document