A Latent Variable Augmentation Method for Image Categorization with Insufficient Training Samples

2021 ◽  
Vol 16 (1) ◽  
pp. 1-35
Author(s):  
Luyue Lin ◽  
Xin Zheng ◽  
Bo Liu ◽  
Wei Chen ◽  
Yanshan Xiao
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Large Scale ◽  
Image Data ◽  
Training Procedure ◽  
Image Categorization ◽  
Data Set ◽  
Training Samples ◽  
Accurate Performance ◽  
Limited Training Samples

Over the past few years, we have made great progress in image categorization based on convolutional neural networks (CNNs). These CNNs are always trained based on a large-scale image data set; however, people may only have limited training samples for training CNN in the real-world applications. To solve this problem, one intuition is augmenting training samples. In this article, we propose an algorithm called Lavagan ( La tent V ariables A ugmentation Method based on G enerative A dversarial N ets) to improve the performance of CNN with insufficient training samples. The proposed Lavagan method is mainly composed of two tasks. The first task is that we augment a number latent variables (LVs) from a set of adaptive and constrained LVs distributions. In the second task, we take the augmented LVs into the training procedure of the image classifier. By taking these two tasks into account, we propose a uniform objective function to incorporate the two tasks into the learning. We then put forward an alternative two-play minimization game to minimize this uniform loss function such that we can obtain the predictive classifier. Moreover, based on Hoeffding’s Inequality and Chernoff Bounding method, we analyze the feasibility and efficiency of the proposed Lavagan method, which manifests that the LV augmentation method is able to improve the performance of Lavagan with insufficient training samples. Finally, the experiment has shown that the proposed Lavagan method is able to deliver more accurate performance than the existing state-of-the-art methods.

scholarly journals Accelerating Relevance-Vector-Machine-Based Classification of Hyperspectral Image with Parallel Computing

2012 ◽  
Vol 2012 ◽  
pp. 1-13
Author(s):  
Chao Dong ◽  
Lianfang Tian
Keyword(s):  
Parallel Computing ◽  
Large Scale ◽  
Message Passing Interface ◽  
Hyperspectral Image ◽  
Relevance Vector Machine ◽  
Support Vector ◽  
Training Procedure ◽  
Data Set ◽  
Computing Technique

Benefiting from the kernel skill and the sparse property, the relevance vector machine (RVM) could acquire a sparse solution, with an equivalent generalization ability compared with the support vector machine. The sparse property requires much less time in the prediction, making RVM potential in classifying the large-scale hyperspectral image. However, RVM is not widespread influenced by its slow training procedure. To solve the problem, the classification of the hyperspectral image using RVM is accelerated by the parallel computing technique in this paper. The parallelization is revealed from the aspects of the multiclass strategy, the ensemble of multiple weak classifiers, and the matrix operations. The parallel RVMs are implemented using the C language plus the parallel functions of the linear algebra packages and the message passing interface library. The proposed methods are evaluated by the AVIRIS Indian Pines data set on the Beowulf cluster and the multicore platforms. It shows that the parallel RVMs accelerate the training procedure obviously.

scholarly journals Distilling vector space model scores for the assessment of constructed responses with bifactor Inbuilt Rubric method and latent variables

2022 ◽  
Author(s):  
José Ángel Martínez-Huertas ◽  
Ricardo Olmos ◽  
Guillermo Jorge-Botana ◽  
José A. León
Keyword(s):  
Vector Space ◽  
Undergraduate Students ◽  
Latent Variables ◽  
Measurement Errors ◽  
Latent Variable ◽  
Discriminant Validity ◽  
Semantic Analysis ◽  
Measurement Models ◽  
Accurate Performance ◽  
Constructed Responses

AbstractIn this paper, we highlight the importance of distilling the computational assessments of constructed responses to validate the indicators/proxies of constructs/trins using an empirical illustration in automated summary evaluation. We present the validation of the Inbuilt Rubric (IR) method that maps rubrics into vector spaces for concepts’ assessment. Specifically, we improved and validated its scores’ performance using latent variables, a common approach in psychometrics. We also validated a new hierarchical vector space, namely a bifactor IR. 205 Spanish undergraduate students produced 615 summaries of three different texts that were evaluated by human raters and different versions of the IR method using latent semantic analysis (LSA). The computational scores were validated using multiple linear regressions and different latent variable models like CFAs or SEMs. Convergent and discriminant validity was found for the IR scores using human rater scores as validity criteria. While this study was conducted in the Spanish language, the proposed scheme is language-independent and applicable to any language. We highlight four main conclusions: (1) Accurate performance can be observed in topic-detection tasks without hundreds/thousands of pre-scored samples required in supervised models. (2) Convergent/discriminant validity can be improved using measurement models for computational scores as they adjust for measurement errors. (3) Nouns embedded in fragments of instructional text can be an affordable alternative to use the IR method. (4) Hierarchical models, like the bifactor IR, can increase the validity of computational assessments evaluating general and specific knowledge in vector space models. R code is provided to apply the classic and bifactor IR method.

scholarly journals Spatial-Adaptive Siamese Residual Network for Multi-/Hyperspectral Classification

2020 ◽  
Vol 12 (10) ◽  
pp. 1640 ◽  
Author(s):  
Zhi He ◽  
Dan He
Keyword(s):  
Large Scale ◽  
Nearest Neighbor ◽  
Three Dimensional ◽  
Classification Performance ◽  
Majority Voting ◽  
Residual Network ◽  
Training Samples ◽  
Simple Linear Iterative Clustering ◽  
Limited Training Samples ◽  
Hyperspectral Classification

Deep learning methods have been successfully applied for multispectral and hyperspectral images classification due to their ability to extract hierarchical abstract features. However, the performance of these methods relies heavily on large-scale training samples. In this paper, we propose a three-dimensional spatial-adaptive Siamese residual network (3D-SaSiResNet) that requires fewer samples and still enhances the performance. The proposed method consists of two main steps: construction of 3D spatial-adaptive patches and Siamese residual network for multiband images classification. In the first step, the spectral dimension of the original multiband images is reduced by a stacked autoencoder and superpixels of each band are obtained by the simple linear iterative clustering (SLIC) method. Superpixels of the original multiband image can be finally generated by majority voting. Subsequently, the 3D spatial-adaptive patch of each pixel is extracted from the original multiband image by reference to the previously generated superpixels. In the second step, a Siamese network composed of two 3D residual networks is designed to extract discriminative features for classification and we train the 3D-SaSiResNet by pairwise inputting the training samples into the networks. The testing samples are then fed into the trained 3D-SaSiResNet and the learned features of the testing samples are classified by the nearest neighbor classifier. Experimental results on three multiband image datasets show the feasibility of the proposed method in enhancing classification performance even with limited training samples.

High-Dimensional Image Data Sets Retrieval: Improving Accuracy Using a Weighted Relevance Feedback

2015 ◽  
Vol 09 (02) ◽  
pp. 239-259
Author(s):  
Abir Gallas ◽  
Walid Barhoumi ◽  
Ezzeddine Zagrouba
Keyword(s):  
Relevance Feedback ◽  
Large Scale ◽  
Nearest Neighbor ◽  
Image Data ◽  
High Dimensional ◽  
General Context ◽  
Query Image ◽  
Data Set ◽  
Dimensional Image ◽  
Minimum Number

The user's interaction with the retrieval engines, while seeking a particular image (or set of images) in large-scale databases, defines better his request. This interaction is essentially provided by a relevance feedback step. In fact, the semantic gap is increasing in a remarkable way due to the application of approximate nearest neighbor (ANN) algorithms aiming at resolving the curse of dimensionality. Therefore, an additional step of relevance feedback is necessary in order to get closer to the user's expectations in the next few retrieval iterations. In this context, this paper details a classification of the different relevance feedback techniques related to region-based image retrieval applications. Moreover, a technique of relevance feedback based on re-weighting regions of the query-image by selecting a set of negative examples is elaborated. Furthermore, the general context to carry out this technique which is the large-scale heterogeneous image collections indexing and retrieval is presented. In fact, the main contribution of the proposed work is affording efficient results with the minimum number of relevance feedback iterations for high dimensional image databases. Experiments and assessments are carried out within an RBIR system for "Wang" data set in order to prove the effectiveness of the proposed approaches.

scholarly journals New Estimators of the Bayes Factor for Models with High-Dimensional Parameter and/or Latent Variable Spaces

Entropy ◽  
2021 ◽  
Vol 23 (4) ◽  
pp. 399
Author(s):  
Anna Pajor
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Bayes Factor ◽  
High Dimensional ◽  
Model Parameters ◽  
Posterior Odds ◽  
Dimensional Parameter ◽  
Data Set ◽  
Multivariate Stochastic Volatility ◽  
Competing Models

Formal Bayesian comparison of two competing models, based on the posterior odds ratio, amounts to estimation of the Bayes factor, which is equal to the ratio of respective two marginal data density values. In models with a large number of parameters and/or latent variables, they are expressed by high-dimensional integrals, which are often computationally infeasible. Therefore, other methods of evaluation of the Bayes factor are needed. In this paper, a new method of estimation of the Bayes factor is proposed. Simulation examples confirm good performance of the proposed estimators. Finally, these new estimators are used to formally compare different hybrid Multivariate Stochastic Volatility–Multivariate Generalized Autoregressive Conditional Heteroskedasticity (MSV-MGARCH) models which have a large number of latent variables. The empirical results show, among other things, that the validity of reduction of the hybrid MSV-MGARCH model to the MGARCH specification depends on the analyzed data set as well as on prior assumptions about model parameters.

scholarly journals Biological data annotation via a human-augmenting AI-based labeling system

2021 ◽  
Vol 4 (1) ◽  
Author(s):  
Douwe van der Wal ◽  
Iny Jhun ◽  
Israa Laklouk ◽  
Jeff Nirschl ◽  
Lara Richer ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Microscopic Analysis ◽  
Image Data ◽  
Cell Types ◽  
Biological Data ◽  
Data Sets ◽  
Data Set ◽  
Data Annotation ◽  
Labeling System

AbstractBiology has become a prime area for the deployment of deep learning and artificial intelligence (AI), enabled largely by the massive data sets that the field can generate. Key to most AI tasks is the availability of a sufficiently large, labeled data set with which to train AI models. In the context of microscopy, it is easy to generate image data sets containing millions of cells and structures. However, it is challenging to obtain large-scale high-quality annotations for AI models. Here, we present HALS (Human-Augmenting Labeling System), a human-in-the-loop data labeling AI, which begins uninitialized and learns annotations from a human, in real-time. Using a multi-part AI composed of three deep learning models, HALS learns from just a few examples and immediately decreases the workload of the annotator, while increasing the quality of their annotations. Using a highly repetitive use-case—annotating cell types—and running experiments with seven pathologists—experts at the microscopic analysis of biological specimens—we demonstrate a manual work reduction of 90.60%, and an average data-quality boost of 4.34%, measured across four use-cases and two tissue stain types.

scholarly journals Characterizing the nonlinear structure of shared variability in cortical neuron populations using latent variable models

2018 ◽  
Author(s):  
Matthew R Whiteway ◽  
Karolina Socha ◽  
Vincent Bonin ◽  
Daniel A Butts
Keyword(s):  
Neural Activity ◽  
Latent Variables ◽  
Latent Variable ◽  
Large Scale ◽  
Latent Variable Models ◽  
Neural Population ◽  
Variable Model ◽  
Experimental Conditions ◽  
Affine Model ◽  
Nonlinear Latent Variable Models

AbstractSensory neurons often have variable responses to repeated presentations of the same stimulus, which can significantly degrade the information contained in those responses. Such variability is often shared across many neurons, which in principle can allow a decoder to mitigate the effects of such noise, depending on the structure of the shared variability and its relationship to sensory encoding at the population level. Latent variable models offer an approach for characterizing the structure of this shared variability in neural population recordings, although they have thus far typically been used under restrictive mathematical assumptions, such as assuming linear transformations between the latent variables and neural activity. Here we leverage recent advances in machine learning to introduce two nonlinear latent variable models for analyzing large-scale neural recordings. We first present a general nonlinear latent variable model that is agnostic to the stimulus tuning properties of the individual neurons, and is hence well suited for exploring neural populations whose tuning properties are not well characterized. This motivates a second class of model, the Generalized Affine Model, which simultaneously determines each neuron’s stimulus selectivity and a set of latent variables that modulate these stimulus responses both additively and multiplicatively. While these approaches can detect general nonlinear relationships in shared neural variability, we find that neural activity recorded in anesthetized primary visual cortex (V1) is best described by a single additive and single multiplicative latent variable, i.e., an “affine model”. In contrast, application of the same models to recordings in awake macaque prefrontal cortex discover more general nonlinearities to compactly describe the population response variability. These results thus demonstrate how nonlinear latent variable models can be used to describe population variability, and suggest that a range of methods is necessary to study different brain regions under different experimental conditions.

Large-scale public lidar and satellite image data set for urban semantic labeling

2018 ◽  
Author(s):  
Myron Z. Brown ◽  
Kevin Foster ◽  
Scott Almes ◽  
Shea Hagstrom ◽  
Marc Bosch ◽  
...  
Keyword(s):  
Large Scale ◽  
Satellite Image ◽  
Image Data ◽  
Data Set ◽  
Semantic Labeling ◽  
Satellite Image Data

scholarly journals Multiple skills underlie arithmetic performance: A large-scale structural equation modeling analysis

2017 ◽  
Vol 3 (2) ◽  
pp. 496-515 ◽  
Author(s):  
Sarit Ashkenazi ◽  
Sarit Silverman
Keyword(s):  
Structural Equation Modeling ◽  
Direct Effect ◽  
Latent Variables ◽  
Latent Variable ◽  
Structural Equation ◽  
Large Scale ◽  
Mathematics Performance ◽  
Equation Modeling ◽  
Theoretical Approaches ◽  
Wide Range

Current theoretical approaches point to the importance of several cognitive skills not specific to mathematics for the etiology of mathematics disorders (MD). In the current study, we examined the role of many of these skills, specifically: rapid automatized naming, attention, reading, and visual perception, on mathematics performance among a large group of college students (N = 1,322) with a wide range of arithmetic proficiency. Using factor analysis, we discovered that our data clustered to four latent variables 1) mathematics, 2) perception speed, 3) attention and 4) reading. In subsequent structural equation modeling, we found that the latent variable perception speed had a strong and meaningful effect on mathematics performance. Moreover, sustained attention, independent from the effect of the latent variable perception speed, had a meaningful, direct effect on arithmetic fact retrieval and procedural knowledge. The latent variable reading had a modest effect on mathematics performance. Specifically, reading comprehension, independent from the effect of the latent variable reading, had a meaningful direct effect on mathematics, and particularly on number line knowledge. Attention, tested by the attention network test, had no effect on mathematics, reading or perception speed. These results indicate that multiple factors can affect mathematics performance supporting a heterogeneous approach to mathematics. These results have meaningful implications for the diagnosis and intervention of pure and comorbid learning disorders.

Sign in / Sign up

Export Citation Format

Share Document