Affective image classification by jointly using low-level visual features and interpretable aesthetic features

AbstractHumans recognize individual faces regardless of variation in the facial view. The view-tuned face neurons in the inferior temporal (IT) cortex are regarded as the neural substrate for view-invariant face recognition. This study approximated visual features encoded by these neurons as combinations of local orientations and colors, originated from natural image fragments. The resultant features reproduced the preference of these neurons to particular facial views. We also found that faces of one identity were separable from the faces of other identities in a space where each axis represented one of these features. These results suggested that view-invariant face representation was established by combining view sensitive visual features. The face representation with these features suggested that, with respect to view-invariant face representation, the seemingly complex and deeply layered ventral visual pathway can be approximated via a shallow network, comprised of layers of low-level processing for local orientations and colors (V1/V2-level) and the layers which detect particular sets of low-level elements derived from natural image fragments (IT-level).

Download Full-text

Multiview Discriminative Geometry Preserving Projection for Image Classification

The Scientific World JOURNAL ◽

10.1155/2014/924090 ◽

2014 ◽

Vol 2014 ◽

pp. 1-11 ◽

Cited By ~ 3

Author(s):

Ziqiang Wang ◽

Xia Sun ◽

Lijun Sun ◽

Yuchun Huang

Keyword(s):

Image Classification ◽

Learning Algorithm ◽

Subspace Learning ◽

Visual Features ◽

Expression Recognition ◽

Single View ◽

Discrimination Information ◽

New Feature ◽

Low Dimensional ◽

Conventional Solution

In many image classification applications, it is common to extract multiple visual features from different views to describe an image. Since different visual features have their own specific statistical properties and discriminative powers for image classification, the conventional solution for multiple view data is to concatenate these feature vectors as a new feature vector. However, this simple concatenation strategy not only ignores the complementary nature of different views, but also ends up with “curse of dimensionality.” To address this problem, we propose a novel multiview subspace learning algorithm in this paper, named multiview discriminative geometry preserving projection (MDGPP) for feature extraction and classification. MDGPP can not only preserve the intraclass geometry and interclass discrimination information under a single view, but also explore the complementary property of different views to obtain a low-dimensional optimal consensus embedding by using an alternating-optimization-based iterative algorithm. Experimental results on face recognition and facial expression recognition demonstrate the effectiveness of the proposed algorithm.

Download Full-text

Bimodal fusion of low-level visual features and high-level semantic features for near-duplicate video clip detection

Signal Processing Image Communication ◽

10.1016/j.image.2011.04.001 ◽

2011 ◽

Vol 26 (10) ◽

pp. 612-627 ◽

Cited By ~ 2

Author(s):

Hyun-seok Min ◽

Jae Young Choi ◽

Wesley De Neve ◽

Yong Man Ro

Keyword(s):

Video Clip ◽

Visual Features ◽

Semantic Features ◽

Low Level ◽

High Level ◽

Duplicate Video

Download Full-text

Robust Image Labeling Using Conditional Random Fields

10.32920/ryerson.14651541 ◽

2021 ◽

Author(s):

Maryam Nematollahi Arani

Keyword(s):

Object Recognition ◽

Mixture Models ◽

Random Fields ◽

Conditional Random Fields ◽

Semantic Gap ◽

Visual Features ◽

Image Labeling ◽

Low Level ◽

Robust Image ◽

High Level

Object recognition has become a central topic in computer vision applications such as image search, robotics and vehicle safety systems. However, it is a challenging task due to the limited discriminative power of low-level visual features in describing the considerably diverse range of high-level visual semantics of objects. Semantic gap between low-level visual features and high-level concepts are a bottleneck in most systems. New content analysis models need to be developed to bridge the semantic gap. In this thesis, algorithms based on conditional random fields (CRF) from the class of probabilistic graphical models are developed to tackle the problem of multiclass image labeling for object recognition. Image labeling assigns a specific semantic category from a predefined set of object classes to each pixel in the image. By well capturing spatial interactions of visual concepts, CRF modeling has proved to be a successful tool for image labeling. This thesis proposes novel approaches to empowering the CRF modeling for robust image labeling. Our primary contributions are twofold. To better represent feature distributions of CRF potentials, new feature functions based on generalized Gaussian mixture models (GGMM) are designed and their efficacy is investigated. Due to its shape parameter, GGMM can provide a proper fit to multi-modal and skewed distribution of data in nature images. The new model proves more successful than Gaussian and Laplacian mixture models. It also outperforms a deep neural network model on Corel imageset by 1% accuracy. Further in this thesis, we apply scene level contextual information to integrate global visual semantics of the image with pixel-wise dense inference of fully-connected CRF to preserve small objects of foreground classes and to make dense inference robust to initial misclassifications of the unary classifier. Proposed inference algorithm factorizes the joint probability of labeling configuration and image scene type to obtain prediction update equations for labeling individual image pixels and also the overall scene type of the image. The proposed context-based dense CRF model outperforms conventional dense CRF model by about 2% in terms of labeling accuracy on MSRC imageset and by 4% on SIFT Flow imageset. Also, the proposed model obtains the highest scene classification rate of 86% on MSRC dataset.

Download Full-text

Applying LCS To Affective Image Classification In Spatial-Frequency Domain

Journal of Artificial Intelligence and Soft Computing Research ◽

10.1515/jaiscr-2015-0002 ◽

2014 ◽

Vol 4 (2) ◽

pp. 99-123 ◽

Cited By ~ 8

Author(s):

Po-Ming Lee ◽

Tzu-Chien Hsiao

Keyword(s):

Pattern Recognition ◽

Spatial Frequency ◽

Image Classification ◽

Frequency Domain ◽

Model Building ◽

Experimental Paradigm ◽

Wide Range ◽

Classification Tasks ◽

Affective Image Classification ◽

Spatial Frequency Domain

Abstract Recent studies have utilizes color, texture, and composition information of images to achieve affective image classification. However, the features related to spatial-frequency domain that were proven to be useful for traditional pattern recognition have not been tested in this field yet. Furthermore, the experiments conducted by previous studies are not internationally-comparable due to the experimental paradigm adopted. In addition, contributed by recent advances in methodology, that are, Hilbert-Huang Transform (HHT) (i.e. Empirical Mode Decomposition (EMD) and Hilbert Transform (HT)), the resolution of frequency analysis has been improved. Hence, the goal of this research is to achieve the affective image-classification task by adopting a standard experimental paradigm introduces by psychologists in order to produce international-comparable and reproducible results; and also to explore the affective hidden patterns of images in the spatial-frequency domain. To accomplish these goals, multiple human-subject experiments were conducted in laboratory. Extended Classifier Systems (XCSs) was used for model building because the XCS has been applied to a wide range of classification tasks and proved to be competitive in pattern recognition. To exploit the information in the spatial-frequency domain, the traditional EMD has been extended to a two-dimensional version. To summarize, the model built by using the XCS achieves Area Under Curve (AUC) = 0.91 and accuracy rate over 86%. The result of the XCS was compared with other traditional machine-learning algorithms (e.g., Radial-Basis Function Network (RBF Network)) that are normally used for classification tasks. Contributed by proper selection of features for model building, user-independent findings were obtained. For example, it is found that the horizontal visual stimulations contribute more to the emotion elicitation than the vertical visual stimulation. The effect of hue, saturation, and brightness; is also presented.

Download Full-text

Toward Semantically Meaningful Feature Spaces for Efficient Indexing in Large Image Databases

Managing Multimedia Semantics ◽

10.4018/978-1-59140-569-6.ch001 ◽

2011 ◽

pp. 1-29

Author(s):

Anne H.H. Ngu ◽

Jialie Shen ◽

John Shepherd

Keyword(s):

Distance Function ◽

Image Features ◽

Image Databases ◽

Multimedia Databases ◽

Visual Features ◽

Hybrid Architecture ◽

Low Level ◽

Feature Vectors ◽

Training Cost ◽

Low Dimensional

The optimized distance-based access methods currently available for multimedia databases are based on two major assumptions: a suitable distance function is known a priori, and the dimensionality of image features is low. The standard approach to building image databases is to represent images via vectors based on low-level visual features and make retrieval based on these vectors. However, due to the large gap between the semantic notions and low-level visual content, it is extremely difficult to define a distance function that accurately captures the similarity of images as perceived by humans. Furthermore, popular dimension reduction methods suffer from either the inability to capture the nonlinear correlations among raw data or very expensive training cost. To address the problems, in this chapter we introduce a new indexing technique called Combining Multiple Visual Features (CMVF) that integrates multiple visual features to get better query effectiveness. Our approach is able to produce low-dimensional image feature vectors that include not only low-level visual properties but also high-level semantic properties. The hybrid architecture can produce feature vectors that capture the salient properties of images yet are small enough to allow the use of existing high-dimensional indexing methods to provide efficient and effective retrieval.

Download Full-text