scholarly journals Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0255109
Author(s):  
Mitchell J. P. Van Zuijlen ◽  
Hubert Lin ◽  
Kavita Bala ◽  
Sylvia C. Pont ◽  
Maarten W. A. Wijntjes
Keyword(s):  
Computer Vision ◽  
Art History ◽  
Human Perception ◽  
Feature Representation ◽  
Rich Source ◽  
Relevant Feature ◽  
Perceptual Cues ◽  
Multidisciplinary Research ◽  
Fine Grained ◽  
Bounding Boxes

In this paper, we capture and explore the painterly depictions of materials to enable the study of depiction and perception of materials through the artists’ eye. We annotated a dataset of 19k paintings with 200k+ bounding boxes from which polygon segments were automatically extracted. Each bounding box was assigned a coarse material label (e.g., fabric) and half was also assigned a fine-grained label (e.g., velvety, silky). The dataset in its entirety is available for browsing and downloading at materialsinpaintings.tudelft.nl. We demonstrate the cross-disciplinary utility of our dataset by presenting novel findings across human perception, art history and, computer vision. Our experiments include a demonstration of how painters create convincing depictions using a stylized approach. We further provide an analysis of the spatial and probabilistic distributions of materials depicted in paintings, in which we for example show that strong patterns exists for material presence and location. Furthermore, we demonstrate how paintings could be used to build more robust computer vision classifiers by learning a more perceptually relevant feature representation. Additionally, we demonstrate that training classifiers on paintings could be used to uncover hidden perceptual cues by visualizing the features used by the classifiers. We conclude that our dataset of painterly material depictions is a rich source for gaining insights into the depiction and perception of materials across multiple disciplines and hope that the release of this dataset will drive multidisciplinary research.

Hierarchical Quality-Relevant Feature Representation for Soft Sensor Modeling: A Novel Deep Learning Strategy

2020 ◽  
Vol 16 (6) ◽  
pp. 3721-3730 ◽  
Author(s):  
Xiaofeng Yuan ◽  
Jiao Zhou ◽  
Biao Huang ◽  
Yalin Wang ◽  
Chunhua Yang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Learning Strategy ◽  
Soft Sensor ◽  
Feature Representation ◽  
Relevant Feature ◽  
Sensor Modeling

scholarly journals Improving Topic Coherence Using Entity Extraction Denoising

2018 ◽  
Vol 110 (1) ◽  
pp. 85-101 ◽  
Author(s):  
Ronald Cardenas ◽  
Kevin Bello ◽  
Alberto Coronado ◽  
Elizabeth Villota
Keyword(s):  
Topic Modeling ◽  
Human Perception ◽  
Relevant Information ◽  
Entity Recognition ◽  
Entity Extraction ◽  
Fine Grained ◽  
Job Advertisement ◽  
Coherence Score ◽  
Probabilistic Topic Modeling ◽  
Promising Solution

Abstract Managing large collections of documents is an important problem for many areas of science, industry, and culture. Probabilistic topic modeling offers a promising solution. Topic modeling is an unsupervised machine learning method and the evaluation of this model is an interesting problem on its own. Topic interpretability measures have been developed in recent years as a more natural option for topic quality evaluation, emulating human perception of coherence with word sets correlation scores. In this paper, we show experimental evidence of the improvement of topic coherence score by restricting the training corpus to that of relevant information in the document obtained by Entity Recognition. We experiment with job advertisement data and find that with this approach topic models improve interpretability in about 40 percentage points on average. Our analysis reveals as well that using the extracted text chunks, some redundant topics are joined while others are split into more skill-specific topics. Fine-grained topics observed in models using the whole text are preserved.

scholarly journals Hybrid Attention Network for Language-Based Person Search

Sensors ◽  
2020 ◽  
Vol 20 (18) ◽  
pp. 5279
Author(s):  
Yang Li ◽  
Huahu Xu ◽  
Junsheng Xiao
Keyword(s):  
Image Features ◽  
Attention Mechanism ◽  
Feature Representation ◽  
Semantic Features ◽  
Retrieval Task ◽  
Attention Network ◽  
Fine Grained ◽  
Person Search ◽  
High Level ◽  
Language Description

Language-based person search retrieves images of a target person using natural language description and is a challenging fine-grained cross-modal retrieval task. A novel hybrid attention network is proposed for the task. The network includes the following three aspects: First, a cubic attention mechanism for person image, which combines cross-layer spatial attention and channel attention. It can fully excavate both important midlevel details and key high-level semantics to obtain better discriminative fine-grained feature representation of a person image. Second, a text attention network for language description, which is based on bidirectional LSTM (BiLSTM) and self-attention mechanism. It can better learn the bidirectional semantic dependency and capture the key words of sentences, so as to extract the context information and key semantic features of the language description more effectively and accurately. Third, a cross-modal attention mechanism and a joint loss function for cross-modal learning, which can pay more attention to the relevant parts between text and image features. It can better exploit both the cross-modal and intra-modal correlation and can better solve the problem of cross-modal heterogeneity. Extensive experiments have been conducted on the CUHK-PEDES dataset. Our approach obtains higher performance than state-of-the-art approaches, demonstrating the advantage of the approach we propose.

scholarly journals ORIENTED VEHICLE DETECTION IN HIGH-RESOLUTION REMOTE SENSING IMAGES BASED ON FEATURE AMPLIFICATION AND CATEGORY BALANCE BY OVERSAMPLING DATA AUGMENTATION

2020 ◽  
Vol XLIII-B3-2020 ◽  
pp. 153-159
Author(s):  
N. Mo ◽  
L. Yan
Keyword(s):  
Remote Sensing ◽  
High Resolution ◽  
Vehicle Detection ◽  
Training Dataset ◽  
Remote Sensing Images ◽  
Feature Maps ◽  
Fine Grained ◽  
Bounding Boxes ◽  
The Impact ◽  
Oriented Bounding Boxes

Abstract. Vehicles usually lack detailed information and are difficult to be trained on the high-resolution remote sensing images because of small size. In addition, vehicles contain multiple fine-grained categories that are slightly different, randomly located and oriented. Therefore, it is difficult to locate and identify these fine categories of vehicles. Considering the above problems in high-resolution remote sensing images, this paper proposes an oriented vehicle detection approach. First of all, we propose an oversampling and stitching method to augment the training dataset by increasing the frequency of objects with fewer training samples in order to balance the number of objects in each fine-grained vehicle category. Then considering the effect of the pooling operations on representing small objects, we propose to improve the resolution of feature maps so that detailed information hidden in feature maps can be enriched and they can better distinguish the fine-grained vehicle categories. Finally, we design a joint training loss function for horizontal and oriented bounding boxes with center loss, to decrease the impact of small between-class diversity on vehicle detection. Experimental verification is performed on the VEDAI dataset consisting of 9 fine-grained vehicle categories so as to evaluate the proposed framework. The experimental results show that the proposed framework performs better than most of competitive approaches in terms of a mean average precision of 60.7% and 60.4% in detecting horizontal and oriented bounding boxes respectively.

scholarly journals Enhancing the Human with Computers: Ongoing research at the Computer Graphics, Image Processing and Interaction Group

2011 ◽  
Vol 2 (2) ◽  
pp. 1
Author(s):  
Luciana Nedel ◽  
Anderson Maciel ◽  
Carla Dal Sasso Freitas ◽  
Claudio Jung ◽  
Manuel Oliveira ◽  
...  
Keyword(s):  
Image Processing ◽  
Computer Vision ◽  
Computer Graphics ◽  
Data Visualization ◽  
Human Perception ◽  
Natural World ◽  
Massive Data ◽  
3D Interaction ◽  
Ongoing Research ◽  
Surgery Planning

The Computer Graphics, Image Processing and Interaction (CGIP) group at UFRGS concentrates expertise from many different and complementary graphics related domains. In this paper we introduce the group and present our re- search lines and some ongoing projects. We selected mainly the projects related to 3D interaction and navigation, which includes applications as massive data visualization, surgery planning and simulation, tracking and computer vision algorithms, and modeling approaches for human perception and natural world.

scholarly journals DGANet: A Dilated Graph Attention-Based Network for Local Feature Extraction on 3D Point Clouds

2021 ◽  
Vol 13 (17) ◽  
pp. 3484
Author(s):  
Jie Wan ◽  
Zhong Xie ◽  
Yongyang Xu ◽  
Ziyin Zeng ◽  
Ding Yuan ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Long Range ◽  
Point Clouds ◽  
Feature Representation ◽  
Learning Ability ◽  
Geometric Feature ◽  
Cloud Data ◽  
Fine Grained ◽  
3D Point Clouds ◽  
Benchmark Datasets

Feature extraction on point clouds is an essential task when analyzing and processing point clouds of 3D scenes. However, there still remains a challenge to adequately exploit local fine-grained features on point cloud data due to its irregular and unordered structure in a 3D space. To alleviate this problem, a Dilated Graph Attention-based Network (DGANet) with a certain feature for learning ability is proposed. Specifically, we first build a local dilated graph-like region for each input point to establish the long-range spatial correlation towards its corresponding neighbors, which allows the proposed network to access a wider range of geometric information of local points with their long-range dependencies. Moreover, by integrating the dilated graph attention module (DGAM) implemented by a novel offset–attention mechanism, the proposed network promises to highlight the differing ability of each edge of the constructed local graph to uniquely learn the discrepancy feature of geometric attributes between the connected point pairs. Finally, all the learned edge attention features are further aggregated, allowing the most significant geometric feature representation of local regions by the graph–attention pooling to fully extract local detailed features for each point. The validation experiments using two challenging benchmark datasets demonstrate the effectiveness and powerful generation ability of our proposed DGANet in both 3D object classification and segmentation tasks.

scholarly journals Towards Optimal Fine Grained Retrieval via Decorrelated Centralized Loss with Normalize-Scale Layer

2019 ◽  
Vol 33 ◽  
pp. 9291-9298 ◽  
Author(s):  
Xiawu Zheng ◽  
Rongrong Ji ◽  
Xiaoshuai Sun ◽  
Baochang Zhang ◽  
Yongjian Wu ◽  
...  
Keyword(s):  
Metric Learning ◽  
Feature Representation ◽  
Superior Performance ◽  
Pairwise Distance ◽  
Training Procedure ◽  
Fine Grained ◽  
Scale Layer ◽  
Low Efficiency ◽  
Metric Distance ◽  
Ranking Loss

Recent advances on fine-grained image retrieval prefer learning convolutional neural network (CNN) with specific fullyconnect layer designed loss function for discriminative feature representation. Essentially, such loss should establish a robust metric to efficiently distinguish high-dimensional features within and outside fine-grained categories. To this end, the existing loss functions are defected in two aspects: (a) The feature relationship is encoded inside the training batch. Such a local scope leads to low accuracy. (b) The error is established by the mean square, which needs pairwise distance computation in training set and results in low efficiency. In this paper, we propose a novel metric learning scheme, termed Normalize-Scale Layer and Decorrelated Global Centralized Ranking Loss, which achieves extremely efficient and discriminative learning, i.e., 5× speedup over triplet loss and 12% recall boost on CARS196. Our method originates from the classic softmax loss, which has a global structure but does not directly optimize the distance metric as well as the inter/intra class distance. We tackle this issue through a hypersphere layer and a global centralized ranking loss with a pairwise decorrelated learning. In particular, we first propose a Normalize-Scale Layer to eliminate the gap between metric distance (for measuring distance in retrieval) and dot product (for dimension reduction in classification). Second, the relationship between features is encoded under a global centralized ranking loss, which targets at optimizing metric distance globally and accelerating learning procedure. Finally, the centers are further decorrelated by Gram-Schmidt process, leading to extreme efficiency (with 20 epochs in training procedure) and discriminability in feature learning. We have conducted quantitative evaluations on two fine-grained retrieval benchmark. The superior performance demonstrates the merits of the proposed approach over the state-of-the-arts.

Sign in / Sign up

Export Citation Format

Share Document