Large-scale interactive retrieval in art collections using multi-style feature aggregation

Finding objects and motifs across artworks is of great importance for art history as it helps to understand individual works and analyze relations between them. The advent of digitization has produced extensive digital art collections with many research opportunities. However, manual approaches are inadequate to handle this amount of data, and it requires appropriate computer-based methods to analyze them. This article presents a visual search algorithm and user interface to support art historians to find objects and motifs in extensive datasets. Artistic image collections are subject to significant domain shifts induced by large variations in styles, artistic media, and materials. This poses new challenges to most computer vision models which are trained on photographs. To alleviate this problem, we introduce a multi-style feature aggregation that projects images into the same distribution, leading to more accurate and style-invariant search results. Our retrieval system is based on a voting procedure combined with fast nearest-neighbor search and enables finding and localizing motifs within an extensive image collection in seconds. The presented approach significantly improves the state-of-the-art in terms of accuracy and search time on various datasets and applies to large and inhomogeneous collections. In addition to the search algorithm, we introduce a user interface that allows art historians to apply our algorithm in practice. The interface enables users to search for single regions, multiple regions regarding different connection types and holds an interactive feedback system to improve retrieval results further. With our methodological contribution and easy-to-use user interface, this work manifests further progress towards a computer-based analysis of visual art.

Download Full-text

Accurate and Efficient CPU/GPU-Based 3-DOF Haptic Rendering of Complex Static Virtual Environments

Presence Teleoperators & Virtual Environments ◽

10.1162/pres.18.5.340 ◽

2009 ◽

Vol 18 (5) ◽

pp. 340-360 ◽

Cited By ~ 3

Author(s):

Jong-Phil Kim ◽

Beom-Chan Lee ◽

Hyungon Kim ◽

Jaeha Kim ◽

Jeha Ryu

Keyword(s):

Virtual Environments ◽

Large Scale ◽

Search Algorithm ◽

Search Time ◽

Haptic Rendering ◽

Nearest Neighbor Search ◽

Local Geometry ◽

Complex Data ◽

Haptic Interaction ◽

Depth Maps

This paper proposes a novel, accurate, and efficient hybrid CPU/GPU-based 3-DOF haptic rendering algorithm for highly complex and large-scale virtual environments (VEs) that may simultaneously contain different types of object data representations. In a slower rendering process on the GPU, local geometry near the haptic interaction point (HIP) is obtained in the form of six directional depth maps from virtual cameras adaptively located around the object to be touched. In a faster rendering process on the CPU, collision detection and response computations are performed using the directional depth maps without the need for any complex data hierarchy of virtual objects, or data conversion of multiple data formats. To efficiently find an ideal HIP (IHIP), the proposed algorithm uses a new “abstract” local occupancy map instance (LOMI) and the nearest neighbor search algorithm, which does not require physical memory for storing voxel types during online voxelization and reduces the search time by a factor of about 10. Finally, in order to achieve accurate haptic interaction, sub-voxelization of a voxel in LOMI is proposed. The effectiveness of the proposed algorithm is subsequently demonstrated with several benchmark examples.

Download Full-text

Context-aware Adaptive Surgery

Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies ◽

10.1145/3478073 ◽

2021 ◽

Vol 5 (3) ◽

pp. 1-22

Author(s):

Hongli Wang ◽

Bin Guo ◽

Jiaqi Liu ◽

Sicong Liu ◽

Yungang Wu ◽

...

Keyword(s):

Real Time ◽

Large Scale ◽

Resource Constraints ◽

Search Algorithm ◽

Search Time ◽

State Graph ◽

Context Aware ◽

Optimal Partition ◽

Research Attention ◽

Neighbor Effect

Deep Neural Networks (DNNs) have made massive progress in many fields and deploying DNNs on end devices has become an emerging trend to make intelligence closer to users. However, it is challenging to deploy large-scale and computation-intensive DNNs on resource-constrained end devices due to their small size and lightweight. To this end, model partition, which aims to partition DNNs into multiple parts to realize the collaborative computing of multiple devices, has received extensive research attention. To find the optimal partition, most existing approaches need to run from scratch under given resource constraints. However, they ignore that resources of devices (e.g., storage, battery power), and performance requirements (e.g., inference latency), are often continuously changing, making the optimal partition solution change constantly during processing. Therefore, it is very important to reduce the tuning latency of model partition to realize the real-time adaption under the changing processing context. To address these problems, we propose the Context-aware Adaptive Surgery (CAS) framework to actively perceive the changing processing context, and adaptively find the appropriate partition solution in real-time. Specifically, we construct the partition state graph to comprehensively model different partition solutions of DNNs by import context resources. Then "the neighbor effect" is proposed, which provides the heuristic rule for the search process. When the processing context changes, CAS adopts the runtime search algorithm, Graph-based Adaptive DNN Surgery (GADS), to quickly find the appropriate partition that satisfies resource constraints under the guidance of the neighbor effect. The experimental results show that CAS realizes adaptively rapid tuning of the model partition solutions in 10ms scale even for large DNNs (2.25x to 221.7x search time improvement than the state-of-the-art researches), and the total inference latency still keeps the same level with baselines.

Download Full-text

Complexity of Error-Correcting Code Based on Nearest Neighbor Search Algorithm

Mathematical Problems of Computer Science ◽

10.51408/1963-0030 ◽

2019 ◽

pp. 7-20

Author(s):

Levon Arsalanyan ◽

Hayk Danoyan

Keyword(s):

Time Complexity ◽

Nearest Neighbor ◽

Search Algorithm ◽

Error Correcting Code ◽

Error Correcting Codes ◽

Code Word ◽

Nearest Neighbor Search ◽

Perfect Codes ◽

Neighbor Search ◽

Weight Structures

The Nearest Neighbor search algorithm considered in this paper is well known (Elias algorithm). It uses error-correcting codes and constructs appropriate hash-coding schemas. These schemas preprocess the data in the form of lists. Each list is contained in some sphere, centered at a code-word. The algorithm is considered for the cases of perfect codes, so the spheres and, consequently, the lists do not intersect. As such codes exist for the limited set of parameters, the algorithm is considered for some other generalizations of perfect codes, and then the same data point may be contained in different lists. A formula of time complexity of the algorithm is obtained for these cases, using coset weight structures of the mentioned codes

Download Full-text

A Meta-Algorithm for Improving Top-N Prediction Efficiency of Matrix Factorization Models in Collaborative Filtering

International Journal of Pattern Recognition and Artificial Intelligence ◽

10.1142/s0218001420590077 ◽

2019 ◽

Vol 34 (03) ◽

pp. 2059007

Author(s):

A. Murat Yagci ◽

Tevfik Aytekin ◽

Fikret S. Gurgen

Keyword(s):

Collaborative Filtering ◽

Matrix Factorization ◽

Large Scale ◽

Nearest Neighbor ◽

Nearest Neighbor Search ◽

Space Efficiency ◽

Neighbor Search ◽

Prediction Time ◽

Low Dimensional ◽

Prediction Efficiency

Matrix factorization models often reveal the low-dimensional latent structure in high-dimensional spaces while bringing space efficiency to large-scale collaborative filtering problems. Improving training and prediction time efficiencies of these models are also important since an accurate model may raise practical concerns if it is slow to capture the changing dynamics of the system. For the training task, powerful improvements have been proposed especially using SGD, ALS, and their parallel versions. In this paper, we focus on the prediction task and combine matrix factorization with approximate nearest neighbor search methods to improve the efficiency of top-N prediction queries. Our efforts result in a meta-algorithm, MMFNN, which can employ various common matrix factorization models, drastically improve their prediction efficiency, and still perform comparably to standard prediction approaches or sometimes even better in terms of predictive power. Using various batch, online, and incremental matrix factorization models, we present detailed empirical analysis results on many large implicit feedback datasets from different application domains.

Download Full-text