Large Scale Multimedia Management: A Comprehensive Review

Multimedia applications deal, in most cases, with an extremely high volume of multimedia data (2D and 3D images, sounds, videos). That is why efficient algorithms should be developed to analyze and process these large datasets. On the other hand, multimedia management is based on efficient representation of knowledge which allows efficient data processing and retrieval. The main challenge in this era is to achieve clever and quick access to these huge datasets to allow easy access to the data and in a reasonable time. In this context, large-scale image retrieval is a fundamental task. Many methods have been developed in the literature to achieve fast and efficient navigating in large databases by using the famous content-based image retrieval (CBIR) methods associated with these methods allowing a decrease in the computing time, such as dimensional reduction and hashing methods. More recently, these methods based on convolutional neural networks (CNNs) for feature extraction and image classification are widely used. In this paper, we present a comprehensive review of recent multimedia retrieval methods and algorithms applied to large datasets of 2D/3D images and videos. This editorial paper discusses the mains challenges of multimedia retrieval in a context of large databases.

Download Full-text

Efficient Multimedia Similarity Measurement Using Similar Elements

10.20944/preprints201909.0232.v1 ◽

2019 ◽

Author(s):

Jun Long ◽

Lei Zhu ◽

Xinpan Yuan ◽

Longzhi Sun

Keyword(s):

Large Scale ◽

Multimedia Data ◽

Multimedia Retrieval ◽

Similarity Measurement ◽

Visual Words ◽

Indexing Structure ◽

Measurement Framework ◽

The Arts ◽

Definition Of ◽

Similar Elements

Online social networking techniques and large-scale multimedia retrieval are developing rapidly, which not only has brought great convenience to our daily life, but generated, collected, and stored large-scale multimedia data as well. This trend has put forward higher requirements and greater challenges on massive multimedia retrieval. In this paper, we investigate the problem of image similarity measurement, which is one of the key problems of multimedia retrieval. Firstly, the definition of similarity measurement of images and the related notions are proposed. Then, an efficient similarity measurement framework is proposed. Besides, we present a novel basic method of similarity measurement named SMIN. To improve the performance of similarity measurement, we carefully design a novel indexing structure called SMI Temp Index (SMII for short). Moreover, we establish an index of potential similar visual words off-line to solve to problem that the index cannot be reused. Experimental evaluations on two real image datasets demonstrate that the proposed approach outperforms state-of-the-arts.

Download Full-text

TDCMR: Triplet-Based Deep Cross-Modal Retrieval for Geo-Multimedia Data

Applied Sciences ◽

10.3390/app112210803 ◽

2021 ◽

Vol 11 (22) ◽

pp. 10803

Author(s):

Jiagang Song ◽

Yunwu Lin ◽

Jiayu Song ◽

Weiren Yu ◽

Leyuan Zhang

Keyword(s):

High Performance ◽

Large Scale ◽

Location Based Services ◽

Multimedia Data ◽

Multimedia Retrieval ◽

Geographical Information ◽

Superior Performance ◽

Hybrid Index ◽

High Level ◽

Hash Codes

Mass multimedia data with geographical information (geo-multimedia) are collected and stored on the Internet due to the wide application of location-based services (LBS). How to find the high-level semantic relationship between geo-multimedia data and construct efficient index is crucial for large-scale geo-multimedia retrieval. To combat this challenge, the paper proposes a deep cross-modal hashing framework for geo-multimedia retrieval, termed as Triplet-based Deep Cross-Modal Retrieval (TDCMR), which utilizes deep neural network and an enhanced triplet constraint to capture high-level semantics. Besides, a novel hybrid index, called TH-Quadtree, is developed by combining cross-modal binary hash codes and quadtree to support high-performance search. Extensive experiments are conducted on three common used benchmarks, and the results show the superior performance of the proposed method.

Download Full-text

Cloud-Based Image Retrieval Using GPU Platforms

Computers ◽

10.3390/computers8020048 ◽

2019 ◽

Vol 8 (2) ◽

pp. 48 ◽

Cited By ~ 2

Author(s):

Sidi Ahmed Mahmoudi ◽

Mohammed Amin Belarbi ◽

El Wardani Dadi ◽

Saïd Mahmoudi ◽

Mohammed Benjelloun

Keyword(s):

Image Retrieval ◽

Performance Improvement ◽

Computation Time ◽

Image Features ◽

Multimedia Retrieval ◽

Large Database ◽

Computation Complexity ◽

Multimedia Documents ◽

Large Databases ◽

Reduction Of Energy Consumption

The process of image retrieval presents an interesting tool for different domains related to computer vision such as multimedia retrieval, pattern recognition, medical imaging, video surveillance and movements analysis. Visual characteristics of images such as color, texture and shape are used to identify the content of images. However, the retrieving process becomes very challenging due to the hard management of large databases in terms of storage, computation complexity, temporal performance and similarity representation. In this paper, we propose a cloud-based platform in which we integrate several features extraction algorithms used for content-based image retrieval (CBIR) systems. Moreover, we propose an efficient combination of SIFT and SURF descriptors that allowed to extract and match image features and hence improve the process of image retrieval. The proposed algorithms have been implemented on the CPU and also adapted to fully exploit the power of GPUs. Our platform is presented with a responsive web solution that offers for users the possibility to exploit, test and evaluate image retrieval methods. The platform offers to users a simple-to-use access for different algorithms such as SIFT, SURF descriptors without the need to setup the environment or install anything while spending minimal efforts on preprocessing and configuring. On the other hand, our cloud-based CPU and GPU implementations are scalable, which means that they can be used even with large database of multimedia documents. The obtained results showed: 1. Precision improvement in terms of recall and precision; 2. Performance improvement in terms of computation time as a result of exploiting GPUs in parallel; 3. Reduction of energy consumption.

Download Full-text

Discrete Semantics-Guided Asymmetric Hashing for Large-Scale Multimedia Retrieval

Applied Sciences ◽

10.3390/app11188769 ◽

2021 ◽

Vol 11 (18) ◽

pp. 8769

Author(s):

Jun Long ◽

Longzhi Sun ◽

Liujie Hua ◽

Zhan Yang

Keyword(s):

Code Generation ◽

Hash Function ◽

Large Scale ◽

Semantic Information ◽

Multimedia Data ◽

Multimedia Retrieval ◽

Coordinate Descent Algorithm ◽

Hash Code ◽

Benchmark Datasets ◽

Hash Codes

Cross-modal hashing technology is a key technology for real-time retrieval of large-scale multimedia data in real-world applications. Although the existing cross-modal hashing methods have achieved impressive accomplishment, there are still some limitations: (1) some cross-modal hashing methods do not make full consider the rich semantic information and noise information in labels, resulting in a large semantic gap, and (2) some cross-modal hashing methods adopt the relaxation-based or discrete cyclic coordinate descent algorithm to solve the discrete constraint problem, resulting in a large quantization error or time consumption. Therefore, in order to solve these limitations, in this paper, we propose a novel method, named Discrete Semantics-Guided Asymmetric Hashing (DSAH). Specifically, our proposed DSAH leverages both label information and similarity matrix to enhance the semantic information of the learned hash codes, and the ℓ2,1 norm is used to increase the sparsity of matrix to solve the problem of the inevitable noise and subjective factors in labels. Meanwhile, an asymmetric hash learning scheme is proposed to efficiently perform hash learning. In addition, a discrete optimization algorithm is proposed to fast solve the hash code directly and discretely. During the optimization process, the hash code learning and the hash function learning interact, i.e., the learned hash codes can guide the learning process of the hash function and the hash function can also guide the hash code generation simultaneously. Extensive experiments performed on two benchmark datasets highlight the superiority of DSAH over several state-of-the-art methods.

Download Full-text

Learning Deep Unsupervised Binary Codes for Image Retrieval

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2018/85 ◽

2018 ◽

Cited By ~ 4

Author(s):

Junjie Chen ◽

William K. Cheung ◽

Anran Wang

Keyword(s):

Image Retrieval ◽

Large Scale ◽

Nearest Neighbor ◽

Nearest Neighbor Search ◽

Multimedia Retrieval ◽

Binary Codes ◽

Linear Transformations ◽

Neighbor Search ◽

Compact Representations ◽

The Difference

Hashing is an efficient approximate nearest neighbor search method and has been widely adopted for large-scale multimedia retrieval. While supervised learning is more popular for the data-dependent hashing, deep unsupervised hashing methods have recently been developed to learn non-linear transformations for converting multimedia inputs to binary codes. Most of existing deep unsupervised hashing methods make use of a quadratic constraint for minimizing the difference between the compact representations and the target binary codes, which inevitably causes severe information loss. In this paper, we propose a novel deep unsupervised method called DeepQuan for hashing. The DeepQuan model utilizes a deep autoencoder network, where the encoder is used to learn compact representations and the decoder is for manifold preservation. To contrast with the existing unsupervised methods, DeepQuan learns the binary codes by minimizing the quantization error through product quantization technique. Furthermore, a weighted triplet loss is proposed to avoid trivial solution and poor generalization. Extensive experimental results on standard datasets show that the proposed DeepQuan model outperforms the state-of-the-art unsupervised hashing methods for image retrieval tasks.

Download Full-text

Efficient Image Retrieval approach for Large-scale Chest X Ray data using Hand-Crafted Features and Machine Learning Algorithms

International Journal of Computer Sciences and Engineering ◽

10.26438/ijcse/v6i11.890896 ◽

2018 ◽

Vol 6 (11) ◽

pp. 890-896

Author(s):

Irene Getzi S ◽

D. Christopher Durairaj ◽

V Joseph Raj

Keyword(s):

Machine Learning ◽

Image Retrieval ◽

Large Scale ◽

Learning Algorithms ◽

Machine Learning Algorithms ◽

X Ray ◽

Chest X Ray

Download Full-text

Large-Scale E-Commerce Image Retrieval with Top-Weighted Convolutional Neural Networks

Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval - ICMR '16 ◽

10.1145/2911996.2912052 ◽

2016 ◽

Cited By ~ 2

Author(s):

Shichao Zhao ◽

Youjiang Xu ◽

Yahong Han

Keyword(s):

Neural Networks ◽

Image Retrieval ◽

Convolutional Neural Networks ◽

Large Scale

Download Full-text

Improved Visual Localization via Graph Filtering

Journal of Imaging ◽

10.3390/jimaging7020020 ◽

2021 ◽

Vol 7 (2) ◽

pp. 20

Author(s):

Carlos Lassance ◽

Yasir Latif ◽

Ravi Garg ◽

Vincent Gripon ◽

Ian Reid

Keyword(s):

Image Retrieval ◽

Large Scale ◽

Visual Localization ◽

Localization Accuracy ◽

Additional Information ◽

Feature Representations ◽

The Mean ◽

Classical Image ◽

Vision Based Localization ◽

Improve Reliability

Vision-based localization is the problem of inferring the pose of the camera given a single image. One commonly used approach relies on image retrieval where the query input is compared against a database of localized support examples and its pose is inferred with the help of the retrieved items. This assumes that images taken from the same places consist of the same landmarks and thus would have similar feature representations. These representations can learn to be robust to different variations in capture conditions like time of the day or weather. In this work, we introduce a framework which aims at enhancing the performance of such retrieval-based localization methods. It consists in taking into account additional information available, such as GPS coordinates or temporal proximity in the acquisition of the images. More precisely, our method consists in constructing a graph based on this additional information that is later used to improve reliability of the retrieval process by filtering the feature representations of support and/or query images. We show that the proposed method is able to significantly improve the localization accuracy on two large scale datasets, as well as the mean average precision in classical image retrieval scenarios.

Download Full-text