Efficient Multi-modal Hashing with Online Query Adaption for Multimedia Retrieval

Multi-modal hashing supports efficient multimedia retrieval well. However, existing methods still suffer from two problems: (1) Fixed multi-modal fusion. They collaborate the multi-modal features with fixed weights for hash learning, which cannot adaptively capture the variations of online streaming multimedia contents. (2) Binary optimization challenge. To generate binary hash codes, existing methods adopt either two-step relaxed optimization that causes significant quantization errors or direct discrete optimization that consumes considerable computation and storage cost. To address these problems, we first propose a Supervised Multi-modal Hashing with Online Query-adaption method. A self-weighted fusion strategy is designed to adaptively preserve the multi-modal features into hash codes by exploiting their complementarity. Besides, the hash codes are efficiently learned with the supervision of pair-wise semantic labels to enhance their discriminative capability while avoiding the challenging symmetric similarity matrix factorization. Further, we propose an efficient Unsupervised Multi-modal Hashing with Online Query-adaption method with an adaptive multi-modal quantization strategy. The hash codes are directly learned without the reliance on the specific objective formulations. Finally, in both methods, we design a parameter-free online hashing module to adaptively capture query variations at the online retrieval stage. Experiments validate the superiority of our proposed methods.

Research of collision properties of the modified UMAC algorithm on crypto-code constructions

The transfer of information by telecommunication channels is accompanied by message hashing to control the integrity of the data and confirm the authenticity of the data. When using a reliable hash function, it is computationally difficult to create a fake message with a pre-existing hash code, however, due to the weaknesses of specific hashing algorithms, this threat can be feasible. To increase the level of cryptographic strength of transmitted messages over telecommunication channels, there are ways to create hash codes, which, according to practical research, are imperfect in terms of the speed of their formation and the degree of cryptographic strength. The collisional properties of hashing functions formed using the modified UMAC algorithm using the methodology for assessing the universality and strict universality of hash codes are investigated. Based on the results of the research, an assessment of the impact of the proposed modifications at the last stage of the generation of authentication codes on the provision of universal hashing properties was presented. The analysis of the advantages and disadvantages that accompany the formation of the hash code by the previously known methods is carried out. The scheme of cascading generation of data integrity and authenticity control codes using the UMAC algorithm on crypto-code constructions has been improved. Schemes of algorithms for checking hash codes were developed to meet the requirements of universality and strict universality. The calculation and analysis of collision search in the set of generated hash codes was carried out according to the requirements of a universal and strictly universal class for creating hash codes

Blockchain Technology

Blockchain is a distributed decentralized peer-to-peer network aiming to facilitate the immutability and security of data. Towards the service orientation, blockchain is a collection of distributed blocks having unique hash codes without any point of failure. Each block is stored on distributed ledgers, and transactions with them are secure, transparent, immutable, and traceable. To create a new block and allow a transaction to complete, an agreement between all parties is required. To reach an agreement in a blockchain network, consensus algorithms are used. In this chapter, fundamental principles and algorithms of blockchain networks have been discussed, and a detailed review of the blockchain consensus algorithms PoW, PoS, DPoS, PoET, PoWeight, PoB, PoA, and PoC have been provided including the merits and demerits of consensus algorithms with analysis to provide a deep understanding of the current research trends and future challenges.

Convolutional Neural Network Copy Detection with Neural Network Perceptual Hashing

In recent years, many model intellectual property (IP) proof methods for IP protection have been proposed, such as model watermarking and model fingerprinting. However, as an important part of the model IP protection system, the model copy detection task has not received enough attention. With the increasing number of neural network models transmitted and deployed on the Internet, the search for similar models is in great demand, which concurrently triggers the security problem of copy detection of models for IP protection. Due to the high computational complexity, both model watermarking and model fingerprinting lack the capability to efficiently find suspected infringing models among tens of millions of models. In this paper, inspired by the hash-based image retrieval methods, we propose a perceptual hashing algorithm for convolutional neural networks (CNNs). The proposed perceptual hashing algorithm can convert the weights of CNNs to fixed-length binary hash codes so that the lightly modified version has the similar hash code as the original model. By comparing the similarity of a pair of hash codes between a query model and a test model in the model library, the similar versions of a query model can be retrieved efficiently. To the best of our knowledge, this is the first perceptual hashing algorithm for CNNs. The experiment performed on a model library containing 3,565 models indicates that our proposed perceptual hashing scheme has a superior copy detection performance.

Convolutional Neural Network Copy Detection with Neural Network Perceptual Hashing

In recent years, many model intellectual property (IP) proof methods for IP protection have been proposed, such as model watermarking and model fingerprinting. However, as an important part of the model IP protection system, the model copy detection task has not received enough attention. With the increasing number of neural network models transmitted and deployed on the Internet, the search for similar models is in great demand, which concurrently triggers the security problem of copy detection of models for IP protection. Due to the high computational complexity, both model watermarking and model fingerprinting lack the capability to efficiently find suspected infringing models among tens of millions of models. In this paper, inspired by the hash-based image retrieval methods, we propose a perceptual hashing algorithm for convolutional neural networks (CNNs). The proposed perceptual hashing algorithm can convert the weights of CNNs to fixed-length binary hash codes so that the lightly modified version has the similar hash code as the original model. By comparing the similarity of a pair of hash codes between a query model and a test model in the model library, the similar versions of a query model can be retrieved efficiently. To the best of our knowledge, this is the first perceptual hashing algorithm for CNNs. The experiment performed on a model library containing 3,565 models indicates that our proposed perceptual hashing scheme has a superior copy detection performance.

Deep Learning to Ternary Hash Codes by Continuation

Recently, it has been observed that $\{0,\pm1\}$-ternary codes which are simply generated from deep features by hard thresholding, tend to outperform $\{-1, 1\}$-binary codes in image retrieval. To obtain better ternary codes, we for the first time propose to jointly learn the features with the codes by appending a smoothed function to the networks. During training, the function could evolve into a non-smoothed ternary function by a continuation method, and then generate ternary codes. The method circumvents the difficulty of directly training discrete functions and reduces the quantization errors of ternary codes. Experiments show that the proposed joint learning indeed could produce better ternary codes.

Deep Learning to Ternary Hash Codes by Continuation

Recently, it has been observed that $\{0,\pm1\}$-ternary codes which are simply generated from deep features by hard thresholding, tend to outperform $\{-1, 1\}$-binary codes in image retrieval. To obtain better ternary codes, we for the first time propose to jointly learn the features with the codes by appending a smoothed function to the networks. During training, the function could evolve into a non-smoothed ternary function by a continuation method, and then generate ternary codes. The method circumvents the difficulty of directly training discrete functions and reduces the quantization errors of ternary codes. Experiments show that the proposed joint learning indeed could produce better ternary codes.

TDCMR: Triplet-Based Deep Cross-Modal Retrieval for Geo-Multimedia Data

Mass multimedia data with geographical information (geo-multimedia) are collected and stored on the Internet due to the wide application of location-based services (LBS). How to find the high-level semantic relationship between geo-multimedia data and construct efficient index is crucial for large-scale geo-multimedia retrieval. To combat this challenge, the paper proposes a deep cross-modal hashing framework for geo-multimedia retrieval, termed as Triplet-based Deep Cross-Modal Retrieval (TDCMR), which utilizes deep neural network and an enhanced triplet constraint to capture high-level semantics. Besides, a novel hybrid index, called TH-Quadtree, is developed by combining cross-modal binary hash codes and quadtree to support high-performance search. Extensive experiments are conducted on three common used benchmarks, and the results show the superior performance of the proposed method.

Deep supervised hashing for gait retrieval

Background: Gait recognition is perceived as the most promising biometric approach for future decades especially because of its efficient applicability in surveillance systems. Due to recent growth in the use of gait biometrics across surveillance systems, the ability to rapidly search for the required data has become an emerging need. Therefore, we addressed the gait retrieval problem, which retrieves people with gaits similar to a query subject from a large-scale dataset. Methods: This paper presents the deep gait retrieval hashing (DGRH) model to address the gait retrieval problem for large-scale datasets. Our proposed method is based on a supervised hashing method with a deep convolutional network. We use the ability of the convolutional neural network (CNN) to capture the semantic gait features for feature representation and learn the compact hash codes with the compatible hash function. Therefore, our DGRH model combines gait feature learning with binary hash codes. In addition, the learning loss is designed with a classification loss function that learns to preserve similarity and a quantization loss function that controls the quality of the hash codes Results: The proposed method was evaluated against the CASIA-B, OUISIR-LP, and OUISIR-MVLP benchmark datasets and received the promising result for gait retrieval tasks. Conclusions: The end-to-end deep supervised hashing model is able to learn discriminative gait features and is efficient in terms of the storage memory and speed for gait retrieval.

Scalable supervised online hashing for image retrieval

Online hashing methods aim to learn compact binary codes of the new data stream, and update the hash function to renew the codes of the existing data. However, the addition of new data streams has a vital impact on the retrieval performance of the entire retrieval system, especially the similarity measurement between new data streams and existing data, which has always been one of the focuses of online retrieval research. In this paper, we present a novel scalable supervised online hashing method, to solve the above problems within a unified framework. Specifically, the similarity matrix is established by the label matrix of the existing data and the new data stream. The projection of the existing data label matrix is then used as an intermediate term to approximate the binary codes of the existing data, which not only realizes the semantic information of the hash codes learning but also effectively alleviates the problem of data imbalance. In addition, an alternate optimization algorithm is proposed to efficiently make the solution of the model. Extensive experiments on three widely used datasets validate its superior performance over several state-of-the-art methods in terms of both accuracy and scalability for online retrieval task.