Hybrid Deep Neural Network-Based Cross-Modal Image and Text Retrieval Method for Large-Scale Data

2020 ◽  
pp. 2150018
Author(s):  
Baohua Qiang ◽  
Ruidong Chen ◽  
Yuan Xie ◽  
Mingliang Zhou ◽  
Riwei Pan ◽  
...  
Keyword(s):  
Neural Network ◽  
Large Scale ◽  
Deep Neural Network ◽  
Text Retrieval ◽  
Retrieval Method ◽  
Multimodal Data ◽  
Hierarchical Learning ◽  
Fine Grained ◽  
Large Scale Data ◽  
Scale Data

In this paper, we propose the hybrid deep neural network-based cross-modal image and text retrieval method to explore complex cross-modal correlation by considering multi-layer learning. First, we propose intra-modal and inter-modal representations to achieve a complementary single-modal representation that preserves the correlation between the modalities. Second, we build an association between different modalities through hierarchical learning to further mine the fine-grained latent semantic association among multimodal data. The experimental results show that our algorithm substantially enhances retrieval performance and consistently outperforms four comparison methods.

scholarly journals Multimodal Data Acquisition at SARS-CoV-2 Drive Through Screening Centers: Setup Description and Experiences in Saarland, Germany

2020 ◽  
Author(s):  
Philipp Flotho ◽  
Mayur J. Bhamborae ◽  
Tobias Grün ◽  
Carlos Trenado ◽  
David Thinnes ◽  
...  
Keyword(s):  
Data Acquisition ◽  
Large Scale ◽  
Screening Methods ◽  
Infection Rates ◽  
Multimodal Data ◽  
Large Scale Data ◽  
High Infection ◽  
Speed Up ◽  
Specific Symptoms ◽  
Scale Data

AbstractSARS-CoV-2 drive through screening centers (DTSC) have been implemented worldwide as a fast and secure way of mass screening. We use DTSCs as a platform for the acquisition of multimodal datasets that are needed for the development of remote screening methods. Our acquisition setup consists of an array of thermal, infrared and RGB cameras as well as microphones and we apply methods from computer vision and computer audition for the contactless estimation of physiological parameters. We have recorded a multimodal dataset of DTSC participants in Germany for the development of remote screening methods and symptom identification. Acquisition in the early stages of a pandemic and in regions with high infection rates can facilitate and speed up the identification of infection specific symptoms and large scale data acquisition at DTSC is possible without disturbing the flow of operation.

scholarly journals High-Performance Machine Learning for Large-Scale Data Classification considering Class Imbalance

2020 ◽  
Vol 2020 ◽  
pp. 1-16
Author(s):  
Yang Liu ◽  
Xiang Li ◽  
Xianbang Chen ◽  
Xi Wang ◽  
Huaqiang Li
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
High Performance ◽  
Large Scale ◽  
Class Imbalance ◽  
Data Classification ◽  
Training Dataset ◽  
Large Scale Data ◽  
Input Layer ◽  
Scale Data

Currently, data classification is one of the most important ways to analysis data. However, along with the development of data collection, transmission, and storage technologies, the scale of the data has been sharply increased. Additionally, due to multiple classes and imbalanced data distribution in the dataset, the class imbalance issue is also gradually highlighted. The traditional machine learning algorithms lack of abilities for handling the aforementioned issues so that the classification efficiency and precision may be significantly impacted. Therefore, this paper presents an improved artificial neural network in enabling the high-performance classification for the imbalanced large volume data. Firstly, the Borderline-SMOTE (synthetic minority oversampling technique) algorithm is employed to balance the training dataset, which potentially aims at improving the training of the back propagation neural network (BPNN), and then, zero-mean, batch-normalization, and rectified linear unit (ReLU) are further employed to optimize the input layer and hidden layers of BPNN. At last, the ensemble learning-based parallelization of the improved BPNN is implemented using the Hadoop framework. Positive conclusions can be summarized according to the experimental results. Benefitting from Borderline-SMOTE, the imbalanced training dataset can be balanced, which improves the training performance and the classification accuracy. The improvements for the input layer and hidden layer also enhance the training performances in terms of convergence. The parallelization and the ensemble learning techniques enable BPNN to implement the high-performance large-scale data classification. The experimental results show the effectiveness of the presented classification algorithm.

Sign in / Sign up

Export Citation Format

Share Document