Voice Keyword Retrieval Method Using Attention Mechanism and Multimodal Information Fusion

A cross-modal speech-text retrieval method using interactive learning convolution automatic encoder (CAE) is proposed. First, an interactive learning autoencoder structure is proposed, including two inputs of speech and text, as well as processing links such as encoding, hidden layer interaction, and decoding, to complete the modeling of cross-modal speech-text retrieval. Then, the original audio signal is preprocessed and the Mel frequency cepstrum coefficient (MFCC) feature is extracted. In addition, the word bag model is used to extract the text features, and then the attention mechanism is used to combine the text and speech features. Through interactive learning CAE, the shared features of speech and text modes are obtained and then sent to modal classifier to identify modal information, so as to realize cross-modal voice text retrieval. Finally, experiments show that the performance of the proposed algorithm is better than that of the contrast algorithm in terms of recall rate, accuracy rate, and false recognition rate.

Download Full-text

A Cross-Modal Image and Text Retrieval Method Based on Efficient Feature Extraction and Interactive Learning CAE

Scientific Programming ◽

10.1155/2022/7314599 ◽

2022 ◽

Vol 2022 ◽

pp. 1-12

Author(s):

Xiuye Yin ◽

Liyong Chen

Keyword(s):

Feature Extraction ◽

Short Term Memory ◽

Interactive Learning ◽

Principal Component ◽

Middle Layer ◽

Recall Rate ◽

Text Retrieval ◽

Image Features ◽

Retrieval Method ◽

Text Features

In view of the complexity of the multimodal environment and the existing shallow network structure that cannot achieve high-precision image and text retrieval, a cross-modal image and text retrieval method combining efficient feature extraction and interactive learning convolutional autoencoder (CAE) is proposed. First, the residual network convolution kernel is improved by incorporating two-dimensional principal component analysis (2DPCA) to extract image features and extracting text features through long short-term memory (LSTM) and word vectors to efficiently extract graphic features. Then, based on interactive learning CAE, cross-modal retrieval of images and text is realized. Among them, the image and text features are respectively input to the two input terminals of the dual-modal CAE, and the image-text relationship model is obtained through the interactive learning of the middle layer to realize the image-text retrieval. Finally, based on Flickr30K, MSCOCO, and Pascal VOC 2007 datasets, the proposed method is experimentally demonstrated. The results show that the proposed method can complete accurate image retrieval and text retrieval. Moreover, the mean average precision (MAP) has reached more than 0.3, the area of precision-recall rate (PR) curves are better than other comparison methods, and they are applicable.

Download Full-text

Information Technology and an Audio Retrieval Method Based on a Novel Audience Rating System

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.886.664 ◽

2014 ◽

Vol 886 ◽

pp. 664-667

Author(s):

Lan Tian ◽

Qing Hua Song ◽

Xiao Shan Lu

Keyword(s):

Recognition Rate ◽

Audio Signal ◽

Rating System ◽

Test Results ◽

Simulation Test ◽

Retrieval Method ◽

Vector Correlation ◽

Audio Features ◽

Audio Retrieval ◽

Audience Rating

A novel and universal audience rating system based on TV audio features is introduced. In this system, the audio signal is sampled from the audio outlet of TV set and high compressed into a specific spectrum features packages. The packaged audio features are high robustness for different types of television sets and sound volumes, and have no disturbance of surroundings. In the channel audio retrieval method, feature vector correlation analysis and pattern matching are adopted successively. The simulation test results show that the TV channel recognition rate is above 93%, and other sounds from non-TV channels can be detected reliably.

Download Full-text

Non-Intrusive Load Disaggregation Based on a Multi-Scale Attention Residual Network

Applied Sciences ◽

10.3390/app10249132 ◽

2020 ◽

Vol 10 (24) ◽

pp. 9132

Author(s):

Liguo Weng ◽

Xiaodong Zhang ◽

Junhao Qian ◽

Min Xia ◽

Yiqing Xu ◽

...

Keyword(s):

Smart Grids ◽

Recognition Rate ◽

Low Frequency ◽

Attention Mechanism ◽

Learning Ability ◽

Residual Network ◽

Multi Scale ◽

Energy Disaggregation ◽

Benchmark Datasets ◽

Load Disaggregation

Non-intrusive load disaggregation (NILD) is of great significance to the development of smart grids. Current energy disaggregation methods extract features from sequences, and this process easily leads to a loss of load features and difficulties in detecting, resulting in a low recognition rate of low-use electrical appliances. To solve this problem, a non-intrusive sequential energy disaggregation method based on a multi-scale attention residual network is proposed. Multi-scale convolutions are used to learn features, and the attention mechanism is used to enhance the learning ability of load features. The residual learning further improves the performance of the algorithm, avoids network degradation, and improves the precision of load decomposition. The experimental results on two benchmark datasets show that the proposed algorithm has more advantages than the existing algorithms in terms of load disaggregation accuracy and judgments of the on/off state, and the attention mechanism can further improve the disaggregation accuracy of low-frequency electrical appliances.

Download Full-text

Multi-keyword Cipher-text Retrieval Method for Smart Grid Edge Computing*

Journal of Physics Conference Series ◽

10.1088/1742-6596/1754/1/012076 ◽

2021 ◽

Vol 1754 (1) ◽

pp. 012076

Author(s):

Jing Zhu ◽

Tao Wu ◽

Jintao Li ◽

Yanbin Liu ◽

Qixin Jiang

Keyword(s):

Smart Grid ◽

Text Retrieval ◽

Edge Computing ◽

Retrieval Method ◽

Cipher Text

Download Full-text

Predicting Outcomes of Business Process Executions Based on LSTM Neural Networks and Attention Mechanism

10.21203/rs.3.rs-260970/v1 ◽

2021 ◽

Author(s):

Jiaojiao Wang ◽

Dongjin Yu ◽

Chengfei Liu ◽

Xiaoxiao Sun

Keyword(s):

Short Term Memory ◽

Attention Mechanism ◽

Sequential Data ◽

Time Prediction ◽

Prediction Time ◽

Highly Sensitive ◽

Long Short Term Memory ◽

Early Decision ◽

Lstm Network ◽

Hidden Layer

Abstract To effectively predict the outcome of an on-going process instance helps make an early decision, which plays an important role in so-called predictive process monitoring. Existing methods in this field are tailor-made for some empirical operations such as the prefix extraction, clustering, and encoding, leading that their relative accuracy is highly sensitive to the dataset. Moreover, they have limitations in real-time prediction applications due to the lengthy prediction time. Since Long Short-term Memory (LSTM) neural network provides a high precision in the prediction of sequential data in several areas, this paper investigates LSTM and its enhancements and proposes three different approaches to build more effective and efficient models for outcome prediction. The first move on enhancement is that we combine the original LSTM network from two directions, forward and backward, to capture more features from the completed cases. The second move on enhancement is that we add attention mechanism after extracting features in the hidden layer of LSTM network to distinct them from their attention weight. A series of extensive experiments are evaluated on twelve real datasets when comparing with other approaches. The results show that our approaches outperform the state-of-the-art ones in terms of prediction effectiveness and time performance.

Download Full-text

Text-Image Retrieval With Salient Features

Journal of Database Management ◽

10.4018/jdm.2021100101 ◽

2021 ◽

Vol 32 (4) ◽

pp. 1-13

Author(s):

Xia Feng ◽

Zhiyi Hu ◽

Caihua Liu ◽

W. H. Ip ◽

Huiying Chen

Keyword(s):

Image Retrieval ◽

Recall Rate ◽

Image Features ◽

Feature Representation ◽

Image Feature ◽

Text And Image ◽

Retrieval Task ◽

Retrieval Method ◽

Salient Features ◽

Object Level

In recent years, deep learning has achieved remarkable results in the text-image retrieval task. However, only global image features are considered, and the vital local information is ignored. This results in a failure to match the text well. Considering that object-level image features can help the matching between text and image, this article proposes a text-image retrieval method that fuses salient image feature representation. Fusion of salient features at the object level can improve the understanding of image semantics and thus improve the performance of text-image retrieval. The experimental results show that the method proposed in the paper is comparable to the latest methods, and the recall rate of some retrieval results is better than the current work.

Download Full-text

Method for Retrieving Digital Agricultural Text Information Based on Local Matching

Symmetry ◽

10.3390/sym12071103 ◽

2020 ◽

Vol 12 (7) ◽

pp. 1103

Author(s):

Yue Song ◽

Minjuan Wang ◽

Wanlin Gao

Keyword(s):

Recall Rate ◽

Retrieval Method ◽

Retrieval Time ◽

Search Results ◽

Retrieval Efficiency ◽

Query Tree ◽

Text Information ◽

Relationship Of ◽

The Relationship ◽

Precision Rate

In order to improve the retrieval results of digital agricultural text information and improve the efficiency of retrieval, the method for searching digital agricultural text information based on local matching is proposed. The agricultural text tree and the query tree are constructed to generate the relationship of ancestor–descendant in the query and map it to the agricultural text. According to the retrieval method of the local matching, the vector retrieval method is used to calculate the digital agricultural text and submit the similarity between the queries. The similarity is sorted from large to small so that the agricultural text tree can output digital agricultural text information in turn. In the case of adding interference information, the recall rate and precision rate of the proposed method are above 99.5%; the average retrieval time is between 4s and 6s, and the average retrieval efficiency is above 99%. The proposed method is more efficient in information retrieval and can obtain comprehensive and accurate search results, which can be used for the rapid retrieval of digital agricultural text information.

Download Full-text

Emotion-Semantic-Enhanced Bidirectional LSTM with Multi-Head Attention Mechanism for Microblog Sentiment Analysis

Information ◽

10.3390/info11050280 ◽

2020 ◽

Vol 11 (5) ◽

pp. 280

Author(s):

Shaoxiu Wang ◽

Yonghua Zhu ◽

Wenjing Gao ◽

Meng Cao ◽

Mengyao Li

Keyword(s):

Sentiment Analysis ◽

Short Term Memory ◽

Syntactic Structure ◽

Contextual Information ◽

Research Field ◽

Attention Mechanism ◽

Feature Representation ◽

Mechanism Model ◽

Hidden Layer ◽

The Impact

The sentiment analysis of microblog text has always been a challenging research field due to the limited and complex contextual information. However, most of the existing sentiment analysis methods for microblogs focus on classifying the polarity of emotional keywords while ignoring the transition or progressive impact of words in different positions in the Chinese syntactic structure on global sentiment, as well as the utilization of emojis. To this end, we propose the emotion-semantic-enhanced bidirectional long short-term memory (BiLSTM) network with the multi-head attention mechanism model (EBILSTM-MH) for sentiment analysis. This model uses BiLSTM to learn feature representation of input texts, given the word embedding. Subsequently, the attention mechanism is used to assign the attentive weights of each words to the sentiment analysis based on the impact of emojis. The attentive weights can be combined with the output of the hidden layer to obtain the feature representation of posts. Finally, the sentiment polarity of microblog can be obtained through the dense connection layer. The experimental results show the feasibility of our proposed model on microblog sentiment analysis when compared with other baseline models.

Download Full-text

A Method of Extracting Feature of Phonic Signal Based on the Matrix Analysis

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.457-458.1200 ◽

2013 ◽

Vol 457-458 ◽

pp. 1200-1203

Author(s):

Yang Xu ◽

Fang Chao Hu

Keyword(s):

Embedded System ◽

Feature Vector ◽

Recognition Rate ◽

Audio Signal ◽

Coefficient Matrix ◽

Matrix Analysis ◽

Discrete Wavelet ◽

Feature Vectors ◽

The Matrix ◽

The Embedded System

In the speech recognition technology, feature extraction is essential for the system recognition rate, taking amount of strategies to find the better feature vectors are most researchers target. This paper presents a method of extracting feature of audio signal based on the discrete wavelet transform, then decomposed the coefficient matrix by the matrix analysis way, through this method to find a new thinking on the way of extracting feature vector. The method can be achieved in the procedure. The main purpose is to reduce the dimension of feature vector, make the vector briefer, and then reduce the computing complexity in the embedded system. This method can reduce the feature vectors dimension, accelerated the computing velocity.

Download Full-text