scholarly journals Learning Document Embeddings with Crossword Prediction

2019 ◽  
Vol 33 ◽  
pp. 9993-9994
Author(s):  
Junyu Luo ◽  
Min Yang ◽  
Ying Shen ◽  
Qiang Qu ◽  
Haixia Chai
Keyword(s):  
Partial Information ◽  
Document Retrieval ◽  
Document Classification ◽  
Experimental Results ◽  
Context Information ◽  
Decoder Architecture ◽  
Text Reconstruction

In this paper, we propose a Document Embedding Network (DEN) to learn document embeddings in an unsupervised manner. Our model uses the encoder-decoder architecture as its backbone, which tries to reconstruct the input document from an encoded document embedding. Unlike the standard decoder for text reconstruction, we randomly block some words in the input document, and use the incomplete context information and the encoded document embedding to predict the blocked words in the document, inspired by the crossword game. Thus, our decoder can keep the balance between the known and unknown information, and consider both global and partial information when decoding the missing words. We evaluate the learned document embeddings on two tasks: document classification and document retrieval. The experimental results show that our model substantially outperforms the compared methods.1.

I Know What You Need: Investigating Document Retrieval Effectiveness with Partial Session Contexts

2022 ◽  
Vol 40 (3) ◽  
pp. 1-30
Author(s):  
Procheta Sen ◽  
Debasis Ganguly ◽  
Gareth J. F. Jones
Keyword(s):  
Relevant Information ◽  
Document Retrieval ◽  
Context Information ◽  
Information Need ◽  
Search System ◽  
Query Log ◽  
Sequence Modeling ◽  
Joint Embedding ◽  
One Step ◽  
A Current

Reducing user effort in finding relevant information is one of the key objectives of search systems. Existing approaches have been shown to effectively exploit the context from the current search session of users for automatically suggesting queries to reduce their search efforts. However, these approaches do not accomplish the end goal of a search system—that of retrieving a set of potentially relevant documents for the evolving information need during a search session. This article takes the problem of query prediction one step further by investigating the problem of contextual recommendation within a search session. More specifically, given the partial context information of a session in the form of a small number of queries, we investigate how a search system can effectively predict the documents that a user would have been presented with had he continued the search session by submitting subsequent queries. To address the problem, we propose a model of contextual recommendation that seeks to capture the underlying semantics of information need transitions of a current user’s search context. This model leverages information from a number of past interactions of other users with similar interactions from an existing search log. To identify similar interactions, as a novel contribution, we propose an embedding approach that jointly learns representations of both individual query terms and also those of queries (in their entirety) from a search log data by leveraging session-level containment relationships. Our experiments conducted on a large query log, namely the AOL, demonstrate that using a joint embedding of queries and their terms within our proposed framework of document retrieval outperforms a number of text-only and sequence modeling based baselines.

scholarly journals A Method for Constructing Supervised Topic Model Based on Term Frequency-Inverse Topic Frequency

Symmetry ◽  
2019 ◽  
Vol 11 (12) ◽  
pp. 1486
Author(s):  
Zhinan Gou ◽  
Zheng Huo ◽  
Yuanzhen Liu ◽  
Yi Yang
Keyword(s):  
Topic Modeling ◽  
Topic Model ◽  
State Of The Art ◽  
Topic Models ◽  
Document Classification ◽  
Experimental Results ◽  
Tag Recommendation ◽  
Term Frequency ◽  
Series Of Experiments ◽  
Dirichlet Prior

Supervised topic modeling has been successfully applied in the fields of document classification and tag recommendation in recent years. However, most existing models neglect the fact that topic terms have the ability to distinguish topics. In this paper, we propose a term frequency-inverse topic frequency (TF-ITF) method for constructing a supervised topic model, in which the weight of each topic term indicates the ability to distinguish topics. We conduct a series of experiments with not only the symmetric Dirichlet prior parameters but also the asymmetric Dirichlet prior parameters. Experimental results demonstrate that the result of introducing TF-ITF into a supervised topic model outperforms several state-of-the-art supervised topic models.

scholarly journals Document-level Relation Extraction as Semantic Segmentation

2021 ◽  
Author(s):  
Ningyu Zhang ◽  
Xiang Chen ◽  
Xin Xie ◽  
Shumin Deng ◽  
Chuanqi Tan ◽  
...  
Keyword(s):  
Computer Vision ◽  
State Of The Art ◽  
Relation Extraction ◽  
Semantic Segmentation ◽  
Experimental Results ◽  
Context Information ◽  
Global Information ◽  
Benchmark Datasets ◽  
Segmentation Task ◽  
Document Level

Document-level relation extraction aims to extract relations among multiple entity pairs from a document. Previously proposed graph-based or transformer-based models utilize the entities independently, regardless of global information among relational triples. This paper approaches the problem by predicting an entity-level relation matrix to capture local and global information, parallel to the semantic segmentation task in computer vision. Herein, we propose a Document U-shaped Network for document-level relation extraction. Specifically, we leverage an encoder module to capture the context information of entities and a U-shaped segmentation module over the image-style feature map to capture global interdependency among triples. Experimental results show that our approach can obtain state-of-the-art performance on three benchmark datasets DocRED, CDR, and GDA.

scholarly journals Image Completion with Large or Edge-Missing Areas

Algorithms ◽  
2019 ◽  
Vol 13 (1) ◽  
pp. 14
Author(s):  
Jianjian Ji ◽  
Gang Yang
Keyword(s):  
Experimental Results ◽  
The Other ◽  
Context Information ◽  
Network Structures ◽  
Image Completion ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Other Hand ◽  
The One ◽  
Pseudo Color

Existing image completion methods are mostly based on missing regions that are small or located in the middle of the images. When regions to be completed are large or near the edge of the images, due to the lack of context information, the completion results tend to be blurred or distorted, and there will be a large blank area in the final results. In addition, the unstable training of the generative adversarial network is also prone to cause pseudo-color in the completion results. Aiming at the two above-mentioned problems, a method of image completion with large or edge-missing areas is proposed; also, the network structures have been improved. On the one hand, it overcomes the problem of lacking context information, which thereby ensures the reality of generated texture details; on the other hand, it suppresses the generation of pseudo-color, which guarantees the consistency of the whole image both in vision and content. The experimental results show that the proposed method achieves better completion results in completing large or edge-missing areas.

scholarly journals Multitask Learning with Local Attention for Tibetan Speech Recognition

Complexity ◽  
2020 ◽  
Vol 2020 ◽  
pp. 1-10
Author(s):  
Hui Wang ◽  
Fei Gao ◽  
Yue Zhao ◽  
Li Yang ◽  
Jianjian Yue ◽  
...  
Keyword(s):  
Speech Recognition ◽  
Speaker Recognition ◽  
Multitask Learning ◽  
Experimental Results ◽  
Context Information ◽  
Accuracy Rate ◽  
Baseline Model ◽  
Content Recognition ◽  
Tibetan Dialects ◽  
Speech Content

In this paper, we propose to incorporate the local attention in WaveNet-CTC to improve the performance of Tibetan speech recognition in multitask learning. With an increase in task number, such as simultaneous Tibetan speech content recognition, dialect identification, and speaker recognition, the accuracy rate of a single WaveNet-CTC decreases on speech recognition. Inspired by the attention mechanism, we introduce the local attention to automatically tune the weights of feature frames in a window and pay different attention on context information for multitask learning. The experimental results show that our method improves the accuracies of speech recognition for all Tibetan dialects in three-task learning, compared with the baseline model. Furthermore, our method significantly improves the accuracy for low-resource dialect by 5.11% against the specific-dialect model.

scholarly journals Normalizing Item-Based Collaborative Filter Using Context-Aware Scaled Baseline Predictor

2017 ◽  
Vol 2017 ◽  
pp. 1-9 ◽  
Author(s):  
Wenming Ma ◽  
Junfeng Shi ◽  
Ruidong Zhao
Keyword(s):  
Recommendation Systems ◽  
Basic Component ◽  
Experimental Results ◽  
Context Information ◽  
Context Aware ◽  
Collaborative Filter ◽  
Normalization Methods ◽  
Predictor Models ◽  
Baseline Predictor ◽  
Filter Algorithms

Item-based collaborative filter algorithms play an important role in modern commercial recommendation systems (RSs). To improve the recommendation performance, normalization is always used as a basic component for the predictor models. Among a lot of normalizing methods, subtracting the baseline predictor (BLP) is the most popular one. However, the BLP uses a statistical constant without considering the context. We found that slightly scaling the different components of the BLP separately could dramatically improve the performance. This paper proposed some normalization methods based on the scaled baseline predictors according to different context information. The experimental results show that using context-aware scaled baseline predictor for normalization indeed gets better recommendation performance, including RMSE, MAE, precision, recall, and nDCG.

scholarly journals End-to-End Bootstrapping Neural Network for Entity Set Expansion

2020 ◽  
Vol 34 (05) ◽  
pp. 9402-9409
Author(s):  
Lingyong Yan ◽  
Xianpei Han ◽  
Ben He ◽  
Le Sun
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Substantial Improvement ◽  
High Order ◽  
Experimental Results ◽  
Target Category ◽  
Attention Network ◽  
Order Relations ◽  
Decoder Architecture ◽  
End To End

Bootstrapping for entity set expansion (ESE) has long been modeled as a multi-step pipelined process. Such a paradigm, unfortunately, often suffers from two main challenges: 1) the entities are expanded in multiple separate steps, which tends to introduce noisy entities and results in the semantic drift problem; 2) it is hard to exploit the high-order entity-pattern relations for entity set expansion. In this paper, we propose an end-to-end bootstrapping neural network for entity set expansion, named BootstrapNet, which models the bootstrapping in an encoder-decoder architecture. In the encoding stage, a graph attention network is used to capture both the first- and the high-order relations between entities and patterns, and encode useful information into their representations. In the decoding stage, the entities are sequentially expanded through a recurrent neural network, which outputs entities at each stage, and its hidden state vectors, representing the target category, are updated at each expansion step. Experimental results demonstrate substantial improvement of our model over previous ESE approaches.

scholarly journals Clustering Based Classification and Analysis of Data

2014 ◽  
pp. 280-283
Author(s):  
NEERAJ SAHU ◽  
D. S. RAJPUT ◽  
R. S. THAKUR ◽  
G. S. THAKUR
Keyword(s):  
Feature Selection ◽  
Document Classification ◽  
Experimental Results ◽  
Data Sets ◽  
Group Data ◽  
Document Collection ◽  
Text Preprocessing

This paper presents Clustering Based Document classification and analysis of data. The proposed Clustering Based classification and analysis of data approach is based on Unsupervised and Supervised Document Classification. In this paper Unsupervised Document and Supervised Document Classification are used. In this approach Document collection, Text Preprocessing, Feature Selection, Indexing, Clustering Process and Results Analysis steps are used. Twenty News group data sets [20] are used in the Experiments. For experimental results analysis evaluated using the Analytical SAS 9.0 Software is used. The Experimental Results show the proposed approach out performs.

Sign in / Sign up

Export Citation Format

Share Document