scholarly journals A Domain Generalization Perspective on Listwise Context Modeling

2019 ◽  
Vol 33 ◽  
pp. 5965-5972 ◽  
Author(s):  
Lin Zhu ◽  
Yihong Chen ◽  
Bowen He
Keyword(s):  
Data Mining ◽  
Information Retrieval ◽  
State Of The Art ◽  
Learning To Rank ◽  
Context Modeling ◽  
Ranking Problem ◽  
Neural Architecture ◽  
Benchmark Datasets ◽  
Previous State ◽  
Latent Representations

As one of the most popular techniques for solving the ranking problem in information retrieval, Learning-to-rank (LETOR) has received a lot of attention both in academia and industry due to its importance in a wide variety of data mining applications. However, most of existing LETOR approaches choose to learn a single global ranking function to handle all queries, and ignore the substantial differences that exist between queries. In this paper, we propose a domain generalization strategy to tackle this problem. We propose QueryInvariant Listwise Context Modeling (QILCM), a novel neural architecture which eliminates the detrimental influence of inter-query variability by learning query-invariant latent representations, such that the ranking system could generalize better to unseen queries. We evaluate our techniques on benchmark datasets, demonstrating that QILCM outperforms previous state-of-the-art approaches by a substantial margin.

scholarly journals Two-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentation

2020 ◽  
Vol 34 (05) ◽  
pp. 7797-7804
Author(s):  
Goran Glavašš ◽  
Swapna Somasundaran
Keyword(s):  
State Of The Art ◽  
Language Transfer ◽  
Text Segmentation ◽  
Word Embeddings ◽  
Neural Architecture ◽  
Text Coherence ◽  
Sentence Level ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
Cross Lingual

Breaking down the structure of long texts into semantically coherent segments makes the texts more readable and supports downstream applications like summarization and retrieval. Starting from an apparent link between text coherence and segmentation, we introduce a novel supervised model for text segmentation with simple but explicit coherence modeling. Our model – a neural architecture consisting of two hierarchically connected Transformer networks – is a multi-task learning model that couples the sentence-level segmentation objective with the coherence objective that differentiates correct sequences of sentences from corrupt ones. The proposed model, dubbed Coherence-Aware Text Segmentation (CATS), yields state-of-the-art segmentation performance on a collection of benchmark datasets. Furthermore, by coupling CATS with cross-lingual word embeddings, we demonstrate its effectiveness in zero-shot language transfer: it can successfully segment texts in languages unseen in training.

scholarly journals An Experimental Analysis of Model Compression Techniques for Object Detection

2020 ◽  
Author(s):  
Andrey De Aguiar Salvi ◽  
Rodrigo Coelho Barros
Keyword(s):  
Object Detection ◽  
Experimental Analysis ◽  
State Of The Art ◽  
Neural Architecture ◽  
Model Compression ◽  
Processing Power ◽  
Benchmark Datasets ◽  
The Difference ◽  
And Performance ◽  
Consumption Constraints

Recent research on Convolutional Neural Networks focuses on how to create models with a reduced number of parameters and a smaller storage size while keeping the model’s ability to perform its task, allowing the use of the best CNN for automating tasks in limited devices, with reduced processing power, memory, or energy consumption constraints. There are many different approaches in the literature: removing parameters, reduction of the floating-point precision, creating smaller models that mimic larger models, neural architecture search (NAS), etc. With all those possibilities, it is challenging to say which approach provides a better trade-off between model reduction and performance, due to the difference between the approaches, their respective models, the benchmark datasets, or variations in training details. Therefore, this article contributes to the literature by comparing three state-of-the-art model compression approaches to reduce a well-known convolutional approach for object detection, namely YOLOv3. Our experimental analysis shows that it is possible to create a reduced version of YOLOv3 with 90% fewer parameters and still outperform the original model by pruning parameters. We also create models that require only 0.43% of the original model’s inference effort.

Automatic Keyword Extraction From Text Documents

2021 ◽  
pp. 71-91
Author(s):  
Furkan Goz ◽  
Alev Mutlu
Keyword(s):  
Information Retrieval ◽  
State Of The Art ◽  
Online News ◽  
Evaluation Metrics ◽  
Keyword Extraction ◽  
Feature Engineering ◽  
Extraction Techniques ◽  
Text Documents ◽  
Scientific Papers ◽  
Benchmark Datasets

Keyword indexing is the problem of assigning keywords to text documents. It is an important task as keywords play crucial roles in several information retrieval tasks. The problem is also challenging as the number of text documents is increasing, and such documents come in different forms (i.e., scientific papers, online news articles, and microblog posts). This chapter provides an overview of keyword indexing and elaborates on keyword extraction techniques. The authors provide the general motivations behind the supervised and the unsupervised keyword extraction and enumerate several pioneering and state-of-the-art techniques. Feature engineering, evaluation metrics, and benchmark datasets used to evaluate the performance of keyword extraction systems are also discussed.

scholarly journals Interpretable Retrosynthesis Prediction in Two Steps

2020 ◽  
Author(s):  
Chaochao Yan ◽  
Qianggang Ding ◽  
Shuangjia Zheng ◽  
Jinyu Yang ◽  
Yang Yu ◽  
...  
Keyword(s):  
State Of The Art ◽  
Building Blocks ◽  
Computer Aided ◽  
Free Model ◽  
Benchmark Datasets ◽  
Previous State ◽  
Target Molecules ◽  
Template Free ◽  
Synthetic Routes ◽  
High Diversity

Retrosynthesis is the process of recursively decomposing target molecules into available building blocks. Computer-aided retrosynthesis can potentially assist chemists in designing synthetic routes; however, at present, it is cumbersome and lacks interpretability behind their predictions. In this study, we devise a novel template-free model for retrosynthetic expansion by automating the procedure that chemistsusedtodo. Our method plans synthesis in two steps, by first identifying the potential disconnection bonds of the molecule graph with a graph neural network and thereafter generating synthons according to the identified disconnection bonds of the target molecule graph, and then predicting the associated reactants SMILES based on the obtained synthons with a reactant prediction model. While outperforming previous state-of-the-art baselines by a significant margin on the benchmark datasets, our model also provides predictions with high diversity and chemically reasonable interpretation.

scholarly journals Interpretable Retrosynthesis Prediction in Two Steps

2020 ◽  
Author(s):  
Chaochao Yan ◽  
Qianggang Ding ◽  
Peilin Zhao ◽  
Shuangjia Zheng ◽  
Jinyu Yang ◽  
...  
Keyword(s):  
State Of The Art ◽  
Building Blocks ◽  
Computer Aided ◽  
Free Model ◽  
Benchmark Datasets ◽  
Previous State ◽  
Target Molecules ◽  
Template Free ◽  
Synthetic Routes ◽  
High Diversity

Retrosynthesis is the process of recursively decomposing target molecules into available building blocks. Computer-aided retrosynthesis can potentially assist chemists in designing synthetic routes; however, at present, it is cumbersome and lacks interpretability behind their predictions. In this study, we devise a novel template-free model for retrosynthetic expansion by automating the procedure that chemistsusedtodo. Our method plans synthesis in two steps, by first identifying the potential disconnection bonds of the molecule graph with a graph neural network and thereafter generating synthons according to the identified disconnection bonds of the target molecule graph, and then predicting the associated reactants SMILES based on the obtained synthons with a reactant prediction model. While outperforming previous state-of-the-art baselines by a significant margin on the benchmark datasets, our model also provides predictions with high diversity and chemically reasonable interpretation.

scholarly journals A transformer-based approach to irony and sarcasm detection

2020 ◽  
Vol 32 (23) ◽  
pp. 17309-17320
Author(s):  
Rolandos Alexandros Potamias ◽  
Georgios Siolas ◽  
Andreas - Georgios Stafylopatis
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Network Architecture ◽  
Figurative Language ◽  
State Of The Art ◽  
Unresolved Issue ◽  
Discussion Forums ◽  
Large Margin ◽  
Neural Architecture ◽  
Benchmark Datasets

AbstractFigurative language (FL) seems ubiquitous in all social media discussion forums and chats, posing extra challenges to sentiment analysis endeavors. Identification of FL schemas in short texts remains largely an unresolved issue in the broader field of natural language processing, mainly due to their contradictory and metaphorical meaning content. The main FL expression forms are sarcasm, irony and metaphor. In the present paper, we employ advanced deep learning methodologies to tackle the problem of identifying the aforementioned FL forms. Significantly extending our previous work (Potamias et al., in: International conference on engineering applications of neural networks, Springer, Berlin, pp 164–175, 2019), we propose a neural network methodology that builds on a recently proposed pre-trained transformer-based network architecture which is further enhanced with the employment and devise of a recurrent convolutional neural network. With this setup, data preprocessing is kept in minimum. The performance of the devised hybrid neural architecture is tested on four benchmark datasets, and contrasted with other relevant state-of-the-art methodologies and systems. Results demonstrate that the proposed methodology achieves state-of-the-art performance under all benchmark datasets, outperforming, even by a large margin, all other methodologies and published studies.

scholarly journals DeepTileBars: Visualizing Term Distribution for Neural Information Retrieval

2019 ◽  
Vol 33 ◽  
pp. 289-296 ◽  
Author(s):  
Zhiwen Tang ◽  
Grace Hui Yang
Keyword(s):  
Information Retrieval ◽  
State Of The Art ◽  
Discourse Structure ◽  
Interaction Matrix ◽  
Light Weight ◽  
Document Ranking ◽  
Design And Implementation ◽  
Neural Information ◽  
Benchmark Datasets ◽  
Topic Hierarchy

Most neural Information Retrieval (Neu-IR) models derive query-to-document ranking scores based on term-level matching. Inspired by TileBars, a classical term distribution visualization method, in this paper, we propose a novel Neu-IR model that handles query-to-document matching at the subtopic and higher levels. Our system first splits the documents into topical segments, “visualizes” the matchings between the query and the segments, and then feeds an interaction matrix into a Neu-IR model, DeepTileBars, to obtain the final ranking scores. DeepTileBars models the relevance signals occurring at different granularities in a document’s topic hierarchy. It better captures the discourse structure of a document and thus the matching patterns. Although its design and implementation are light-weight, DeepTileBars outperforms other state-of-the-art Neu-IR models on benchmark datasets including the Text REtrieval Conference (TREC) 2010-2012 Web Tracks and LETOR 4.0.

scholarly journals TextScanner: Reading Characters in Order for Robust Scene Text Recognition

2020 ◽  
Vol 34 (07) ◽  
pp. 12120-12127 ◽  
Author(s):  
Zhaoyi Wan ◽  
Minghang He ◽  
Haoran Chen ◽  
Xiang Bai ◽  
Cong Yao
Keyword(s):  
State Of The Art ◽  
Semantic Segmentation ◽  
Text Recognition ◽  
Context Modeling ◽  
Alternative Approach ◽  
Scene Text ◽  
Benchmark Datasets ◽  
Character Position ◽  
Scene Text Recognition ◽  
Character Class

Driven by deep learning and a large volume of data, scene text recognition has evolved rapidly in recent years. Formerly, RNN-attention-based methods have dominated this field, but suffer from the problem of attention drift in certain situations. Lately, semantic segmentation based algorithms have proven effective at recognizing text of different forms (horizontal, oriented and curved). However, these methods may produce spurious characters or miss genuine characters, as they rely heavily on a thresholding procedure operated on segmentation maps. To tackle these challenges, we propose in this paper an alternative approach, called TextScanner, for scene text recognition. TextScanner bears three characteristics: (1) Basically, it belongs to the semantic segmentation family, as it generates pixel-wise, multi-channel segmentation maps for character class, position and order; (2) Meanwhile, akin to RNN-attention-based methods, it also adopts RNN for context modeling; (3) Moreover, it performs paralleled prediction for character position and class, and ensures that characters are transcripted in the correct order. The experiments on standard benchmark datasets demonstrate that TextScanner outperforms the state-of-the-art methods. Moreover, TextScanner shows its superiority in recognizing more difficult text such as Chinese transcripts and aligning with target characters.

scholarly journals Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness

2021 ◽  
Author(s):  
Dazhong Shen ◽  
Chuan Qin ◽  
Chao Wang ◽  
Hengshu Zhu ◽  
Enhong Chen ◽  
...  
Keyword(s):  
Latent Variables ◽  
State Of The Art ◽  
Likelihood Estimation ◽  
Generative Models ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Benchmark Datasets ◽  
Classification Tasks ◽  
Latent Representations ◽  
Low Uncertainty

As one of the most popular generative models, Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. However, when the decoder network is sufficiently expressive, VAE may lead to posterior collapse; that is, uninformative latent representations may be learned. To this end, in this paper, we propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space, and thus the representation can be learned in a meaningful and compact manner. Specifically, we first theoretically demonstrate that it will result in better latent space with high diversity and low uncertainty awareness by controlling the distribution of posterior’s parameters across the whole data accordingly. Then, without the introduction of new loss terms or modifying training strategies, we propose to exploit Dropout on the variances and Batch-Normalization on the means simultaneously to regularize their distributions implicitly. Furthermore, to evaluate the generalization effect, we also exploit DU-VAE for inverse autoregressive flow based-VAE (VAE-IAF) empirically. Finally, extensive experiments on three benchmark datasets clearly show that our approach can outperform state-of-the-art baselines on both likelihood estimation and underlying classification tasks.

Efficient Algorithm for Mining High Utility Pattern Considering Length Constraints

2019 ◽  
Vol 15 (3) ◽  
pp. 1-27
Author(s):  
Kuldeep Singh ◽  
Bhaskar Biswas
Keyword(s):  
Data Mining ◽  
Upper Bound ◽  
Efficient Algorithm ◽  
State Of The Art ◽  
Memory Usage ◽  
Important Data ◽  
Benchmark Datasets ◽  
Projection Techniques ◽  
High Utility ◽  
High Utility Itemsets

High utility itemset (HUI) mining is one of the popular and important data mining tasks. Several studies have been carried out on this topic, which often discovers a very large number of itemsets and rules, which reduces not only the efficiency but also the effectiveness of HUI mining. In order to increase the efficiency and discover more interesting HUIs, constraint-based mining plays an important role. To address this issue, the authors propose an algorithm to discover HUIs with length constraints named EHIL (Efficient High utility Itemsets with Length constraints) to decrease the number of HUIs by removing tiny itemsets. EHIL adopts two new upper bound named sub-tree and local utility for pruning and modify them by incorporating length constraints. To reduce the dataset scans, the proposed algorithm uses transaction merging and dataset projection techniques. The execution time improvements ranged from a modest five percent to two orders of magnitude across benchmark datasets. The memory usage is up to twenty-eight times less than state-of-the-art algorithm FHM+.

Sign in / Sign up

Export Citation Format

Share Document