scholarly journals A Method for Constructing Supervised Time Topic Model Based on Variational Autoencoder

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Zhinan Gou ◽  
Yan Li ◽  
Zheng Huo
Keyword(s):  
Topic Modeling ◽  
Topic Model ◽  
State Of The Art ◽  
Generation Model ◽  
Construction Process ◽  
Research Papers ◽  
Generative Process ◽  
Model Based ◽  
Variational Autoencoder ◽  
Over Time

Topic modeling is a probabilistic generation model to find the representative topic of a document and has been successfully applied to various document-related tasks in recent years. Especially in the supervised topic model and time topic model, many methods have achieved some success. The supervised topic model can learn topics from documents annotated with multiple labels and the time topic model can learn topics that evolve over time in a sequentially organized corpus. However, there are some documents with multiple labels and time-stamped in reality, which need to construct a supervised time topic model to achieve document-related tasks. There are few research papers on the supervised time topic model. To solve this problem, we propose a method for constructing a supervised time topic model. By analysing the generative process of the supervised topic model and time topic model, respectively, we introduce the construction process of the supervised time topic model based on variational autoencoder in detail and conduct preliminary experiments. Experimental results demonstrate that the supervised time topic model outperforms several state-of-the-art topic models.

scholarly journals A Method for Constructing Supervised Topic Model Based on Term Frequency-Inverse Topic Frequency

Symmetry ◽  
2019 ◽  
Vol 11 (12) ◽  
pp. 1486
Author(s):  
Zhinan Gou ◽  
Zheng Huo ◽  
Yuanzhen Liu ◽  
Yi Yang
Keyword(s):  
Topic Modeling ◽  
Topic Model ◽  
State Of The Art ◽  
Topic Models ◽  
Document Classification ◽  
Experimental Results ◽  
Tag Recommendation ◽  
Term Frequency ◽  
Series Of Experiments ◽  
Dirichlet Prior

Supervised topic modeling has been successfully applied in the fields of document classification and tag recommendation in recent years. However, most existing models neglect the fact that topic terms have the ability to distinguish topics. In this paper, we propose a term frequency-inverse topic frequency (TF-ITF) method for constructing a supervised topic model, in which the weight of each topic term indicates the ability to distinguish topics. We conduct a series of experiments with not only the symmetric Dirichlet prior parameters but also the asymmetric Dirichlet prior parameters. Experimental results demonstrate that the result of introducing TF-ITF into a supervised topic model outperforms several state-of-the-art supervised topic models.

scholarly journals It all starts with entities: A Salient entity topic model

2019 ◽  
Vol 26 (5) ◽  
pp. 531-549
Author(s):  
Chuan Wu ◽  
Evangelos Kanoulas ◽  
Maarten de Rijke
Keyword(s):  
Topic Model ◽  
State Of The Art ◽  
Topic Models ◽  
Generation Process ◽  
Qualitative And Quantitative Analysis ◽  
Generative Process ◽  
Qualitative And Quantitative ◽  
Proposed Model ◽  
Topic Distribution ◽  
Document Generation

AbstractEntities play an essential role in understanding textual documents, regardless of whether the documents are short, such as tweets, or long, such as news articles. In short textual documents, all entities mentioned are usually considered equally important because of the limited amount of information. In long textual documents, however, not all entities are equally important: some are salient and others are not. Traditional entity topic models (ETMs) focus on ways to incorporate entity information into topic models to better explain the generative process of documents. However, entities are usually treated equally, without considering whether they are salient or not. In this work, we propose a novel ETM, Salient Entity Topic Model, to take salient entities into consideration in the document generation process. In particular, we model salient entities as a source of topics used to generate words in documents, in addition to the topic distribution of documents used in traditional topic models. Qualitative and quantitative analysis is performed on the proposed model. Application to entity salience detection demonstrates the effectiveness of our model compared to the state-of-the-art topic model baselines.

scholarly journals Layer-Assisted Neural Topic Modeling over Document Networks

2021 ◽  
Author(s):  
Yiming Wang ◽  
Ximing Li ◽  
Jihong Ouyang
Keyword(s):  
Text Classification ◽  
Topic Modeling ◽  
Link Prediction ◽  
Topic Model ◽  
Web Pages ◽  
Text Documents ◽  
Text Data ◽  
Generative Process ◽  
Network Links ◽  
Scientific Papers

Neural topic modeling provides a flexible, efficient, and powerful way to extract topic representations from text documents. Unfortunately, most existing models cannot handle the text data with network links, such as web pages with hyperlinks and scientific papers with citations. To resolve this kind of data, we develop a novel neural topic model , namely Layer-Assisted Neural Topic Model (LANTM), which can be interpreted from the perspective of variational auto-encoders. Our major motivation is to enhance the topic representation encoding by not only using text contents, but also the assisted network links. Specifically, LANTM encodes the texts and network links to the topic representations by an augmented network with graph convolutional modules, and decodes them by maximizing the likelihood of the generative process. The neural variational inference is adopted for efficient inference. Experimental results validate that LANTM significantly outperforms the existing models on topic quality, text classification and link prediction..

scholarly journals Regional stickiness of novel ideas in the scholarly international business community

2019 ◽  
Vol 26 (2) ◽  
pp. 145-165 ◽  
Author(s):  
Richard Franciscus Johannes Haans ◽  
Arjen van Witteloostuijn
Keyword(s):  
International Business ◽  
Topic Modeling ◽  
Negative Binomial ◽  
Topic Model ◽  
Geographic Origin ◽  
Business Community ◽  
Mental Maps ◽  
Content Type ◽  
The World ◽  
Over Time

Purpose The purpose of this paper is to investigate the geographic dissemination of work in International Business (IB) by investigating the extent to which research topics tend to see mostly local use – with authors from the same geographic region as the article identified by the topic model as the first article in JIBS building on the topic – vs global use – where topics are used by authors across the world. Design/methodology/approach Topic modeling is applied to all articles published in the Journal of International Business Studies between 1970 and 2015. The identified topics are traced from introduction until the end of the sampling period using negative binomial regression. These analyses are supplemented by comparing patterns over time. Findings The analyses show strong path dependency between the geographic origin of topics and their spread across the world. This suggests the existence of geographically narrow mental maps in the field, which the authors find have remained constant in North America, widened yet are still present in East Asia, and disappeared in Europe and other regions of the world over time. These results contribute to the study of globalization in the field of IB, and suggest that neither a true globalization nor North American hegemony has occurred in recent decades. Originality/value The application of topic modeling allows investigation of deeper cognitive structures and patterns underpinning the field, as compared to alternative methodologies.

Multiple Relational Topic Modeling for Noisy Short Texts

2018 ◽  
Vol 28 (11n12) ◽  
pp. 1559-1574 ◽  
Author(s):  
Zheng Liu ◽  
Chiyu Liu ◽  
Bin Xia ◽  
Tao Li
Keyword(s):  
Social Networks ◽  
Random Sampling ◽  
Topic Modeling ◽  
Topic Model ◽  
State Of The Art ◽  
Sampling Strategy ◽  
Social Analysis ◽  
High Quality ◽  
Modeling Methods ◽  
Latent Topics

Understanding contents in social networks by inferring high-quality latent topics from short texts is a significant task in social analysis, which is challenging because social network contents are usually extremely short, noisy and full of informal vocabularies. Due to the lack of sufficient word co-occurrence instances, well-known topic modeling methods such as LDA and LSA cannot uncover high-quality topic structures. Existing research works seek to pool short texts from social networks into pseudo documents or utilize the explicit relations among these short texts such as hashtags in tweets to make classic topic modeling methods work. In this paper, we explore this problem by proposing a topic model for noisy short texts with multiple relations called MRTM (Multiple Relational Topic Modeling). MRTM exploits both explicit and implicit relations by introducing a document-attribute distribution and a two-step random sampling strategy. Extensive experiments, compared with the state-of-the-art topic modeling approaches, demonstrate that MRTM can alleviate the word co-occurrence sparsity and uncover high-quality latent topics from noisy short texts.

Using Contextual Topic Model for a Query-Focused Multi-Document Summarizer

2016 ◽  
Vol 25 (01) ◽  
pp. 1660002 ◽  
Author(s):  
Guangbing Yang
Keyword(s):  
Topic Modeling ◽  
Information Overload ◽  
Topic Model ◽  
State Of The Art ◽  
Practical Application ◽  
Document Summarization ◽  
Bayesian Hierarchical ◽  
Novel Approach ◽  
Novel Method ◽  
Latent Topics

Oft-decried information overload is a serious problem that negatively impacts the comprehension of information in the digital age. Text summarization is a helpful process that can be used to alleviate this problem. With the aim of seeking a novel method to enhance the performance of multi-document summarization, this study proposes a novel approach to analyze the problem of multi-document summarization based on a mixture model, consisting of a contextual topic model from a Bayesian hierarchical topic modeling family for selecting candidate summary sentences, and a regression model in machine learning for generating the summary. By investigating hierarchical topics and their correlations with respect to the lexical co-occurrences of words, the proposed contextual topic model can determine the relevance of sentences more effectively, recognize latent topics, and arrange them hierarchically. The quantitative evaluation results from a practical application demonstrates that a system implementing this model can significantly improve the performance of summarization and make it comparable to state-of-the-art summarization systems.

Enhanced Frequent Itemsets Based on Topic Modeling in Information Filtering

2017 ◽  
Vol 5 (4) ◽  
pp. 33-43
Author(s):  
Than Than Wai ◽  
Sint Sint Aung
Keyword(s):  
Topic Modeling ◽  
Information Needs ◽  
Pattern Mining ◽  
Topic Model ◽  
State Of The Art ◽  
Information Filtering ◽  
Frequent Pattern Mining ◽  
Frequent Itemsets ◽  
Frequent Pattern ◽  
Proposed Model

In order to generate user's information needs from a collection of documents, many term-based and pattern-based approaches have been used in Information Filtering. In these approaches, the documents in the collection are all about one topic. However, user's interests can be diverse and the documents in the collection often involve multiple topics. Topic modeling is useful for the area of machine learning and text mining. It generates models to discover the hidden multiple topics in a collection of documents and each of these topics are presented by distribution of words. But its effectiveness in information filtering has not been so well explored. Patterns are always thought to be more discriminative than single terms for describing documents. The major challenge found in frequent pattern mining is a large number of result patterns. As the minimum threshold becomes lower, an exponentially large number of patterns are generated. To deal with the above mentioned limitations and problems, in this paper, a novel information filtering model, EFITM (Enhanced Frequent Itemsets based on Topic Model) model is proposed. Experimental results using the CRANFIELD dataset for the task of information filtering show that the proposed model outperforms over state-of-the-art models.

scholarly journals Learning Topic Models by Neighborhood Aggregation

2019 ◽  
Author(s):  
Ryohei Hisano
Keyword(s):  
Topic Modeling ◽  
Latent Dirichlet Allocation ◽  
Topic Model ◽  
State Of The Art ◽  
Topic Models ◽  
Word Embedding ◽  
Output Function ◽  
Inference Procedure ◽  
Simple Manner ◽  
Classification Tasks

Topic models are frequently used in machine learning owing to their high interpretability and modular structure. However, extending a topic model to include a supervisory signal, to incorporate pre-trained word embedding vectors and to include a nonlinear output function is not an easy task because one has to resort to a highly intricate approximate inference procedure. The present paper shows that topic modeling with pre-trained word embedding vectors can be viewed as implementing a neighborhood aggregation algorithm where messages are passed through a network defined over words. From the network view of topic models, nodes correspond to words in a document and edges correspond to either a relationship describing co-occurring words in a document or a relationship describing the same word in the corpus. The network view allows us to extend the model to include supervisory signals, incorporate pre-trained word embedding vectors and include a nonlinear output function in a simple manner. In experiments, we show that our approach outperforms the state-of-the-art supervised Latent Dirichlet Allocation implementation in terms of held-out document classification tasks.

Action Generation Model Based on Changes in State Patterns

2009 ◽  
Vol 129 (9) ◽  
pp. 1690-1698
Author(s):  
Manabu Gouko ◽  
Naoki Tomi ◽  
Tomoaki Nagano ◽  
Koji Ito
Keyword(s):  
Generation Model ◽  
Model Based

Investigating Diseases and Chemicals in COVID-19 Literature with Text Mining (Preprint)

2020 ◽  
Author(s):  
Amir Karami ◽  
Brandon Bookstaver ◽  
Melissa Nolan
Keyword(s):  
Text Mining ◽  
Literature Review ◽  
Topic Modeling ◽  
Large Scale ◽  
Clinical Manifestations ◽  
International Health ◽  
Research Papers ◽  
Strategic Plans ◽  
Funding Agencies ◽  
The Relationship

BACKGROUND The COVID-19 pandemic has impacted nearly all aspects of life and has posed significant threats to international health and the economy. Given the rapidly unfolding nature of the current pandemic, there is an urgent need to streamline literature synthesis of the growing scientific research to elucidate targeted solutions. While traditional systematic literature review studies provide valuable insights, these studies have restrictions, including analyzing a limited number of papers, having various biases, being time-consuming and labor-intensive, focusing on a few topics, incapable of trend analysis, and lack of data-driven tools. OBJECTIVE This study fills the mentioned restrictions in the literature and practice by analyzing two biomedical concepts, clinical manifestations of disease and therapeutic chemical compounds, with text mining methods in a corpus containing COVID-19 research papers and find associations between the two biomedical concepts. METHODS This research has collected papers representing COVID-19 pre-prints and peer-reviewed research published in 2020. We used frequency analysis to find highly frequent manifestations and therapeutic chemicals, representing the importance of the two biomedical concepts. This study also applied topic modeling to find the relationship between the two biomedical concepts. RESULTS We analyzed 9,298 research papers published through May 5, 2020 and found 3,645 disease-related and 2,434 chemical-related articles. The most frequent clinical manifestations of disease terminology included COVID-19, SARS, cancer, pneumonia, fever, and cough. The most frequent chemical-related terminology included Lopinavir, Ritonavir, Oxygen, Chloroquine, Remdesivir, and water. Topic modeling provided 25 categories showing relationships between our two overarching categories. These categories represent statistically significant associations between multiple aspects of each category, some connections of which were novel and not previously identified by the scientific community. CONCLUSIONS Appreciation of this context is vital due to the lack of a systematic large-scale literature review survey and the importance of fast literature review during the current COVID-19 pandemic for developing treatments. This study is beneficial to researchers for obtaining a macro-level picture of literature, to educators for knowing the scope of literature, to journals for exploring most discussed disease symptoms and pharmaceutical targets, and to policymakers and funding agencies for creating scientific strategic plans regarding COVID-19.

Sign in / Sign up

Export Citation Format

Share Document