Industrial Federated Topic Modeling

2021 ◽  
Vol 12 (1) ◽  
pp. 1-22
Author(s):  
Di Jiang ◽  
Yongxin Tong ◽  
Yuanfeng Song ◽  
Xueyang Wu ◽  
Weiwei Zhao ◽  
...  
Keyword(s):  
Topic Modeling ◽  
Data Privacy ◽  
Topic Models ◽  
Real Life ◽  
Industrial Applications ◽  
High Quality ◽  
Heterogeneous Model ◽  
Data Scarcity ◽  
Probabilistic Topic Modeling ◽  
Training Topic

Probabilistic topic modeling has been applied in a variety of industrial applications. Training a high-quality model usually requires a massive amount of data to provide comprehensive co-occurrence information for the model to learn. However, industrial data such as medical or financial records are often proprietary or sensitive, which precludes uploading to data centers. Hence, training topic models in industrial scenarios using conventional approaches faces a dilemma: A party (i.e., a company or institute) has to either tolerate data scarcity or sacrifice data privacy. In this article, we propose a framework named Industrial Federated Topic Modeling (iFTM), in which multiple parties collaboratively train a high-quality topic model by simultaneously alleviating data scarcity and maintaining immunity to privacy adversaries. iFTM is inspired by federated learning, supports two representative topic models (i.e., Latent Dirichlet Allocation and SentenceLDA) in industrial applications, and consists of novel techniques such as private Metropolis-Hastings, topic-wise normalization, and heterogeneous model integration. We conduct quantitative evaluations to verify the effectiveness of iFTM and deploy iFTM in two real-life applications to demonstrate its utility. Experimental results verify iFTM’s superiority over conventional topic modeling.

scholarly journals Predicting inpatient clinical order patterns with probabilistic topic models vs conventional order sets

2016 ◽  
Vol 24 (3) ◽  
pp. 472-480 ◽  
Author(s):  
Jonathan H Chen ◽  
Mary K Goldstein ◽  
Steven M Asch ◽  
Lester Mackey ◽  
Russ B Altman
Keyword(s):  
Decision Support ◽  
Topic Modeling ◽  
Operating Characteristic ◽  
Topic Model ◽  
Characteristic Curve ◽  
Topic Models ◽  
Probabilistic Topic Models ◽  
Probabilistic Topic Modeling ◽  
Order Sets ◽  
Operating Characteristic Curve

Objective: Build probabilistic topic model representations of hospital admissions processes and compare the ability of such models to predict clinical order patterns as compared to preconstructed order sets. Materials and Methods: The authors evaluated the first 24 hours of structured electronic health record data for > 10 K inpatients. Drawing an analogy between structured items (e.g., clinical orders) to words in a text document, the authors performed latent Dirichlet allocation probabilistic topic modeling. These topic models use initial clinical information to predict clinical orders for a separate validation set of > 4 K patients. The authors evaluated these topic model-based predictions vs existing human-authored order sets by area under the receiver operating characteristic curve, precision, and recall for subsequent clinical orders. Results: Existing order sets predict clinical orders used within 24 hours with area under the receiver operating characteristic curve 0.81, precision 16%, and recall 35%. This can be improved to 0.90, 24%, and 47% (P < 10−20) by using probabilistic topic models to summarize clinical data into up to 32 topics. Many of these latent topics yield natural clinical interpretations (e.g., “critical care,” “pneumonia,” “neurologic evaluation”). Discussion: Existing order sets tend to provide nonspecific, process-oriented aid, with usability limitations impairing more precise, patient-focused support. Algorithmic summarization has the potential to breach this usability barrier by automatically inferring patient context, but with potential tradeoffs in interpretability. Conclusion: Probabilistic topic modeling provides an automated approach to detect thematic trends in patient care and generate decision support content. A potential use case finds related clinical orders for decision support.

Topic Modeling Using Latent Dirichlet allocation

2022 ◽  
Vol 54 (7) ◽  
pp. 1-35
Author(s):  
Uttam Chauhan ◽  
Apurva Shah
Keyword(s):  
Topic Modeling ◽  
Latent Dirichlet Allocation ◽  
Research Work ◽  
Topic Models ◽  
Small Subset ◽  
Distributed Environment ◽  
Future Directions ◽  
Probabilistic Topic Modeling ◽  
Modeling Techniques ◽  
Evaluation Techniques

We are not able to deal with a mammoth text corpus without summarizing them into a relatively small subset. A computational tool is extremely needed to understand such a gigantic pool of text. Probabilistic Topic Modeling discovers and explains the enormous collection of documents by reducing them in a topical subspace. In this work, we study the background and advancement of topic modeling techniques. We first introduce the preliminaries of the topic modeling techniques and review its extensions and variations, such as topic modeling over various domains, hierarchical topic modeling, word embedded topic models, and topic models in multilingual perspectives. Besides, the research work for topic modeling in a distributed environment, topic visualization approaches also have been explored. We also covered the implementation and evaluation techniques for topic models in brief. Comparison matrices have been shown over the experimental results of the various categories of topic modeling. Diverse technical challenges and future directions have been discussed.

scholarly journals An Empirical Study on Crosslingual Transfer in Probabilistic Topic Models

2020 ◽  
Vol 46 (1) ◽  
pp. 95-134
Author(s):  
Shudong Hao ◽  
Michael J. Paul
Keyword(s):  
Knowledge Transfer ◽  
Empirical Study ◽  
Future Development ◽  
Topic Modeling ◽  
Topic Models ◽  
Training Corpus ◽  
Training Conditions ◽  
Probabilistic Topic Models ◽  
Probabilistic Topic Modeling ◽  
Transfer Mechanisms

Probabilistic topic modeling is a common first step in crosslingual tasks to enable knowledge transfer and extract multilingual features. Although many multilingual topic models have been developed, their assumptions about the training corpus are quite varied, and it is not clear how well the different models can be utilized under various training conditions. In this article, the knowledge transfer mechanisms behind different multilingual topic models are systematically studied, and through a broad set of experiments with four models on ten languages, we provide empirical insights that can inform the selection and future development of multilingual topic models.

scholarly journals Improving Topic Coherence Using Entity Extraction Denoising

2018 ◽  
Vol 110 (1) ◽  
pp. 85-101 ◽  
Author(s):  
Ronald Cardenas ◽  
Kevin Bello ◽  
Alberto Coronado ◽  
Elizabeth Villota
Keyword(s):  
Topic Modeling ◽  
Human Perception ◽  
Relevant Information ◽  
Entity Recognition ◽  
Entity Extraction ◽  
Fine Grained ◽  
Job Advertisement ◽  
Coherence Score ◽  
Probabilistic Topic Modeling ◽  
Promising Solution

Abstract Managing large collections of documents is an important problem for many areas of science, industry, and culture. Probabilistic topic modeling offers a promising solution. Topic modeling is an unsupervised machine learning method and the evaluation of this model is an interesting problem on its own. Topic interpretability measures have been developed in recent years as a more natural option for topic quality evaluation, emulating human perception of coherence with word sets correlation scores. In this paper, we show experimental evidence of the improvement of topic coherence score by restricting the training corpus to that of relevant information in the document obtained by Entity Recognition. We experiment with job advertisement data and find that with this approach topic models improve interpretability in about 40 percentage points on average. Our analysis reveals as well that using the extracted text chunks, some redundant topics are joined while others are split into more skill-specific topics. Fine-grained topics observed in models using the whole text are preserved.

scholarly journals A Study on the Characteristics of Academic Topics Related to Renewable Energy Using the Structural Topic Modeling and the Weak Signal Concept

Energies ◽  
2021 ◽  
Vol 14 (5) ◽  
pp. 1497
Author(s):  
Chankook Park ◽  
Minkyu Kim
Keyword(s):  
Renewable Energy ◽  
Energy Storage ◽  
Topic Modeling ◽  
Academic Research ◽  
High Rate ◽  
Weak Signals ◽  
Energy Potential ◽  
Rate Of Increase ◽  
Probabilistic Topic Modeling ◽  
To Receive

It is important to examine in detail how the distribution of academic research topics related to renewable energy is structured and which topics are likely to receive new attention in the future in order for scientists to contribute to the development of renewable energy. This study uses an advanced probabilistic topic modeling to statistically examine the temporal changes of renewable energy topics by using academic abstracts from 2010–2019 and explores the properties of the topics from the perspective of future signs such as weak signals. As a result, in strong signals, methods for optimally integrating renewable energy into the power grid are paid great attention. In weak signals, interest in large-capacity energy storage systems such as hydrogen, supercapacitors, and compressed air energy storage showed a high rate of increase. In not-strong-but-well-known signals, comprehensive topics have been included, such as renewable energy potential, barriers, and policies. The approach of this study is applicable not only to renewable energy but also to other subjects.

scholarly journals The contribution of the Italian residents in neurology to the COVID-19 crisis: admirable generosity but neurological training remains their priority

2021 ◽  
Author(s):  
Cristina Tassorelli ◽  
Vincenzo Silani ◽  
Alessandro Padovani ◽  
Paolo Barone ◽  
Paolo Calabresi ◽  
...  
Keyword(s):  
Clinical Training ◽  
Real Life ◽  
Healthcare Management ◽  
Point Of View ◽  
Research Activity ◽  
High Quality ◽  
Research Activities ◽  
Italian Regions ◽  
Training And Research

Abstract Background The coronavirus disease 2019 (COVID-19) pandemic has severely impacted the Italian healthcare system, underscoring a dramatic shortage of specialized doctors in many disciplines. The situation affected the activity of the residents in neurology, who were also offered the possibility of being formally hired before their training completion. Aims (1) To showcase examples of clinical and research activity of residents in neurology during the COVID-19 pandemic in Italy and (2) to illustrate the point of view of Italian residents in neurology about the possibility of being hired before the completion of their residency program. Results Real-life reports from several areas in Lombardia—one of the Italian regions more affected by COVID-19—show that residents in neurology gave an outstanding demonstration of generosity, collaboration, reliability, and adaptation to the changing environment, while continuing their clinical training and research activities. A very small minority of the residents participated in the dedicated selections for being hired before completion of their training program. The large majority of them prioritized their training over the option of earlier employment. Conclusions Italian residents in neurology generously contributed to the healthcare management of the COVID-19 pandemic in many ways, while remaining determined to pursue their training. Neurology is a rapidly evolving clinical field due to continuous diagnostic and therapeutic progress. Stakeholders need to listen to the strong message conveyed by our residents in neurology and endeavor to provide them with the most adequate training, to ensure high quality of care and excellence in research in the future.

scholarly journals Human Annotated Dialogues Dataset for Natural Conversational Agents

2020 ◽  
Vol 10 (3) ◽  
pp. 762
Author(s):  
Erinc Merdivan ◽  
Deepika Singh ◽  
Sten Hanke ◽  
Johannes Kropf ◽  
Andreas Holzinger ◽  
...  
Keyword(s):  
Natural Language ◽  
Detailed Analysis ◽  
Human Perception ◽  
Natural Language Understanding ◽  
Industrial Applications ◽  
Benchmark Dataset ◽  
Conversational Agents ◽  
High Quality ◽  
Language Understanding ◽  
Major Drawback

Conversational agents are gaining huge popularity in industrial applications such as digital assistants, chatbots, and particularly systems for natural language understanding (NLU). However, a major drawback is the unavailability of a common metric to evaluate the replies against human judgement for conversational agents. In this paper, we develop a benchmark dataset with human annotations and diverse replies that can be used to develop such metric for conversational agents. The paper introduces a high-quality human annotated movie dialogue dataset, HUMOD, that is developed from the Cornell movie dialogues dataset. This new dataset comprises 28,500 human responses from 9500 multi-turn dialogue history-reply pairs. Human responses include: (i) ratings of the dialogue reply in relevance to the dialogue history; and (ii) unique dialogue replies for each dialogue history from the users. Such unique dialogue replies enable researchers in evaluating their models against six unique human responses for each given history. Detailed analysis on how dialogues are structured and human perception on dialogue score in comparison with existing models are also presented.

scholarly journals Towards Expert-Inspired Automatic Criterion to Cut a Dendrogram for Real-Industrial Applications

2021 ◽  
Author(s):  
Shikha Suman ◽  
Ashutosh Karna ◽  
Karina Gibert
Keyword(s):  
Hierarchical Clustering ◽  
Clustering Algorithms ◽  
Computational Cost ◽  
Real Life ◽  
Ground Truth ◽  
Industrial Applications ◽  
Underlying Structure ◽  
Cluster Validity ◽  
Cluster Validity Index ◽  
Number Of Clusters

Hierarchical clustering is one of the most preferred choices to understand the underlying structure of a dataset and defining typologies, with multiple applications in real life. Among the existing clustering algorithms, the hierarchical family is one of the most popular, as it permits to understand the inner structure of the dataset and find the number of clusters as an output, unlike popular methods, like k-means. One can adjust the granularity of final clustering to the goals of the analysis themselves. The number of clusters in a hierarchical method relies on the analysis of the resulting dendrogram itself. Experts have criteria to visually inspect the dendrogram and determine the number of clusters. Finding automatic criteria to imitate experts in this task is still an open problem. But, dependence on the expert to cut the tree represents a limitation in real applications like the fields industry 4.0 and additive manufacturing. This paper analyses several cluster validity indexes in the context of determining the suitable number of clusters in hierarchical clustering. A new Cluster Validity Index (CVI) is proposed such that it properly catches the implicit criteria used by experts when analyzing dendrograms. The proposal has been applied on a range of datasets and validated against experts ground-truth overcoming the results obtained by the State of the Art and also significantly reduces the computational cost.

scholarly journals Modified Energy Statistic for Unsupervised Anomaly Detection

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Rupam Mukherjee
Keyword(s):  
Anomaly Detection ◽  
Real Life ◽  
Industrial Applications ◽  
Test Point ◽  
Distance Metrics ◽  
Data Set ◽  
Energy Distance ◽  
New Energy ◽  
Statistical Distance ◽  
Computational Simplicity

For prognostics in industrial applications, the degree of anomaly of a test point from a baseline cluster is estimated using a statistical distance metric. Among different statistical distance metrics, energy distance is an interesting concept based on Newton’s Law of Gravitation, promising simpler computation than classical distance metrics. In this paper, we review the state of the art formulations of energy distance and point out several reasons why they are not directly applicable to the anomaly-detection problem. Thereby, we propose a new energy-based metric called the P-statistic which addresses these issues, is applicable to anomaly detection and retains the computational simplicity of the energy distance. We also demonstrate its effectiveness on a real-life data-set.

Sign in / Sign up

Export Citation Format

Share Document