scholarly journals Big Data Aspect-Based Opinion Mining Using the SLDA and HME-LDA Models

2020 ◽  
Vol 2020 ◽  
pp. 1-19
Author(s):  
Ling Yuan ◽  
JiaLi Bin ◽  
YinZhen Wei ◽  
Fei Huang ◽  
XiaoFei Hu ◽  
...  
Keyword(s):  
Big Data ◽  
Latent Dirichlet Allocation ◽  
Opinion Mining ◽  
Topic Model ◽  
Inverted Index ◽  
Data Set ◽  
Indicator Variables ◽  
Training Sets ◽  
Dirichlet Allocation

In order to make better use of massive network comment data for decision-making support of customers and merchants in the big data era, this paper proposes two unsupervised optimized LDA (Latent Dirichlet Allocation) models, namely, SLDA (SentiWordNet WordNet-Latent Dirichlet Allocation) and HME-LDA (Hierarchical Clustering MaxEnt-Latent Dirichlet Allocation), for aspect-based opinion mining. One scheme of each of two optimized models, which both use seed words as topic words and construct the inverted index, is designed to enhance the readability of experiment results. Meanwhile, based on the LDA topic model, we introduce new indicator variables to refine the classification of topics and try to classify the opinion target words and the sentiment opinion words by two different schemes. For better classification effect, the similarity between words and seed words is calculated in two ways to offset the fixed parameters in the standard LDA. In addition, based on the SemEval2016ABSA data set and the Yelp data set, we design comparative experiments with training sets of different sizes and different seed words, which prove that the SLDA and the HME-LDA have better performance on the accuracy, recall value, and harmonic value with unannotated training sets.

scholarly journals Intelligent radar software defect classification approach based on the latent Dirichlet allocation topic model

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Xi Liu ◽  
Yongfeng Yin ◽  
Haifeng Li ◽  
Jiabin Chen ◽  
Chang Liu ◽  
...  
Keyword(s):  
Latent Dirichlet Allocation ◽  
Topic Model ◽  
Recall Rate ◽  
Defect Classification ◽  
Software Defects ◽  
Classification Approach ◽  
Software Defect ◽  
Model Combining ◽  
Dirichlet Allocation

AbstractExisting software intelligent defect classification approaches do not consider radar characters and prior statistics information. Thus, when applying these appaoraches into radar software testing and validation, the precision rate and recall rate of defect classification are poor and have effect on the reuse effectiveness of software defects. To solve this problem, a new intelligent defect classification approach based on the latent Dirichlet allocation (LDA) topic model is proposed for radar software in this paper. The proposed approach includes the defect text segmentation algorithm based on the dictionary of radar domain, the modified LDA model combining radar software requirement, and the top acquisition and classification approach of radar software defect based on the modified LDA model. The proposed approach is applied on the typical radar software defects to validate the effectiveness and applicability. The application results illustrate that the prediction precison rate and recall rate of the poposed approach are improved up to 15 ~ 20% compared with the other defect classification approaches. Thus, the proposed approach can be applied in the segmentation and classification of radar software defects effectively to improve the identifying adequacy of the defects in radar software.

scholarly journals CLDA: An Effective Topic Model for Mining User Interest Preference under Big Data Background

Complexity ◽  
2018 ◽  
Vol 2018 ◽  
pp. 1-10 ◽  
Author(s):  
Lirong Qiu ◽  
Jia Yu
Keyword(s):  
Big Data ◽  
Topic Modeling ◽  
Latent Dirichlet Allocation ◽  
Topic Model ◽  
User Interest ◽  
Text Data ◽  
Data Set ◽  
Data Sparsity ◽  
Short Text ◽  
Text Filtering

In the present big data background, how to effectively excavate useful information is the problem that big data is facing now. The purpose of this study is to construct a more effective method of mining interest preferences of users in a particular field in the context of today’s big data. We mainly use a large number of user text data from microblog to study. LDA is an effective method of text mining, but it will not play a very good role in applying LDA directly to a large number of short texts in microblog. In today’s more effective topic modeling project, short texts need to be aggregated into long texts to avoid data sparsity. However, aggregated short texts are mixed with a lot of noise, reducing the accuracy of mining the user’s interest preferences. In this paper, we propose Combining Latent Dirichlet Allocation (CLDA), a new topic model that can learn the potential topics of microblog short texts and long texts simultaneously. The data sparsity of short texts is avoided by aggregating long texts to assist in learning short texts. Short text filtering long text is reused to improve mining accuracy, making long texts and short texts effectively combined. Experimental results in a real microblog data set show that CLDA outperforms many advanced models in mining user interest, and we also confirm that CLDA also has good performance in recommending systems.

scholarly journals Hybrid Sentiment Analyzer for Opinion Mining: Indian Admission Scenario

2020 ◽  
Vol 9 (4) ◽  
pp. 219-223
Keyword(s):  
Sentiment Analysis ◽  
Latent Dirichlet Allocation ◽  
Opinion Mining ◽  
Polar Form ◽  
Data Set ◽  
Admission Process ◽  
Whole Process ◽  
Institute Of Technology ◽  
Dirichlet Allocation

Social media has become one of the widely acclaimed tool for sharing information as well as expressing ideas and emotions. The work depicts the dual aspect task of analyzing and comprehending data available on Twitter platform. This is done using NLP techniques. Using Latent Dirichlet Allocation (LDA) topic technique; the major topics discussed in tweets (of data set taken), have been identified. The input for this Latent Dirichlet Allocation is given by NLP technique – Bag of Words. For further processing, identification of the underlying emotions contained in tweets using the techniques of Sentiment Analysis is done. The result of sentiment analysis is in the polar form. As a case study, a scenario of admissions in India for UG and PG has been considered. The whole process has captured the opinions of stake holders taking part in the admission process. Tweeter data of Indian Institute of Technology (IIT) admission has been used to collect the data in order to conduct the experiment. Major topics discussed in tweets and the fundamental emotions contained are obtained as results along with the polarity of the tweets

LF-LDA

2018 ◽  
Vol 14 (2) ◽  
pp. 18-36 ◽  
Author(s):  
Yongjun Zhang ◽  
Zijian Wang ◽  
Yongtao Yu ◽  
Bolun Chen ◽  
Jialin Ma ◽  
...  
Keyword(s):  
Big Data ◽  
Topic Model ◽  
State Of The Art ◽  
Text Documents ◽  
Data Set ◽  
Proposed Model ◽  
Textual Data ◽  
Noisy Function ◽  
Major Data

This article describes how text documents are a major data structure in the era of big data. With the explosive growth of data, the number of documents with multi-labels has increased dramatically. The popular multi-label classification technology, which is usually employed to handle multinomial text documents, is sensitive to the noise terms of text documents. Therefore, there still exists a huge room for multi-label classification of text documents. This article introduces a supervised topic model, named labeled LDA with function terms (LF-LDA), to filter out the noisy function terms from text documents, which can help to improve the performance of multi-label classification of text documents. The article also shows the derivation of the Gibbs Sampling formulas in detail, which can be generalized to other similar topic models. Based on the textual data set RCV1-v2, the article compared the proposed model with other two state-of-the-art multi-label classifiers, Tuned SVM and labeled LDA, on both Macro-F1 and Micro-F1 metrics. The result shows that LF-LDA outperforms them and has the lowest variance, which indicates the robustness of the LF-LDA classifier.

scholarly journals Intelligent Radar Software Defect Classification Approach based on the Latent Dirichlet Allocation Topic Model

2021 ◽  
Author(s):  
Xi Liu ◽  
Yongfeng Yin ◽  
Haifeng Li ◽  
Jiabin Chen ◽  
Chang Liu ◽  
...  
Keyword(s):  
Latent Dirichlet Allocation ◽  
Topic Model ◽  
Recall Rate ◽  
Defect Classification ◽  
Software Defects ◽  
Classification Approach ◽  
Software Defect ◽  
Model Combining ◽  
Dirichlet Allocation

Abstract Existing software intelligent defect classification approaches don’t consider radar characters and prior statistics information. Thus when applying these appaoraches into radar software testing and validation, the precision rate and recall rate of defect classification are poor and have effect on the reuse effectiveness of software defects. To solve this problem, a new intelligent defect classification approach based on the latent Dirichlet allocation (LDA) topic model is proposed for radar software in this paper. The proposed approach includes the defect text segmentation algorithm based on the dictionary of radar domain, the modified LDA model combining radar software requirement, the top acquisition and classification approach of radar software defect based on the modified LDA model. The proposed approach is applied on the typical radar software defects to validate the effectiveness and applicability. The application results illustrate that the prediction precison rate and recall rate of the poposed approach are improved up to 15%~20% compared with the other defect classification approaches. Thus, the proposed approach can be applied in the segmentation and classification of radar software defecs effectively to improve the identifying adequacy of the defects in radar software.

Mining numerical measure of consumers’ product evaluation expressed in words based on latent Dirichlet allocation

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Ziang Wang ◽  
Feng Yang
Keyword(s):  
Latent Dirichlet Allocation ◽  
Product Evaluation ◽  
Online Reviews ◽  
Product Evaluations ◽  
Product Attributes ◽  
Data Set ◽  
Content Type ◽  
Face To Face ◽  
Online Retailers ◽  
Dirichlet Allocation

Purpose It has always been a hot topic for online retailers to obtain consumers’ product evaluations from massive online reviews. In the process of online shopping, there is no face-to-face interaction between online retailers and customers. After collecting online reviews left by customers, online retailers are eager to acquire answers to some questions. For example, which product attributes will attract consumers? Or which step brings a better experience to consumers during the process of shopping? This paper aims to associate the latent Dirichlet allocation (LDA) model with the consumers’ attitude and provides a method to calculate the numerical measure of consumers’ product evaluation expressed in each word. Design/methodology/approach First, all possible pairs of reviews are organized as a document to build the corpus. After that, latent topics of the traditional LDA model noted as the standard LDA model, are separated into shared and differential topics. Then, the authors associate the model with consumers’ attitudes toward each review which is distinguished as positive review and non-positive review. The product evaluation reflected in consumers’ binary attitude is expanded to each word that appeared in the corpus. Finally, a variational optimization is introduced to calculate parameters mentioned in the expanded LDA model. Findings The experiment’s result illustrates that the LDA model in the research noted as an expanded LDA model, can successfully assign sufficient probability with words related to products attributes or consumers’ product evaluation. Compared with the standard LDA model, the expanded model intended to assign higher probability with words, which have a higher ranking within each topic. Besides, the expanded model also has higher precision on the prediction set, which shows that breaking down the topics into two categories fits better on the data set than the standard LDA model. The product evaluation of each word is calculated by the expanded model and depicted at the end of the experiment. Originality/value This research provides a new method to calculate consumers’ product evaluation from reviews in the level of words. Words may be used to describe product attributes or consumers’ experiences in reviews. Assigning words with numerical measures can analyze consumers’ products evaluation quantitatively. Besides, words are labeled themselves, they can also be ranked if a numerical measure is given. Online retailers can benefit from the result for label choosing, advertising or product recommendation.

Augmented Latent Dirichlet Allocation (Lda) Topic Model with Gaussian Mixture Topics

2018 ◽  
Author(s):  
Kedar S. Prabhudesai ◽  
Boyla O. Mainsah ◽  
Leslie M. Collins ◽  
Chandra S. Throckmorton
Keyword(s):  
Latent Dirichlet Allocation ◽  
Topic Model ◽  
Gaussian Mixture ◽  
Dirichlet Allocation

scholarly journals Application of Latent Dirichlet Allocation (LDA) for clustering financial tweets

2021 ◽  
Vol 297 ◽  
pp. 01071
Author(s):  
Sifi Fatima-Zahrae ◽  
Sabbar Wafae ◽  
El Mzabi Amal
Keyword(s):  
Language Processing ◽  
Latent Dirichlet Allocation ◽  
Sentiment Classification ◽  
Research Areas ◽  
Preprocessing Method ◽  
Long Time ◽  
Standard Text ◽  
The Given ◽  
Dirichlet Allocation

Sentiment classification is one of the hottest research areas among the Natural Language Processing (NLP) topics. While it aims to detect sentiment polarity and classification of the given opinion, requires a large number of aspect extractions. However, extracting aspect takes human effort and long time. To reduce this, Latent Dirichlet Allocation (LDA) method have come out recently to deal with this issue.In this paper, an efficient preprocessing method for sentiment classification is presented and will be used for analyzing user’s comments on Twitter social network. For this purpose, different text preprocessing techniques have been used on the dataset to achieve an acceptable standard text. Latent Dirichlet Allocation has been applied on the obtained data after this fast and accurate preprocessing phase. The implementation of different sentiment analysis methods and the results of these implementations have been compared and evaluated. The experimental results show that the combined uses of the preprocessing method of this paper and Latent Dirichlet Allocation have an acceptable results compared to other basic methods.

Sign in / Sign up

Export Citation Format

Share Document