scholarly journals Assessing the reliability of automatic sentiment analysis tools on rating the sentiment of reviews of NHS dental practices in England

PLoS ONE ◽  
2021 ◽  
Vol 16 (12) ◽  
pp. e0259797
Author(s):  
Matthew Byrne ◽  
Lucy O’Malley ◽  
Anne-Marie Glenny ◽  
Iain Pretty ◽  
Martin Tickle
Keyword(s):  
Sentiment Analysis ◽  
Large Scale ◽  
Online Reviews ◽  
Dental Service ◽  
Kappa Statistics ◽  
Service Contracts ◽  
Analysis Tools ◽  
Patient Reported ◽  
Dental Practices ◽  
Good Agreement

Background Online reviews may act as a rich source of data to assess the quality of dental practices. Assessing the content and sentiment of reviews on a large scale is time consuming and expensive. Automation of the process of assigning sentiment to big data samples of reviews may allow for reviews to be used as Patient Reported Experience Measures for primary care dentistry. Aim To assess the reliability of three different online sentiment analysis tools (Amazon Comprehend DetectSentiment API (ACDAPI), Google and Monkeylearn) at assessing the sentiment of reviews of dental practices working on National Health Service contracts in the United Kingdom. Methods A Python 3 script was used to mine 15800 reviews from 4803 unique dental practices on the NHS.uk websites between April 2018 –March 2019. A random sample of 270 reviews were rated by the three sentiment analysis tools. These reviews were rated by 3 blinded independent human reviewers and a pooled sentiment score was assigned. Kappa statistics and polychoric evalutaiton were used to assess the level of agreement. Disagreements between the automated and human reviewers were qualitatively assessed. Results There was good agreement between the sentiment assigned to reviews by the human reviews and ACDAPI (k = 0.660). The Google (k = 0.706) and Monkeylearn (k = 0.728) showed slightly better agreement at the expense of usability on a massive dataset. There were 33 disagreements in rating between ACDAPI and human reviewers, of which n = 16 were due to syntax errors, n = 10 were due to misappropriation of the strength of conflicting emotions and n = 7 were due to a lack of overtly emotive language in the text. Conclusions There is good agreement between the sentiment of an online review assigned by a group of humans and by cloud-based sentiment analysis. This may allow the use of automated sentiment analysis for quality assessment of dental service provision in the NHS.

Eliciting Attribute-Level User Needs from Online Reviews with Deep Language Models and Information Extraction

2020 ◽  
pp. 1-34
Author(s):  
Yi Han ◽  
Mohsen Moghaddam
Keyword(s):  
Sentiment Analysis ◽  
Large Scale ◽  
User Behavior ◽  
Language Model ◽  
Named Entity Recognition ◽  
Online Reviews ◽  
Entity Recognition ◽  
Language Models ◽  
Attribute Level ◽  
User Needs

Abstract Eliciting user needs for individual components and features of a product or a service on a large scale is a key requirement for innovative design. Gathering and analyzing data as an initial discovery phase of a design process is usually accomplished with a small number of participants, employing qualitative research methods such as observations, focus groups, and interviews. This leaves an entire swath of pertinent user behavior, preferences, and opinions not captured. Sentiment analysis is a key enabler for large-scale need finding from online user reviews generated on a regular basis. A major limitation of current sentiment analysis approaches used in design sciences, however, is the need for laborious labeling and annotation of large review datasets for training, which in turn hinders their scalability and transferability across different domains. This article proposes an efficient and scalable methodology for automated and large-scale elicitation of attribute-level user needs. The methodology builds on the state-of-the-art pretrained deep language model, BERT (Bidirectional Encoder Representations from Transformers), with new convolutional net and named-entity recognition (NER) layers for extracting attribute, description, and sentiment words from online user review corpora. The machine translation algorithm BLEU (BiLingual Evaluation Understudy) is utilized to extract need expressions in the form of predefined part-of-speech combinations (e.g., adjective-noun, verb-noun). Numerical experiments are conducted on a large dataset scraped from a major e-commerce retail store for apparel and footwear to demonstrate the performance, feasibility, and potentials of the developed methodology.

scholarly journals Sentiment Analysis of Japanese Tourism Online Reviews

2019 ◽  
Vol 4 (1) ◽  
pp. 89-113
Author(s):  
Chuanming Yu ◽  
Xingyu Zhu ◽  
Bolin Feng ◽  
Lin Cai ◽  
Lu An
Keyword(s):  
Feature Selection ◽  
Sentiment Analysis ◽  
Large Scale ◽  
Empirical Studies ◽  
Online Reviews ◽  
Learning Technology ◽  
Rule Based ◽  
Document Frequency ◽  
Overall Performance ◽  
Tourism Attractions

AbstractPurposeOnline reviews on tourism attractions provide important references for potential tourists to choose tourism spots. The main goal of this study is conducting sentiment analysis to facilitate users comprehending the large scale of the reviews, based on the comments about Chinese attractions from Japanese tourism website 4Travel.Design/methodology/approachDifferent statistics- and rule-based methods are used to analyze the sentiment of the reviews. Three groups of novel statistics-based methods combining feature selection functions and the traditional term frequency-inverse document frequency (TF-IDF) method are proposed. We also make seven groups of different rules-based methods. The macro-average and micro-average values for the best classification results of the methods are calculated respectively and the performance of the methods are shown.FindingsWe compare the statistics-based and rule-based methods separately and compare the overall performance of the two method. According to the results, it is concluded that the combination of feature selection functions and weightings can strongly improve the overall performance. The emotional vocabulary in the field of tourism (EVT), kaomojis, negative and transitional words can notably improve the performance in all of three categories. The rule-based methods outperform the statistics-based ones with a narrow advantage.Research limitationTwo limitations can be addressed: 1) the empirical studies to verify the validity of the proposed methods are only conducted on Japanese languages; and 2) the deep learning technology is not been incorporated in the methods.Practical implicationsThe results help to elucidate the intrinsic characteristics of the Japanese language and the influence on sentiment analysis. These findings also provide practical usage guidelines within the field of sentiment analysis of Japanese online tourism reviews.Originality/valueOur research is of practicability. Currently, there are no studies that focus on the sentiment analysis of Japanese reviews about Chinese attractions.

scholarly journals COVIDSenti: A Large-Scale Benchmark Twitter Data Set for COVID-19 Sentiment Analysis

2021 ◽  
pp. 1-13
Author(s):  
Usman Naseem ◽  
Imran Razzak ◽  
Matloob Khushi ◽  
Peter W. Eklund ◽  
Jinman Kim
Keyword(s):  
Sentiment Analysis ◽  
Large Scale ◽  
Data Set ◽  
Twitter Data

scholarly journals EZ-ALBI Score for Predicting Hepatocellular Carcinoma Prognosis

Liver Cancer ◽  
2020 ◽  
Vol 9 (6) ◽  
pp. 734-743
Author(s):  
Kazuya Kariyama ◽  
Kazuhiro Nouso ◽  
Atsushi Hiraoka ◽  
Akiko Wakuta ◽  
Ayano Oonishi ◽  
...  
Keyword(s):  
Hepatocellular Carcinoma ◽  
Liver Function ◽  
Large Scale ◽  
Regression Coefficient ◽  
Information Criterion ◽  
Proportional Hazard ◽  
Cox Proportional Hazard ◽  
Highly Correlated ◽  
Survival Risk ◽  
Good Agreement

<b><i>Introduction:</i></b> The ALBI score is acknowledged as the gold standard for the assessment of liver function in patients with hepatocellular carcinoma (HCC). Unlike the Child-Pugh score, the ALBI score uses only objective parameters, albumin (Alb) and total bilirubin (T.Bil), enabling a better evaluation. However, the complex calculation of the ALBI score limits its applicability. Therefore, we developed a simplified ALBI score, based on data from a large-scale HCC database.We used the data of 5,249 naïve HCC cases registered in eight collaborating hospitals. <b><i>Methods:</i></b> We developed a new score, the EZ (Easy)-ALBI score, based on regression coefficients of Alb and T.Bil for survival risk in a multivariate Cox proportional hazard model. We also developed the EZ-ALBI grade and EZ-ALBI-T grade as alternative options for the ALBI grade and ALBI-T grade and evaluated their stratifying ability. <b><i>Results:</i></b> The equation used to calculate the EZ-ALBI score was simple {[T.Bil (mg/dL)] – [9 × Alb (g/dL)]}; this value highly correlated with the ALBI score (correlation coefficient, 0.981; <i>p</i> &#x3c; 0.0001). The correlation was preserved across different Barcelona clinic liver cancer grade scores (regression coefficient, 0.93–0.98) and across different hospitals (regression coefficient, 0.98–0.99), indicating good generalizability. Although a good agreement was observed between ALBI and EZ-ALBI, discrepancies were observed in patients with poor liver function (T.Bil, ≥3 mg/dL; regression coefficient, 0.877). The stratifying ability of EZ-ALBI grade and EZ-ALBI-T grade were good and their Akaike’s information criterion values (35,897 and 34,812, respectively) were comparable with those of ALBI grade and ALBI-T grade (35,914 and 34,816, respectively). <b><i>Conclusions:</i></b> The EZ-ALBI score, EZ-ALBI grade, and EZ-ALBI-T grade are useful, simple scores, which might replace the conventional ALBI score in the future.

Aspect-based sentiment analysis for online reviews with hybrid attention networks

2021 ◽  
Author(s):  
Yuming Lin ◽  
Yu Fu ◽  
You Li ◽  
Guoyong Cai ◽  
Aoying Zhou
Keyword(s):  
Sentiment Analysis ◽  
Online Reviews ◽  
Attention Networks

Classifying Sentiment Based on LDA Model

2013 ◽  
Vol 427-429 ◽  
pp. 2614-2617
Author(s):  
Qing Xi Peng
Keyword(s):  
Sentiment Analysis ◽  
Topic Models ◽  
Online Reviews ◽  
Simultaneous Analysis ◽  
Experimental Result ◽  
Proposed Model

Online reviews as a new textual domain offer a unique proposition for sentiment analysis. Their short document length suggests any sentiment they contain is compact and explicit. Although supersized methods have obtained good results, a large amount of corpus should be trained beforehand. Recently, topic models have been introduced for the simultaneous analysis for sentiment in the document. However, the LDA model makes the assumption that, given the parameters the words in the document are all independent. It obviously isnt the case. The words in the document express the sentiment of the author. This paper proposes a model to solve the problem. We assume that the sentiments are related to the topic in the documents. A sentiment layer is added to the LDA model to improve it. Experimental result in the dataset demonstrates the advantage of the proposed model.

Sign in / Sign up

Export Citation Format

Share Document