scholarly journals Comparison of pretraining models and strategies for health-related social media text classification

2021 ◽  
Author(s):  
Yuting Guo ◽  
Yao Ge ◽  
Yuan-Chi Yang ◽  
Mohammed Ali Al-Garadi ◽  
Abeed Sarker
Keyword(s):  
Social Media ◽  
Language Processing ◽  
Text Classification ◽  
High Performance ◽  
Language Models ◽  
Learning Performance ◽  
Health Related ◽  
Social Media Text ◽  
Performance Results ◽  
Better Than

Motivation Pretrained contextual language models proposed in the recent past have been reported to achieve state-of-the-art performances in many natural language processing (NLP) tasks. There is a need to benchmark such models for targeted NLP tasks, and to explore effective pretraining strategies to improve machine learning performance. Results In this work, we addressed the task of health-related social media text classification. We benchmarked five models-RoBERTa, BERTweet, TwitterBERT, BioClinical_BERT, and BioBERT on 22 tasks. We attempted to boost performance for the best models by comparing distinct pretraining strategies-domain-adaptive pretraining (DAPT), source-adaptive pretraining (SAPT), and topic-specific pretraining (TSPT). RoBERTa and BERTweet performed comparably in most tasks, and better than others. For pretraining strategies, SAPT performed better or comparable to the off-the-shelf models, and significantly outperformed DAPT. SAPT+TSPT showed consistently high performance, with statistically significant improvement in one task. Our findings demonstrate that RoBERTa and BERTweet are excellent off-the-shelf models for health-related social media text classification, and extended pretraining using SAPT and TSPT can further improve performance.

scholarly journals Text Classification Models for the Automatic Detection of Nonmedical Prescription Medication Use from Social Media

2020 ◽  
Author(s):  
Ali Al-Garadi Mohammed ◽  
Yuan-Chi Yang ◽  
Haitao Cai ◽  
Yucheng Ruan ◽  
Karen O’Connor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Social Media ◽  
Language Processing ◽  
Text Classification ◽  
Prescription Medication ◽  
The United States ◽  
Language Models ◽  
Classification Model ◽  
Classification Models ◽  
Active Monitoring

ABSTRACTPrescription medication (PM) misuse/abuse has emerged as a national crisis in the United States, and social media has been suggested as a potential resource for performing active monitoring. However, automating a social media-based monitoring system is challenging—requiring advanced natural language processing (NLP) and machine learning methods. In this paper, we describe the development and evaluation of automatic text classification models for detecting self-reports of PM abuse from Twitter. We experimented with state-of-the-art bi-directional transformer-based language models, which utilize tweet-level representations that enable transfer learning (e.g., BERT, RoBERTa, XLNet, AlBERT, and DistilBERT), proposed fusion-based approaches, and compared the developed models with several traditional machine learning, including deep learning, approaches. Using a public dataset, we evaluated the performances of the classifiers on their abilities to classify the non-majority “abuse/misuse” class. Our proposed fusion-based model performs significantly better than the best traditional model (F1-score [95% CI]: 0.67 [0.64-0.69] vs. 0.45 [0.42-0.48]). We illustrate, via experimentation using differing training set sizes, that the transformer-based models are more stable and require less annotated data compared to the other models. The significant improvements achieved by our best-performing classification model over past approaches makes it suitable for automated continuous monitoring of nonmedical PM use from Twitter.

Joint Representations of Texts and Labels with Compositional Loss for Short Text Classification

2021 ◽  
Author(s):  
Ming Hao ◽  
Weijing Wang ◽  
Fang Zhou
Keyword(s):  
Language Processing ◽  
Text Classification ◽  
Ground Truth ◽  
Language Models ◽  
Text Representation ◽  
Short Text ◽  
Practical Applications ◽  
Classical Models ◽  
Multi Class Classification ◽  
Public Datasets

Short text classification is an important foundation for natural language processing (NLP) tasks. Though, the text classification based on deep language models (DLMs) has made a significant headway, in practical applications however, some texts are ambiguous and hard to classify in multi-class classification especially, for short texts whose context length is limited. The mainstream method improves the distinction of ambiguous text by adding context information. However, these methods rely only the text representation, and ignore that the categories overlap and are not completely independent of each other. In this paper, we establish a new general method to solve the problem of ambiguous text classification by introducing label embedding to represent each category, which makes measurable difference between the categories. Further, a new compositional loss function is proposed to train the model, which makes the text representation closer to the ground-truth label and farther away from others. Finally, a constraint is obtained by calculating the similarity between the text representation and label embedding. Errors caused by ambiguous text can be corrected by adding constraints to the output layer of the model. We apply the method to three classical models and conduct experiments on six public datasets. Experiments show that our method can effectively improve the classification accuracy of the ambiguous texts. In addition, combining our method with BERT, we obtain the state-of-the-art results on the CNT dataset.

scholarly journals Public Perception on Healthcare Services: Evidence from Social Media Platforms in China

2019 ◽  
Vol 16 (7) ◽  
pp. 1273 ◽  
Author(s):  
Guangyu Hu ◽  
Xueyan Han ◽  
Huixuan Zhou ◽  
Yuanli Liu
Keyword(s):  
Social Media ◽  
Language Processing ◽  
Public Perception ◽  
Content Volume ◽  
Healthcare Services ◽  
Service Improvement ◽  
Data Resource ◽  
Health Related Research ◽  
Health Related ◽  
Social Media Platforms

Social media has been used as data resource in a growing number of health-related research. The objectives of this study were to identify content volume and sentiment polarity of social media records relevant to healthcare services in China. A list of the key words of healthcare services were used to extract data from WeChat and Qzone, between June 2017 and September 2017. The data were put into a corpus, where content analyses were performed using Tencent natural language processing (NLP). The final corpus contained approximately 29 million records. Records on patient safety were the most frequently mentioned topic (approximately 8.73 million, 30.1% of the corpus), with the contents on humanistic care having received the least social media references (0.43 Million, 1.5%). Sentiment analyses showed 36.1%, 16.4%, and 47.4% of positive, neutral, and negative emotions, respectively. The doctor-patient relationship category had the highest proportion of negative contents (74.9%), followed by service efficiency (59.5%), and nursing service (53.0%). Neutral disposition was found to be the highest (30.4%) in the contents on appointment-booking services. This study added evidence to the magnitude and direction of public perceptions on healthcare services in China’s hospital and pointed to the possibility of monitoring healthcare service improvement, using readily available data in social media.

scholarly journals Automated extraction of treatment patterns from social media posts: an exploratory analysis in renal cell carcinoma

2019 ◽  
Vol 15 (31) ◽  
pp. 3587-3596
Author(s):  
Sreeram V Ramagopalan ◽  
Bill Malcolm ◽  
Evie Merinopoulou ◽  
Laura McDonald ◽  
Andrew Cox
Keyword(s):  
Renal Cell Carcinoma ◽  
Social Media ◽  
Cell Carcinoma ◽  
Language Processing ◽  
Renal Cell ◽  
Treatment Patterns ◽  
The Social ◽  
Health Related ◽  
Development And Validation ◽  
Further Development

Aim: The use of health-related social media forums by patients is increasing and the size of these forums creates a rich record of patient opinions and experiences, including treatment histories. This study aimed to understand the possibility of extracting treatment patterns in an automated manner for patients with renal cell carcinoma, using natural language processing, rule-based decisions, and machine learning. Patients & methods: Obtained results were compared with those from published observational studies. Results: 42 comparisons across seven therapies, three lines of treatment, and two-time periods were made; 37 of the social media estimates fell within the variation seen across the published studies. Conclusion: This exploratory work shows that estimating treatment patterns from social media is possible and generates results within the variation seen in published studies, although further development and validation of the approach is needed.

Social Media News Classification in Healthcare Communication

2019 ◽  
Vol 9 (6) ◽  
pp. 1215-1223 ◽  
Author(s):  
Fiaz Majeed ◽  
Muhammad Waqas Asif ◽  
Muhammad Awais Hassan ◽  
Syed Ali Abbas ◽  
M. Ikramullah Lali
Keyword(s):  
Social Media ◽  
Support Vector ◽  
Learning Approaches ◽  
Healthcare Organizations ◽  
Significant Information ◽  
Health News ◽  
Related Data ◽  
The Social ◽  
Health Related ◽  
Better Than

The trend of news transmission is rapidly shifting from electronic media to social media. Currently, news channels in general, while health news channels specifically send health related news on social media sites. These news are beneficial for the patients, medical professionals and the general public. A lot of health related data is available on the social media that may be used to extract significant information and present several predictions from it to assist physicians, patients and healthcare organizations for decision making. However, A little research is found on health news data using machine learning approaches, thus in this paper, we have proposed a framework for the data collection, modeling, and visualization of the health related patterns. For the analysis, the tweets of 13 news channels are collected from the Twitter. The dataset holds approximately 28k tweets available under 280 hashtags. Furthermore, a comprehensive set of experiments are performed to extract patterns from the data. A comparative analysis is carried among the baseline method and four classification algorithms which include Naive Bayes (NB), Support Vector Machine (SVM), Logistic Regression (LR), Decision Tree (J48). For the evaluation of the results, the standard measures accuracy, precision, recall and f-measure have been used. The results of the study are encouraging and better than the other studies of such kind.

scholarly journals Data and systems for medication-related text classification and concept normalization from Twitter: insights from the Social Media Mining for Health (SMM4H)-2017 shared task

2018 ◽  
Vol 25 (10) ◽  
pp. 1274-1283 ◽  
Author(s):  
Abeed Sarker ◽  
Maksim Belousov ◽  
Jasper Friedrichs ◽  
Kai Hakala ◽  
Svetlana Kiritchenko ◽  
...  
Keyword(s):  
Social Media ◽  
Text Classification ◽  
Training Data ◽  
Support Vector ◽  
Shared Task ◽  
Social Media Mining ◽  
The Social ◽  
Related Text ◽  
Health Related ◽  
Media Mining

AbstractObjectiveWe executed the Social Media Mining for Health (SMM4H) 2017 shared tasks to enable the community-driven development and large-scale evaluation of automatic text processing methods for the classification and normalization of health-related text from social media. An additional objective was to publicly release manually annotated data.Materials and MethodsWe organized 3 independent subtasks: automatic classification of self-reports of 1) adverse drug reactions (ADRs) and 2) medication consumption, from medication-mentioning tweets, and 3) normalization of ADR expressions. Training data consisted of 15 717 annotated tweets for (1), 10 260 for (2), and 6650 ADR phrases and identifiers for (3); and exhibited typical properties of social-media-based health-related texts. Systems were evaluated using 9961, 7513, and 2500 instances for the 3 subtasks, respectively. We evaluated performances of classes of methods and ensembles of system combinations following the shared tasks.ResultsAmong 55 system runs, the best system scores for the 3 subtasks were 0.435 (ADR class F1-score) for subtask-1, 0.693 (micro-averaged F1-score over two classes) for subtask-2, and 88.5% (accuracy) for subtask-3. Ensembles of system combinations obtained best scores of 0.476, 0.702, and 88.7%, outperforming individual systems.DiscussionAmong individual systems, support vector machines and convolutional neural networks showed high performance. Performance gains achieved by ensembles of system combinations suggest that such strategies may be suitable for operational systems relying on difficult text classification tasks (eg, subtask-1).ConclusionsData imbalance and lack of context remain challenges for natural language processing of social media text. Annotated data from the shared task have been made available as reference standards for future studies (http://dx.doi.org/10.17632/rxwfb3tysd.1).

scholarly journals Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features

2015 ◽  
Vol 22 (3) ◽  
pp. 671-681 ◽  
Author(s):  
Azadeh Nikfarjam ◽  
Abeed Sarker ◽  
Karen O’Connor ◽  
Rachel Ginn ◽  
Graciela Gonzalez
Keyword(s):  
Machine Learning ◽  
Social Media ◽  
Language Processing ◽  
High Performance ◽  
Conditional Random Fields ◽  
Training Data ◽  
Data Sets ◽  
Social Media Mining ◽  
Medical Concepts ◽  
Media Mining

Abstract Objective Social media is becoming increasingly popular as a platform for sharing personal health-related information. This information can be utilized for public health monitoring tasks, particularly for pharmacovigilance, via the use of natural language processing (NLP) techniques. However, the language in social media is highly informal, and user-expressed medical concepts are often nontechnical, descriptive, and challenging to extract. There has been limited progress in addressing these challenges, and thus far, advanced machine learning-based NLP techniques have been underutilized. Our objective is to design a machine learning-based approach to extract mentions of adverse drug reactions (ADRs) from highly informal text in social media. Methods We introduce ADRMine, a machine learning-based concept extraction system that uses conditional random fields (CRFs). ADRMine utilizes a variety of features, including a novel feature for modeling words’ semantic similarities. The similarities are modeled by clustering words based on unsupervised, pretrained word representation vectors (embeddings) generated from unlabeled user posts in social media using a deep learning technique. Results ADRMine outperforms several strong baseline systems in the ADR extraction task by achieving an F-measure of 0.82. Feature analysis demonstrates that the proposed word cluster features significantly improve extraction performance. Conclusion It is possible to extract complex medical concepts, with relatively high performance, from informal, user-generated content. Our approach is particularly scalable, suitable for social media mining, as it relies on large volumes of unlabeled data, thus diminishing the need for large, annotated training data sets.

scholarly journals New explainability method for BERT-based model in fake news detection

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Mateusz Szczepański ◽  
Marek Pawlicki ◽  
Rafał Kozik ◽  
Michał Choraś
Keyword(s):  
Artificial Intelligence ◽  
Social Media ◽  
Language Processing ◽  
High Performance ◽  
Real Life ◽  
Contemporary Society ◽  
Fake News ◽  
Interpretable Model ◽  
Deep Integration ◽  
The Impact

AbstractThe ubiquity of social media and their deep integration in the contemporary society has granted new ways to interact, exchange information, form groups, or earn money—all on a scale never seen before. Those possibilities paired with the widespread popularity contribute to the level of impact that social media display. Unfortunately, the benefits brought by them come at a cost. Social Media can be employed by various entities to spread disinformation—so called ‘Fake News’, either to make a profit or influence the behaviour of the society. To reduce the impact and spread of Fake News, a diverse array of countermeasures were devised. These include linguistic-based approaches, which often utilise Natural Language Processing (NLP) and Deep Learning (DL). However, as the latest advancements in the Artificial Intelligence (AI) domain show, the model’s high performance is no longer enough. The explainability of the system’s decision is equally crucial in real-life scenarios. Therefore, the objective of this paper is to present a novel explainability approach in BERT-based fake news detectors. This approach does not require extensive changes to the system and can be attached as an extension for operating detectors. For this purposes, two Explainable Artificial Intelligence (xAI) techniques, Local Interpretable Model-Agnostic Explanations (LIME) and Anchors, will be used and evaluated on fake news data, i.e., short pieces of text forming tweets or headlines. This focus of this paper is on the explainability approach for fake news detectors, as the detectors themselves were part of previous works of the authors.

Sign in / Sign up

Export Citation Format

Share Document