scholarly journals Transfer Learning for Risk Classification of Social Media Posts: Model Evaluation Study (Preprint)

2019 ◽  
Author(s):  
Derek Howard ◽  
Marta M Maslej ◽  
Justin Lee ◽  
Jacob Ritchie ◽  
Geoffrey Woollard ◽  
...  
Keyword(s):  
Mental Health ◽  
Machine Learning ◽  
Social Media ◽  
Transfer Learning ◽  
Computational Linguistics ◽  
Feature Representation ◽  
Fine Tuning ◽  
Language Models ◽  
Universal Sentence ◽  
Text Feature

BACKGROUND Mental illness affects a significant portion of the worldwide population. Online mental health forums can provide a supportive environment for those afflicted and also generate a large amount of data that can be mined to predict mental health states using machine learning methods. OBJECTIVE This study aimed to benchmark multiple methods of text feature representation for social media posts and compare their downstream use with automated machine learning (AutoML) tools. We tested on datasets that contain posts labeled for perceived suicide risk or moderator attention in the context of self-harm. Specifically, we assessed the ability of the methods to prioritize posts that a moderator would identify for immediate response. METHODS We used 1588 labeled posts from the Computational Linguistics and Clinical Psychology (CLPsych) 2017 shared task collected from the Reachout.com forum. Posts were represented using lexicon-based tools, including Valence Aware Dictionary and sEntiment Reasoner, Empath, and Linguistic Inquiry and Word Count, and also using pretrained artificial neural network models, including DeepMoji, Universal Sentence Encoder, and Generative Pretrained Transformer-1 (GPT-1). We used Tree-based Optimization Tool and Auto-Sklearn as AutoML tools to generate classifiers to triage the posts. RESULTS The top-performing system used features derived from the GPT-1 model, which was fine-tuned on over 150,000 unlabeled posts from Reachout.com. Our top system had a macroaveraged F1 score of 0.572, providing a new state-of-the-art result on the CLPsych 2017 task. This was achieved without additional information from metadata or preceding posts. Error analyses revealed that this top system often misses expressions of hopelessness. In addition, we have presented visualizations that aid in the understanding of the learned classifiers. CONCLUSIONS In this study, we found that transfer learning is an effective strategy for predicting risk with relatively little labeled data and noted that fine-tuning of pretrained language models provides further gains when large amounts of unlabeled text are available.

scholarly journals Transfer Learning for Risk Classification of Social Media Posts: Model Evaluation Study

10.2196/15371 ◽  
2020 ◽  
Vol 22 (5) ◽  
pp. e15371 ◽  
Author(s):  
Derek Howard ◽  
Marta M Maslej ◽  
Justin Lee ◽  
Jacob Ritchie ◽  
Geoffrey Woollard ◽  
...  
Keyword(s):  
Mental Health ◽  
Machine Learning ◽  
Social Media ◽  
Transfer Learning ◽  
Computational Linguistics ◽  
Feature Representation ◽  
Fine Tuning ◽  
Language Models ◽  
Universal Sentence ◽  
Text Feature

Background Mental illness affects a significant portion of the worldwide population. Online mental health forums can provide a supportive environment for those afflicted and also generate a large amount of data that can be mined to predict mental health states using machine learning methods. Objective This study aimed to benchmark multiple methods of text feature representation for social media posts and compare their downstream use with automated machine learning (AutoML) tools. We tested on datasets that contain posts labeled for perceived suicide risk or moderator attention in the context of self-harm. Specifically, we assessed the ability of the methods to prioritize posts that a moderator would identify for immediate response. Methods We used 1588 labeled posts from the Computational Linguistics and Clinical Psychology (CLPsych) 2017 shared task collected from the Reachout.com forum. Posts were represented using lexicon-based tools, including Valence Aware Dictionary and sEntiment Reasoner, Empath, and Linguistic Inquiry and Word Count, and also using pretrained artificial neural network models, including DeepMoji, Universal Sentence Encoder, and Generative Pretrained Transformer-1 (GPT-1). We used Tree-based Optimization Tool and Auto-Sklearn as AutoML tools to generate classifiers to triage the posts. Results The top-performing system used features derived from the GPT-1 model, which was fine-tuned on over 150,000 unlabeled posts from Reachout.com. Our top system had a macroaveraged F1 score of 0.572, providing a new state-of-the-art result on the CLPsych 2017 task. This was achieved without additional information from metadata or preceding posts. Error analyses revealed that this top system often misses expressions of hopelessness. In addition, we have presented visualizations that aid in the understanding of the learned classifiers. Conclusions In this study, we found that transfer learning is an effective strategy for predicting risk with relatively little labeled data and noted that fine-tuning of pretrained language models provides further gains when large amounts of unlabeled text are available.

scholarly journals Sentiment Analysis Methods for HPV Vaccines Related Tweets Based on Transfer Learning

Healthcare ◽  
2020 ◽  
Vol 8 (3) ◽  
pp. 307
Author(s):  
Li Zhang ◽  
Haimeng Fan ◽  
Chengxia Peng ◽  
Guozheng Rao ◽  
Qing Cong
Keyword(s):  
Social Media ◽  
Sentiment Analysis ◽  
Transfer Learning ◽  
Hpv Vaccination ◽  
Experimental Results ◽  
Fine Tuning ◽  
Hpv Vaccines ◽  
Social Media Data ◽  
Public Sentiment ◽  
Media Data

The widespread use of social media provides a large amount of data for public sentiment analysis. Based on social media data, researchers can study public opinions on human papillomavirus (HPV) vaccines on social media using machine learning-based approaches that will help us understand the reasons behind the low vaccine coverage. However, social media data is usually unannotated, and data annotation is costly. The lack of an abundant annotated dataset limits the application of deep learning methods in effectively training models. To tackle this problem, we propose three transfer learning approaches to analyze the public sentiment on HPV vaccines on Twitter. One was transferring static embeddings and embeddings from language models (ELMo) and then processing by bidirectional gated recurrent unit with attention (BiGRU-Att), called DWE-BiGRU-Att. The others were fine-tuning pre-trained models with limited annotated data, called fine-tuning generative pre-training (GPT) and fine-tuning bidirectional encoder representations from transformers (BERT). The fine-tuned GPT model was built on the pre-trained generative pre-training (GPT) model. The fine-tuned BERT model was constructed with BERT model. The experimental results on the HPV dataset demonstrated the efficacy of the three methods in the sentiment analysis of the HPV vaccination task. The experimental results on the HPV dataset demonstrated the efficacy of the methods in the sentiment analysis of the HPV vaccination task. The fine-tuned BERT model outperforms all other methods. It can help to find strategies to improve vaccine uptake.

scholarly journals Transfer Learning and Deep Domain Adaptation

2020 ◽  
Author(s):  
Wen Xu ◽  
Jing He ◽  
Yanfeng Shu
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Transfer Learning ◽  
Real World ◽  
Deep Neural Networks ◽  
Domain Adaptation ◽  
Fine Tuning ◽  
Real World Applications ◽  
Comprehensive Survey ◽  
Sample Reconstruction

Transfer learning is an emerging technique in machine learning, by which we can solve a new task with the knowledge obtained from an old task in order to address the lack of labeled data. In particular deep domain adaptation (a branch of transfer learning) gets the most attention in recently published articles. The intuition behind this is that deep neural networks usually have a large capacity to learn representation from one dataset and part of the information can be further used for a new task. In this research, we firstly present the complete scenarios of transfer learning according to the domains and tasks. Secondly, we conduct a comprehensive survey related to deep domain adaptation and categorize the recent advances into three types based on implementing approaches: fine-tuning networks, adversarial domain adaptation, and sample-reconstruction approaches. Thirdly, we discuss the details of these methods and introduce some typical real-world applications. Finally, we conclude our work and explore some potential issues to be further addressed.

scholarly journals Schizophrenia Detection Using Machine Learning Approach from Social Media Content

Sensors ◽  
2021 ◽  
Vol 21 (17) ◽  
pp. 5924
Author(s):  
Yi Ji Bae ◽  
Midan Shim ◽  
Won Hee Lee
Keyword(s):  
Mental Health ◽  
Machine Learning ◽  
Social Media ◽  
Mental Health Problems ◽  
Negative Emotion ◽  
Supervised Machine Learning ◽  
Control Group ◽  
Learning Approaches ◽  
Linguistic Features ◽  
Media Texts

Schizophrenia is a severe mental disorder that ranks among the leading causes of disability worldwide. However, many cases of schizophrenia remain untreated due to failure to diagnose, self-denial, and social stigma. With the advent of social media, individuals suffering from schizophrenia share their mental health problems and seek support and treatment options. Machine learning approaches are increasingly used for detecting schizophrenia from social media posts. This study aims to determine whether machine learning could be effectively used to detect signs of schizophrenia in social media users by analyzing their social media texts. To this end, we collected posts from the social media platform Reddit focusing on schizophrenia, along with non-mental health related posts (fitness, jokes, meditation, parenting, relationships, and teaching) for the control group. We extracted linguistic features and content topics from the posts. Using supervised machine learning, we classified posts belonging to schizophrenia and interpreted important features to identify linguistic markers of schizophrenia. We applied unsupervised clustering to the features to uncover a coherent semantic representation of words in schizophrenia. We identified significant differences in linguistic features and topics including increased use of third person plural pronouns and negative emotion words and symptom-related topics. We distinguished schizophrenic from control posts with an accuracy of 96%. Finally, we found that coherent semantic groups of words were the key to detecting schizophrenia. Our findings suggest that machine learning approaches could help us understand the linguistic characteristics of schizophrenia and identify schizophrenia or otherwise at-risk individuals using social media texts.

scholarly journals Leveraging Transfer Learning to Analyze Opinions, Attitudes, and Behavioral Intentions Toward COVID-19 Vaccines: Social Media Content and Temporal Analysis (Preprint)

2021 ◽  
Author(s):  
Siru Liu ◽  
Jili Li ◽  
Jialin Liu
Keyword(s):  
Machine Learning ◽  
Social Media ◽  
Transfer Learning ◽  
Behavioral Intentions ◽  
Temporal Analysis ◽  
Public Understanding ◽  
Support Vector ◽  
Learning Models ◽  
The Public ◽  
Over Time

BACKGROUND The COVID-19 vaccine is considered to be the most promising approach to alleviate the pandemic. However, in recent surveys, acceptance of the COVID-19 vaccine has been low. To design more effective outreach interventions, there is an urgent need to understand public perceptions of COVID-19 vaccines. OBJECTIVE Our objective was to analyze the potential of leveraging transfer learning to detect tweets containing opinions, attitudes, and behavioral intentions toward COVID-19 vaccines, and to explore temporal trends as well as automatically extract topics across a large number of tweets. METHODS We developed machine learning and transfer learning models to classify tweets, followed by temporal analysis and topic modeling on a dataset of COVID-19 vaccine–related tweets posted from November 1, 2020 to January 31, 2021. We used the F1 values as the primary outcome to compare the performance of machine learning and transfer learning models. The statistical values and <i>P</i> values from the Augmented Dickey-Fuller test were used to assess whether users’ perceptions changed over time. The main topics in tweets were extracted by latent Dirichlet allocation analysis. RESULTS We collected 2,678,372 tweets related to COVID-19 vaccines from 841,978 unique users and annotated 5000 tweets. The F1 values of transfer learning models were 0.792 (95% CI 0.789-0.795), 0.578 (95% CI 0.572-0.584), and 0.614 (95% CI 0.606-0.622) for these three tasks, which significantly outperformed the machine learning models (logistic regression, random forest, and support vector machine). The prevalence of tweets containing attitudes and behavioral intentions varied significantly over time. Specifically, tweets containing positive behavioral intentions increased significantly in December 2020. In addition, we selected tweets in the following categories: positive attitudes, negative attitudes, positive behavioral intentions, and negative behavioral intentions. We then identified 10 main topics and relevant terms for each category. CONCLUSIONS Overall, we provided a method to automatically analyze the public understanding of COVID-19 vaccines from real-time data in social media, which can be used to tailor educational programs and other interventions to effectively promote the public acceptance of COVID-19 vaccines.

scholarly journals Machine Learning for Mental Health in Social Media: Bibliometric Study (Preprint)

2020 ◽  
Author(s):  
Jina Kim ◽  
Daeun Lee ◽  
Eunil Park
Keyword(s):  
Mental Health ◽  
Machine Learning ◽  
Social Media ◽  
Large Scale ◽  
Web Of Science ◽  
Research Area ◽  
Research Subjects ◽  
Comprehensive Overview ◽  
Time Saving ◽  
Continuous Growth

BACKGROUND Social media platforms provide an easily accessible and time-saving communication approach for individuals with mental disorders compared to face-to-face meetings with medical providers. Recently, machine learning (ML)-based mental health exploration using large-scale social media data has attracted significant attention. OBJECTIVE We aimed to provide a bibliometric analysis and discussion on research trends of ML for mental health in social media. METHODS Publications addressing social media and ML in the field of mental health were retrieved from the Scopus and Web of Science databases. We analyzed the publication distribution to measure productivity on sources, countries, institutions, authors, and research subjects, and visualized the trends in this field using a keyword co-occurrence network. The research methodologies of previous studies with high citations are also thoroughly described. RESULTS We obtained a total of 565 relevant papers published from 2015 to 2020. In the last 5 years, the number of publications has demonstrated continuous growth with <i>Lecture Notes in Computer Science</i> and <i>Journal of Medical Internet Research</i> as the two most productive sources based on Scopus and Web of Science records. In addition, notable methodological approaches with data resources presented in high-ranking publications were investigated. CONCLUSIONS The results of this study highlight continuous growth in this research area. Moreover, we retrieved three main discussion points from a comprehensive overview of highly cited publications that provide new in-depth directions for both researchers and practitioners.

scholarly journals Automated Paraphrase Quality Assessment Using Language Models and Transfer Learning

Computers ◽  
2021 ◽  
Vol 10 (12) ◽  
pp. 166
Author(s):  
Bogdan Nicula ◽  
Mihai Dascalu ◽  
Natalie N. Newton ◽  
Ellen Orcutt ◽  
Danielle S. McNamara
Keyword(s):  
Quality Assessment ◽  
Transfer Learning ◽  
Network Models ◽  
Learning Task ◽  
Fine Tuning ◽  
Language Models ◽  
Neural Network Models ◽  
Writing Ability ◽  
Four Dimensions ◽  
Timely Feedback

Learning to paraphrase supports both writing ability and reading comprehension, particularly for less skilled learners. As such, educational tools that integrate automated evaluations of paraphrases can be used to provide timely feedback to enhance learner paraphrasing skills more efficiently and effectively. Paraphrase identification is a popular NLP classification task that involves establishing whether two sentences share a similar meaning. Paraphrase quality assessment is a slightly more complex task, in which pairs of sentences are evaluated in-depth across multiple dimensions. In this study, we focus on four dimensions: lexical, syntactical, semantic, and overall quality. Our study introduces and evaluates various machine learning models using handcrafted features combined with Extra Trees, Siamese neural networks using BiLSTM RNNs, and pretrained BERT-based models, together with transfer learning from a larger general paraphrase corpus, to estimate the quality of paraphrases across the four dimensions. Two datasets are considered for the tasks involving paraphrase quality: ULPC (User Language Paraphrase Corpus) containing 1998 paraphrases and a smaller dataset with 115 paraphrases based on children’s inputs. The paraphrase identification dataset used for the transfer learning task is the MSRP dataset (Microsoft Research Paraphrase Corpus) containing 5801 paraphrases. On the ULPC dataset, our BERT model improves upon the previous baseline by at least 0.1 in F1-score across the four dimensions. When using fine-tuning from ULPC for the children dataset, both the BERT and Siamese neural network models improve upon their original scores by at least 0.11 F1-score. The results of these experiments suggest that transfer learning using generic paraphrase identification datasets can be successful, while at the same time obtaining comparable results in fewer epochs.

scholarly journals Automatic Incident Triage in Radiation Oncology Incident Learning System

Healthcare ◽  
2020 ◽  
Vol 8 (3) ◽  
pp. 272
Author(s):  
Khajamoinuddin Syed ◽  
William Sleeman ◽  
Michael Hagan ◽  
Jatinder Palta ◽  
Rishabh Kapoor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Transfer Learning ◽  
Language Processing ◽  
Radiation Oncology ◽  
Learning System ◽  
Fine Tuning ◽  
Support Vector ◽  
Learning Approach ◽  
Veterans Health ◽  
Incident Reports

The Radiotherapy Incident Reporting and Analysis System (RIRAS) receives incident reports from Radiation Oncology facilities across the US Veterans Health Affairs (VHA) enterprise and Virginia Commonwealth University (VCU). In this work, we propose a computational pipeline for analysis of radiation oncology incident reports. Our pipeline uses machine learning (ML) and natural language processing (NLP) based methods to predict the severity of the incidents reported in the RIRAS platform using the textual description of the reported incidents. These incidents in RIRAS are reviewed by a radiation oncology subject matter expert (SME), who initially triages some incidents based on the salient elements in the incident report. To automate the triage process, we used the data from the VHA treatment centers and the VCU radiation oncology department. We used NLP combined with traditional ML algorithms, including support vector machine (SVM) with linear kernel, and compared it against the transfer learning approach with the universal language model fine-tuning (ULMFiT) algorithm. In RIRAS, severities are divided into four categories; A, B, C, and D, with A being the most severe to D being the least. In this work, we built models to predict High (A & B) vs. Low (C & D) severity instead of all the four categories. Models were evaluated with macro-averaged precision, recall, and F1-Score. The Traditional ML machine learning (SVM-linear) approach did well on the VHA dataset with 0.78 F1-Score but performed poorly on the VCU dataset with 0.5 F1-Score. The transfer learning approach did well on both datasets with 0.81 F1-Score on VHA dataset and 0.68 F1-Score on the VCU dataset. Overall, our methods show promise in automating the triage and severity determination process from radiotherapy incident reports.

scholarly journals Quantifying Suicidal Ideation on Social Media using Machine Learning: A Critical Review

2021 ◽  
pp. 4092-4100
Author(s):  
Syed Tanzeel Rabani ◽  
Qamar Rayees Khan ◽  
Akib Mohi Ud Din Khanday
Keyword(s):  
Mental Health ◽  
Machine Learning ◽  
Social Media ◽  
Suicidal Ideation ◽  
Social Isolation ◽  
Suicide Prevention ◽  
Online Social Networks ◽  
Public Information ◽  
Machine Learning Techniques ◽  
Suicidal Thoughts

Suicidal ideation is one of the severe mental health issues and a serious social problem faced by our society. This problem has been usually dealt with through the psychological point of view, using clinical face to face settings. There are various risk factors associated with suicides, including social isolation, anxiety, depression, etc., that decrease the threshold for suicide. The COVID-19 pandemic further increases social isolation, posing a great threat to the human population. Posting suicidal thoughts on social media is gaining much attention due to the social stigma associated with the mental health. Online Social Networks (OSN) are increasingly used to express the suicidal thoughts. Recently, a top Indian actor industry took the harsh step of suicide. The last Instagram posts revealed signs of depression, which if anticipated could have saved the precious life. Recent research indicated that the public information on social media provides valuable insights on detecting the users with the suicidal ideation. The motive of this study is to provide a systematic review of the work done already in the use of social media for suicide prevention and propose a novel classification approach that classifies the suicide related tweets/ posts into three levels of distress. Moreover, our proposed classification task which was implemented through various machine learning techniques revealed high accuracy in classifying the suicidal posts. Among all algorithms, the best performing algorithm was that of the decision tree, with an F1 score ranging 0.95-0.97. After thoroughly studying the work achieved by different researchers in the area of suicide prevention, our study critically analyses those works and finds various research gaps and solves some of them. We believe that our work will motivate research community to look into other gaps that will in turn help psychiatrists, psychologists, and counsellors to protect individuals suffering from suicidal ideation.

Sign in / Sign up

Export Citation Format

Share Document