Global User-Level Perception of COVID-19 Contact Tracing Applications: A Data-Driven Approach Using Natural Language Processing (Preprint)

BACKGROUND Contact tracing has been globally adopted in the fight to control the infection rate of COVID-19. Thanks to digital technologies, such as smartphones and wearable devices, contacts of COVID-19 patients can be easily traced and informed about their potential exposure to the virus. To this aim, several mobile applications have been developed. However, there are ever-growing concerns over the working mechanism and performance of these applications. The literature already provides some interesting exploratory studies on the community’s response to the applications by analyzing information from different sources, such as news and users’ reviews of the applications. However, to the best of our knowledge, there is no existing solution that automatically analyzes users’ reviews and extracts the evoked sentiments. We believe such solutions combined with a user-friendly interface can be used as a rapid surveillance tool to monitor how effective an application is and to make immediate changes without going through an intense participatory design method which, although in normal circumstances is optimal, but not optimal in emergency situations where a mobile device needs to be deployed immediately with little to no user input from the beginning for the greater public good. OBJECTIVE In this paper, we aim to analyze the efficacy of AI models and Natural Language Processing (NLP) techniques in automatically extracting and classifying the polarity of users’ sentiments by proposing a sentiment analysis framework to automatically analyze users’ reviews on COVID-19 contact tracing mobile applications. We also aim to provide a large-scale annotated benchmark dataset to facilitate future research in the domain. As a proof of concepts, we also develop a potential web application, based on the proposed solutions, with a user-friendly interface to automatically analyze and classify users’ reviews on the COVID-19 contact tracing applications. The proposed framework combined with the interface which is expected to help the community in quickly analyzing users’ perception about such mobile applications and can be used as a rapid surveillance tool to monitor effectiveness of mobile applications and to make immediate changes without going through an intense participatory design method in emergency situations. METHODS We propose a pipeline starting from manual annotation via a crowd-sourcing study and concluding on the development and training of AI models for automatic sentiment analysis of users’ reviews. In detail, we collected and annotated a large- scale dataset of Android and iOS mobile applications users’ reviews for COVID-19 contact tracing. After manually analyzing and annotating users’ reviews, we employed both classical (i.e., Naïve Bayes, SVM, Random Forest) and deep learning (i.e., fastText, and different transformers) methods for classification experiments. This resulted in eight different classification models. RESULTS We employed eight different methods on three different tasks achieving up to an average F1-Scores 94.8% indicating the feasibility and applicability of automatic sentiment analysis of users’ reviews on the COVID-19 contact tracing applications. Moreover, the crowd-sourcing activity resulted in a large-scale benchmark dataset composed of 34,534 reviews manually annotated from the contract tracing applications of 46 distinct countries. The resulted dataset is also made publicly available for research usage. CONCLUSIONS The existing literature mostly relies on the manual/exploratory analysis of users’ reviews on the application, which is a tedious and time-consuming process. Moreover, in the existing studies, generally, data from fewer applications are analyzed. In this work, we showed that AI and NLP techniques provide good results in analyzing and classifying users’ sentiments’ polarity, and that the automatic sentiment analysis can help in analyzing users’ responses to the application more quickly with a significant accuracy. Moreover, we also provided a large-scale benchmark dataset composed of 34,534 reviews from 47 different applications. We believe the presented analysis, dataset, and the proposed solutions combined with a user-friendly interface can be used as a rapid surveillance tool to analyze and monitor mobile applications deployed in emergency situations leading to rapid changes in the applications without going through an intense participatory design method.

Download Full-text

Sentiment Analysis of Users’ Reviews on COVID-19 Contact Tracing Apps with a Benchmark Dataset (Preprint)

10.2196/preprints.28371 ◽

2021 ◽

Author(s):

Kashif Ahmad ◽

Firoj Alam ◽

Junaid Qadir ◽

Basheer Qolomony ◽

Imran Khan ◽

...

Keyword(s):

Sentiment Analysis ◽

Mobile Applications ◽

Large Scale ◽

Benchmark Dataset ◽

Contact Tracing ◽

Future Research ◽

Crowd Sourcing ◽

Large Scale Dataset ◽

And Performance ◽

Different Sources

Download Full-text

Using Natural Language Processing and Sentiment Analysis to Augment Traditional User-Centered Design: Development and Usability Study

JMIR mhealth and uhealth ◽

10.2196/16862 ◽

2020 ◽

Vol 8 (8) ◽

pp. e16862

Author(s):

Curtis Lee Petersen ◽

Ryan Halter ◽

David Kotz ◽

Lorie Loeb ◽

Summer Cook ◽

...

Keyword(s):

Remote Sensing ◽

Older Adults ◽

Natural Language Processing ◽

Sentiment Analysis ◽

Design Process ◽

Language Processing ◽

User Centered Design ◽

The Mean ◽

User Friendly ◽

User Centered

Background Sarcopenia, defined as the age-associated loss of muscle mass and strength, can be effectively mitigated through resistance-based physical activity. With compliance at approximately 40% for home-based exercise prescriptions, implementing a remote sensing system would help patients and clinicians to better understand treatment progress and increase compliance. The inclusion of end users in the development of mobile apps for remote-sensing systems can ensure that they are both user friendly and facilitate compliance. With advancements in natural language processing (NLP), there is potential for these methods to be used with data collected through the user-centered design process. Objective This study aims to develop a mobile app for a novel device through a user-centered design process with both older adults and clinicians while exploring whether data collected through this process can be used in NLP and sentiment analysis Methods Through a user-centered design process, we conducted semistructured interviews during the development of a geriatric-friendly Bluetooth-connected resistance exercise band app. We interviewed patients and clinicians at weeks 0, 5, and 10 of the app development. Each semistructured interview consisted of heuristic evaluations, cognitive walkthroughs, and observations. We used the Bing sentiment library for a sentiment analysis of interview transcripts and then applied NLP-based latent Dirichlet allocation (LDA) topic modeling to identify differences and similarities in patient and clinician participant interviews. Sentiment was defined as the sum of positive and negative words (each word with a +1 or −1 value). To assess utility, we used quantitative assessment questionnaires—System Usability Scale (SUS) and Usefulness, Satisfaction, and Ease of use (USE). Finally, we used multivariate linear models—adjusting for age, sex, subject group (clinician vs patient), and development—to explore the association between sentiment analysis and SUS and USE outcomes. Results The mean age of the 22 participants was 68 (SD 14) years, and 17 (77%) were female. The overall mean SUS and USE scores were 66.4 (SD 13.6) and 41.3 (SD 15.2), respectively. Both patients and clinicians provided valuable insights into the needs of older adults when designing and building an app. The mean positive-negative sentiment per sentence was 0.19 (SD 0.21) and 0.47 (SD 0.21) for patient and clinician interviews, respectively. We found a positive association with positive sentiment in an interview and SUS score (ß=1.38; 95% CI 0.37 to 2.39; P=.01). There was no significant association between sentiment and the USE score. The LDA analysis found no overlap between patients and clinicians in the 8 identified topics. Conclusions Involving patients and clinicians allowed us to design and build an app that is user friendly for older adults while supporting compliance. This is the first analysis using NLP and usability questionnaires in the quantification of user-centered design of technology for older adults.

Download Full-text

Solving Analogies on Words based on Minimal Complexity Transformation

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/256 ◽

2020 ◽

Author(s):

Pierre-Alexandre Murena ◽

Marie Al-Ghossein ◽

Jean-Louis Dessalles ◽

Antoine Cornuéjols

Keyword(s):

Natural Language Processing ◽

Natural Language ◽

Language Processing ◽

Large Scale ◽

State Of The Art ◽

Benchmark Dataset ◽

Alternative Approach

Analogies are 4-ary relations of the form "A is to B as C is to D". When A, B and C are fixed, we call analogical equation the problem of finding the correct D. A direct applicative domain is Natural Language Processing, in which it has been shown successful on word inflections, such as conjugation or declension. If most approaches rely on the axioms of proportional analogy to solve these equations, these axioms are known to have limitations, in particular in the nature of the considered flections. In this paper, we propose an alternative approach, based on the assumption that optimal word inflections are transformations of minimal complexity. We propose a rough estimation of complexity for word analogies and an algorithm to find the optimal transformations. We illustrate our method on a large-scale benchmark dataset and compare with state-of-the-art approaches to demonstrate the interest of using complexity to solve analogies on words.

Download Full-text

Using Natural Language Processing and Sentiment Analysis to Augment Traditional User-Centered Design: Development and Usability Study (Preprint)

10.2196/preprints.16862 ◽

2019 ◽

Author(s):

Curtis Lee Petersen ◽

Ryan Halter ◽

David Kotz ◽

Lorie Loeb ◽

Summer Cook ◽

...

Keyword(s):

Remote Sensing ◽

Older Adults ◽

Natural Language Processing ◽

Sentiment Analysis ◽

Design Process ◽

Language Processing ◽

User Centered Design ◽

The Mean ◽

User Friendly ◽

User Centered

BACKGROUND Sarcopenia, defined as the age-associated loss of muscle mass and strength, can be effectively mitigated through resistance-based physical activity. With compliance at approximately 40% for home-based exercise prescriptions, implementing a remote sensing system would help patients and clinicians to better understand treatment progress and increase compliance. The inclusion of end users in the development of mobile apps for remote-sensing systems can ensure that they are both user friendly and facilitate compliance. With advancements in natural language processing (NLP), there is potential for these methods to be used with data collected through the user-centered design process. OBJECTIVE This study aims to develop a mobile app for a novel device through a user-centered design process with both older adults and clinicians while exploring whether data collected through this process can be used in NLP and sentiment analysis METHODS Through a user-centered design process, we conducted semistructured interviews during the development of a geriatric-friendly Bluetooth-connected resistance exercise band app. We interviewed patients and clinicians at weeks 0, 5, and 10 of the app development. Each semistructured interview consisted of heuristic evaluations, cognitive walkthroughs, and observations. We used the Bing sentiment library for a sentiment analysis of interview transcripts and then applied NLP-based latent Dirichlet allocation (LDA) topic modeling to identify differences and similarities in patient and clinician participant interviews. Sentiment was defined as the sum of positive and negative words (each word with a +1 or −1 value). To assess utility, we used quantitative assessment questionnaires—System Usability Scale (SUS) and Usefulness, Satisfaction, and Ease of use (USE). Finally, we used multivariate linear models—adjusting for age, sex, subject group (clinician vs patient), and development—to explore the association between sentiment analysis and SUS and USE outcomes. RESULTS The mean age of the 22 participants was 68 (SD 14) years, and 17 (77%) were female. The overall mean SUS and USE scores were 66.4 (SD 13.6) and 41.3 (SD 15.2), respectively. Both patients and clinicians provided valuable insights into the needs of older adults when designing and building an app. The mean positive-negative sentiment per sentence was 0.19 (SD 0.21) and 0.47 (SD 0.21) for patient and clinician interviews, respectively. We found a positive association with positive sentiment in an interview and SUS score (ß=1.38; 95% CI 0.37 to 2.39; <i>P</i>=.01). There was no significant association between sentiment and the USE score. The LDA analysis found no overlap between patients and clinicians in the 8 identified topics. CONCLUSIONS Involving patients and clinicians allowed us to design and build an app that is user friendly for older adults while supporting compliance. This is the first analysis using NLP and usability questionnaires in the quantification of user-centered design of technology for older adults.

Download Full-text

Sentiment analysis of public perception on the use of digital contact tracing tools post COVID-19 lockdown (Preprint)

10.2196/preprints.33314 ◽

2021 ◽

Author(s):

Zhilian Huang ◽

Evonne Tay ◽

Dillon Wee ◽

Huiling Guo ◽

Hannah YeeFen Lim ◽

...

Keyword(s):

Sentiment Analysis ◽

Language Processing ◽

Public Perception ◽

Age Groups ◽

Contact Tracing ◽

Cross Sectional Survey ◽

Cross Sectional ◽

The Media ◽

Short Time ◽

Over Time

BACKGROUND Singapore’s national Digital Contact Tracing (DCT) tool—TraceTogether—attained an above 70% uptake by December 2020 after a slew of measures. Sentiment analysis can help policymakers to assess public sentiments on the implementation of new policy measures at a short time but there is a paucity of sentiment analysis studies on the usage of DCT tools. OBJECTIVE We sought to understand the public’s knowledge of and concerns with using TraceTogether, and their preferences for the type of TraceTogether tool. METHODS We conducted a cross-sectional survey one-month post-COVID-19 lockdown at a large public hospital in Singapore from July 2020 through February 2021. Four thousand and ninety-seven respondents aged 21 – 80 were sampled proportionately by gender and four age groups. The open-ended responses were processed and analyzed using natural language processing tools. We manually corrected the language and logic errors and replaced phrases with words available in the “Syuzhet” sentiment library without altering the original meaning of the phrases. The sentiment scores were computed by summing the scores of all the tokens (phrases split into smaller units) in the phrase. Stopwords (prepositions and connectors) were removed, followed by implementing the bag-of-words model to calculate the bigrams and trigrams occurrence in the dataset. Demographic and time filters were applied to segment the responses. RESULTS Respondents’ knowledge of and concerns with TraceTogether changed over time from a focus on “contact tracing” and “Bluetooth activation” in July-August 2020 to “QR code scanning” and “location check-ins” in January-February 2021. Younger males had the highest TraceTogether uptake (60%), while older females had the lowest uptake (23.5%) in the first half of July 2020. This trend was reversed in mid-October after the announcement on mandatory TraceTogether check-ins at public venues. Although their TraceTogether uptake increased over time, older females continued to have lower sentiment scores. The mean sentiment scores were the lowest in January 2021 when the media reported that data collected by TraceTogether were used for criminal investigations. Smartphone apps were initially preferred over tokens, but the preference for the type of the TraceTogether tool equalized over time as tokens became accessible to the whole population. The sentiments on token-related comments became more positive as the preference for tokens increased. CONCLUSIONS The public’s knowledge of and concerns with the use of a mandatory DCT tool varied with the national regulations and public communications over time with the evolution of the COVID-19 pandemic. Effective communications tailored to sub-populations and greater transparency in data handling will help allay public concerns with data misuse and improve trust in the authorities. Having alternative forms of the DCT tool can increase the uptake of and positive sentiments on DCT.

Download Full-text

VADER Natural Language Processing in Market Sentiment Analysis

SSRN Electronic Journal ◽

10.2139/ssrn.3676706 ◽

2020 ◽

Author(s):

Jonathan Seror

Keyword(s):

Natural Language Processing ◽

Natural Language ◽

Sentiment Analysis ◽

Language Processing ◽

Market Sentiment

Download Full-text

Neural methods for effective, efficient, and exposure-aware information retrieval

ACM SIGIR Forum ◽

10.1145/3476415.3476434 ◽

2021 ◽

Vol 55 (1) ◽

pp. 1-2

Author(s):

Bhaskar Mitra

Keyword(s):

Information Retrieval ◽

Language Processing ◽

Large Scale ◽

Web Search ◽

Real Life ◽

Inverted Index ◽

Information Need ◽

Product Model ◽

Performance Improvements ◽

Deep Model

Neural networks with deep architectures have demonstrated significant performance improvements in computer vision, speech recognition, and natural language processing. The challenges in information retrieval (IR), however, are different from these other application areas. A common form of IR involves ranking of documents---or short passages---in response to keyword-based queries. Effective IR systems must deal with query-document vocabulary mismatch problem, by modeling relationships between different query and document terms and how they indicate relevance. Models should also consider lexical matches when the query contains rare terms---such as a person's name or a product model number---not seen during training, and to avoid retrieving semantically related but irrelevant results. In many real-life IR tasks, the retrieval involves extremely large collections---such as the document index of a commercial Web search engine---containing billions of documents. Efficient IR methods should take advantage of specialized IR data structures, such as inverted index, to efficiently retrieve from large collections. Given an information need, the IR system also mediates how much exposure an information artifact receives by deciding whether it should be displayed, and where it should be positioned, among other results. Exposure-aware IR systems may optimize for additional objectives, besides relevance, such as parity of exposure for retrieved items and content publishers. In this thesis, we present novel neural architectures and methods motivated by the specific needs and challenges of IR tasks. We ground our contributions with a detailed survey of the growing body of neural IR literature [Mitra and Craswell, 2018]. Our key contribution towards improving the effectiveness of deep ranking models is developing the Duet principle [Mitra et al., 2017] which emphasizes the importance of incorporating evidence based on both patterns of exact term matches and similarities between learned latent representations of query and document. To efficiently retrieve from large collections, we develop a framework to incorporate query term independence [Mitra et al., 2019] into any arbitrary deep model that enables large-scale precomputation and the use of inverted index for fast retrieval. In the context of stochastic ranking, we further develop optimization strategies for exposure-based objectives [Diaz et al., 2020]. Finally, this dissertation also summarizes our contributions towards benchmarking neural IR models in the presence of large training datasets [Craswell et al., 2019] and explores the application of neural methods to other IR tasks, such as query auto-completion.

Download Full-text

Design of a Variable-Stiffness Compliant Skin for a Morphing Leading Edge

Applied Sciences ◽

10.3390/app11073165 ◽

2021 ◽

Vol 11 (7) ◽

pp. 3165

Author(s):

Zhigang Wang ◽

Yu Yang

Keyword(s):

Composite Laminate ◽

Large Scale ◽

Design Method ◽

Leading Edge ◽

Variable Stiffness ◽

Manufacturing Constraints ◽

Aircraft Wing ◽

Noise Abatement ◽

Civil Aircraft ◽

Final Profile

A seamless and smooth morphing leading edge has remarkable potential for noise abatement and drag reduction of civil aircraft. Variable-stiffness compliant skin based on tailored composite laminate is a concept with great potential for morphing leading edge, but the currently proposed methods have difficulty in taking the manufacturing constraints or layup sequence into account during the optimization process. This paper proposes an innovative two-step design method for a variable-stiffness compliant skin of a morphing leading edge, which includes layup optimization and layup adjustment. The combination of these two steps can not only improve the deformation accuracy of the final profile of the compliant skin but also easily and effectively determine the layup sequence of the composite layup. With the design framework, an optimization model is created for a variable-stiffness compliant skin, and an adjustment method for its layups is presented. Finally, the deformed profiles between the directly optimized layups and the adjusted ones are compared to verify its morphing ability and accuracy. The final results demonstrate that the obtained deforming ability and accuracy are suitable for a large-scale aircraft wing.

Download Full-text

Twitter Sentiment Analysis towards COVID-19 Vaccines in the Philippines Using Naïve Bayes

Information ◽

10.3390/info12050204 ◽

2021 ◽

Vol 12 (5) ◽

pp. 204

Author(s):

Charlyn Villavicencio ◽

Julio Jerison Macrohon ◽

X. Alphonse Inbaraj ◽

Jyh-Horng Jeng ◽

Jer-Guang Hsieh

Keyword(s):

Sentiment Analysis ◽

Language Processing ◽

Data Science ◽

Naive Bayes ◽

The Philippines ◽

Naïve Bayes ◽

Social Networking Site ◽

Bayes Model ◽

The Government ◽

Processing Techniques

A year into the COVID-19 pandemic and one of the longest recorded lockdowns in the world, the Philippines received its first delivery of COVID-19 vaccines on 1 March 2021 through WHO’s COVAX initiative. A month into inoculation of all frontline health professionals and other priority groups, the authors of this study gathered data on the sentiment of Filipinos regarding the Philippine government’s efforts using the social networking site Twitter. Natural language processing techniques were applied to understand the general sentiment, which can help the government in analyzing their response. The sentiments were annotated and trained using the Naïve Bayes model to classify English and Filipino language tweets into positive, neutral, and negative polarities through the RapidMiner data science software. The results yielded an 81.77% accuracy, which outweighs the accuracy of recent sentiment analysis studies using Twitter data from the Philippines.

Download Full-text