Tracing Topic Transitions with Temporal Graph Clusters

Twitter serves as a data source for many Natural Language Processing (NLP) tasks. It can be challenging to identify topics on Twitter due to continuous updating data stream. In this paper, we present an unsupervised graph based framework to identify the evolution of sub-topics within two weeks of real-world Twitter data. We first employ a Markov Clustering Algorithm (MCL) with a node removal method to identify optimal graph clusters from temporal Graph-of-Words (GoW). Subsequently, we model the clustering transitions between the temporal graphs to identify the topic evolution. Finally, the transition flows generated from both computational approach and human annotations are compared to ensure the validity of our framework.

Download Full-text

Effective Clustering Algorithm for Probabilistic Data Stream

Journal of Software ◽

10.3724/sp.j.1001.2009.03303 ◽

2010 ◽

Vol 20 (5) ◽

pp. 1313-1328 ◽

Cited By ~ 8

Author(s):

Dong-Bo DAI ◽

Gang ZHAO ◽

Sheng-Li SUN

Keyword(s):

Data Stream ◽

Clustering Algorithm ◽

Probabilistic Data

Download Full-text

Similarity detection of English text and teaching evaluation based on improved TCUSS clustering algorithm

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-189576 ◽

2020 ◽

pp. 1-11

Author(s):

Yu Wang

Keyword(s):

Language Processing ◽

Calculation Method ◽

Clustering Algorithm ◽

Research Result ◽

Structural Features ◽

English Text ◽

Support Vector ◽

Similarity Calculation ◽

Other Information ◽

Calculation Task

The semantic similarity calculation task of English text has important influence on other fields of natural language processing and has high research value and application prospect. At present, research on the similarity calculation of short texts has achieved good results, but the research result on long text sets is still poor. This paper proposes a similarity calculation method that combines planar features with structured features and uses support vector regression models. Moreover, this paper uses PST and PDT to represent the syntax, semantics and other information of the text. In addition, through the two structural features suitable for text similarity calculation, this paper proposes a similarity calculation method combining structural features with Tree-LSTM model. Experiments show that this method provides a new idea for interest network extraction.

Download Full-text

Nonprobability Sampling and Twitter

Social Science Computer Review ◽

10.1177/0894439317709431 ◽

2017 ◽

Vol 36 (2) ◽

pp. 195-211 ◽

Cited By ~ 9

Author(s):

Patrick Rafail

Keyword(s):

Social Sciences ◽

Application Programming Interface ◽

Sampling Strategies ◽

Twitter Data ◽

Large Databases ◽

The Social ◽

Application Programming ◽

Data Source ◽

Twitter Activity ◽

Programming Interface

Twitter data are widely used in the social sciences. The Twitter Application Programming Interface (API) allows researchers to build large databases of user activity efficiently. Despite the potential of Twitter as a data source, less attention has been paid to issues of sampling, and in particular, the implications of different sampling strategies on overall data quality. This research proposes a set of conceptual distinctions between four types of populations that emerge when analyzing Twitter data and suggests sampling strategies that facilitate more comprehensive data collection from the Twitter API. Using three applications drawn from large databases of Twitter activity, this research also compares the results from the proposed sampling strategies, which provide defensible representations of the population of activity, to those collected with more frequently used hashtag samples. The results suggest that hashtag samples misrepresent important aspects of Twitter activity and may lead researchers to erroneous conclusions.

Download Full-text

Adaptation in clustering algorithm by algorithm output granularity for mobile data stream mining

International Conference on Information Communication and Embedded Systems (ICICES2014) ◽

10.1109/icices.2014.7033790 ◽

2014 ◽

Author(s):

Rahul Wasule ◽

R. A. Fadnavis

Keyword(s):

Data Stream ◽

Clustering Algorithm ◽

Data Stream Mining ◽

Stream Mining ◽

Mobile Data

Download Full-text

A Study of the Effects of the COVID-19 Pandemic on the Experience of Back Pain Reported on Twitter® in the United States: A Natural Language Processing Approach

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph18094543 ◽

2021 ◽

Vol 18 (9) ◽

pp. 4543

Author(s):

Krzysztof Fiok ◽

Waldemar Karwowski ◽

Edgar Gutierrez ◽

Maham Saeidi ◽

Awad M. Aljuaid ◽

...

Keyword(s):

United States ◽

Natural Language Processing ◽

Back Pain ◽

Natural Language ◽

Language Processing ◽

The United States ◽

Daily Routine ◽

Body Movements ◽

Data Source ◽

Twitter Users

The COVID-19 pandemic has changed our lifestyles, habits, and daily routine. Some of the impacts of COVID-19 have been widely reported already. However, many effects of the COVID-19 pandemic are still to be discovered. The main objective of this study was to assess the changes in the frequency of reported physical back pain complaints reported during the COVID-19 pandemic. In contrast to other published studies, we target the general population using Twitter as a data source. Specifically, we aim to investigate differences in the number of back pain complaints between the pre-pandemic and during the pandemic. A total of 53,234 and 78,559 tweets were analyzed for November 2019 and November 2020, respectively. Because Twitter users do not always complain explicitly when they tweet about the experience of back pain, we have designed an intelligent filter based on natural language processing (NLP) to automatically classify the examined tweets into the back pain complaining class and other tweets. Analysis of filtered tweets indicated an 84% increase in the back pain complaints reported in November 2020 compared to November 2019. These results might indicate significant changes in lifestyle during the COVID-19 pandemic, including restrictions in daily body movements and reduced exposure to routine physical exercise.

Download Full-text

A heterogeneous parallel implementation of the Markov clustering algorithm for large-scale biological networks on distributed CPU–GPU clusters

The Journal of Supercomputing ◽

10.1007/s11227-021-04204-6 ◽

2022 ◽

Author(s):

You Fu ◽

Wei Zhou

Keyword(s):

Biological Networks ◽

Large Scale ◽

Clustering Algorithm ◽

Parallel Implementation ◽

Gpu Clusters ◽

Markov Clustering

Download Full-text

Event Monitoring and Intelligence Gathering Using Twitter Based Real-Time Event Summarization and Pre-Trained Model Techniques

Applied Sciences ◽

10.3390/app112210596 ◽

2021 ◽

Vol 11 (22) ◽

pp. 10596

Author(s):

Chung-Hong Lee ◽

Hsin-Chang Yang ◽

Yenming J. Chen ◽

Yung-Lin Chuang

Keyword(s):

Real Time ◽

Data Science ◽

Clustering Algorithm ◽

Language Models ◽

Event Monitoring ◽

Data Set ◽

Intelligence Gathering ◽

Twitter Data ◽

The Government

Recently, an emerging application field through Twitter messages and algorithmic computation to detect real-time world events has become a new paradigm in the field of data science applications. During a high-impact event, people may want to know the latest information about the development of the event because they want to better understand the situation and possible trends of the event for making decisions. However, often in emergencies, the government or enterprises are usually unable to notify people in time for early warning and avoiding risks. A sensible solution is to integrate real-time event monitoring and intelligence gathering functions into their decision support system. Such a system can provide real-time event summaries, which are updated whenever important new events are detected. Therefore, in this work, we combine a developed Twitter-based real-time event detection algorithm with pre-trained language models for summarizing emergent events. We used an online text-stream clustering algorithm and self-adaptive method developed to gather the Twitter data for detection of emerging events. Subsequently we used the Xsum data set with a pre-trained language model, namely T5 model, to train the summarization model. The Rouge metrics were used to compare the summary performance of various models. Subsequently, we started to use the trained model to summarize the incoming Twitter data set for experimentation. In particular, in this work, we provide a real-world case study, namely the COVID-19 pandemic event, to verify the applicability of the proposed method. Finally, we conducted a survey on the example resulting summaries with human judges for quality assessment of generated summaries. From the case study and experimental results, we have demonstrated that our summarization method provides users with a feasible method to quickly understand the updates in the specific event intelligence based on the real-time summary of the event story.

Download Full-text

Using social media data to map the areas most affected by ISIS in Syria

Proceedings of the International conference “InterCarto/InterGIS” ◽

10.35595/2414-9179-2020-1-26-464-470 ◽

2020 ◽

Vol 26 (1) ◽

pp. 464-470

Author(s):

Mohamad Hasan

Keyword(s):

Social Media ◽

Language Processing ◽

Geographical Information ◽

Data Mapping ◽

Islamic State ◽

Social Media Data ◽

The Social ◽

Mapping Process ◽

Data Source ◽

Media Data

This paper presents a model to collect, save, geocode, and analyze social media data. The model is used to collect and process the social media data concerned with the ISIS terrorist group (the Islamic State in Iraq and Syria), and to map the areas in Syria most affected by ISIS accordingly to the social media data. Mapping process is assumed automated compilation of a density map for the geocoded tweets. Data mined from social media (e.g., Twitter and Facebook) is recognized as dynamic and easily accessible resources that can be used as a data source in spatial analysis and geographical information system. Social media data can be represented as a topic data and geocoding data basing on the text of the mined from social media and processed using Natural Language Processing (NLP) methods. NLP is a subdomain of artificial intelligence concerned with the programming computers to analyze natural human language and texts. NLP allows identifying words used as an initial data by developed geocoding algorithm. In this study, identifying the needed words using NLP was done using two corpora. First corpus contained the names of populated places in Syria. The second corpus was composed in result of statistical analysis of the number of tweets and picking the words that have a location meaning (i.e., schools, temples, etc.). After identifying the words, the algorithm used Google Maps geocoding API in order to obtain the coordinates for posts.

Download Full-text