scholarly journals Event Monitoring and Intelligence Gathering Using Twitter Based Real-Time Event Summarization and Pre-Trained Model Techniques

2021 ◽  
Vol 11 (22) ◽  
pp. 10596
Author(s):  
Chung-Hong Lee ◽  
Hsin-Chang Yang ◽  
Yenming J. Chen ◽  
Yung-Lin Chuang
Keyword(s):  
Real Time ◽  
Data Science ◽  
Clustering Algorithm ◽  
Language Models ◽  
Event Monitoring ◽  
Data Set ◽  
Intelligence Gathering ◽  
Twitter Data ◽  
The Government

Recently, an emerging application field through Twitter messages and algorithmic computation to detect real-time world events has become a new paradigm in the field of data science applications. During a high-impact event, people may want to know the latest information about the development of the event because they want to better understand the situation and possible trends of the event for making decisions. However, often in emergencies, the government or enterprises are usually unable to notify people in time for early warning and avoiding risks. A sensible solution is to integrate real-time event monitoring and intelligence gathering functions into their decision support system. Such a system can provide real-time event summaries, which are updated whenever important new events are detected. Therefore, in this work, we combine a developed Twitter-based real-time event detection algorithm with pre-trained language models for summarizing emergent events. We used an online text-stream clustering algorithm and self-adaptive method developed to gather the Twitter data for detection of emerging events. Subsequently we used the Xsum data set with a pre-trained language model, namely T5 model, to train the summarization model. The Rouge metrics were used to compare the summary performance of various models. Subsequently, we started to use the trained model to summarize the incoming Twitter data set for experimentation. In particular, in this work, we provide a real-world case study, namely the COVID-19 pandemic event, to verify the applicability of the proposed method. Finally, we conducted a survey on the example resulting summaries with human judges for quality assessment of generated summaries. From the case study and experimental results, we have demonstrated that our summarization method provides users with a feasible method to quickly understand the updates in the specific event intelligence based on the real-time summary of the event story.

Twitter users exhibited coping behaviours during the COVID-19 lockdown: an analysis of tweets using mixed methods

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Ruchi Mittal ◽  
Wasim Ahmed ◽  
Amit Mittal ◽  
Ishan Aggarwal
Keyword(s):  
Data Analysis ◽  
The Body ◽  
Data Set ◽  
Content Type ◽  
Twitter Data ◽  
Long Run ◽  
The World ◽  
Twitter Users ◽  
The Government ◽  
Using Data

Purpose Using data from Twitter, the purpose of this paper is to assess the coping behaviour and reactions of social media users in response to the initial days of the COVID-19-related lockdown in different parts of the world. Design/methodology/approach This study follows the quasi-inductive approach which allows the development of pre-categories from other theories before the sampling and coding processes begin, for use in those processes. Data was extracted using relevant keywords from Twitter, and a sample was drawn from the Twitter data set to ensure the data is more manageable from a qualitative research standpoint and that meaningful interpretations can be drawn from the data analysis results. The data analysis is discussed in two parts: extraction and classification of data from Twitter using automated sentiment analysis; and qualitative data analysis of a smaller Twitter data sample. Findings This study found that during the lockdown the majority of users on Twitter shared positive opinions towards the lockdown. The results also found that people are keeping themselves engaged and entertained. Governments around the world have also gained support from Twitter users. This is despite the hardships being faced by citizens. The authors also found a number of users expressing negative sentiments. The results also found that several users on Twitter were fence-sitters and their opinions and emotions could swing either way depending on how the pandemic progresses and what action is taken by governments around the world. Research limitations/implications The authors add to the body of literature that has examined Twitter discussions around H1N1 using in-depth qualitative methods and conspiracy theories around COVID-19. In the long run, the government can help citizens develop routines that help the community adapt to a new dangerous environment – this has very effectively been shown in the context of wildfires in the context of disaster management. In the context of this research, the dominance of the positive themes within tweets is promising for policymakers and governments around the world. However, sentiments may wish to be monitored going forward as large-spikes in negative sentiment may highlight lockdown-fatigue. Social implications The psychology of humans during a pandemic can have a profound impact on how COVID-19 shapes up, and this shall also include how people behave with other people and with the larger environment. Lockdowns are the opposite of what societies strive to achieve, i.e. socializing. Originality/value This study is based on original Twitter data collected during the initial days of the COVID-19-induced lockdown. The topic of “lockdowns” and the “COVID-19” pandemic have not been studied together thus far. This study is highly topical.

scholarly journals Classification Connection of Twitter Data using K-Means Clustering

2019 ◽  
Vol 8 (6S4) ◽  
pp. 14-22
Keyword(s):  
Social Media ◽  
Real Time ◽  
Clustering Methods ◽  
Time Analysis ◽  
Data Set ◽  
Dynamic Data ◽  
Real Time Analysis ◽  
Twitter Data ◽  
Social Media Platforms ◽  
Insight Into

The rise of social media platforms like Twitter and the increasing adoption by people in order to stay connected provide a large source of data to perform analysis based on the various trends, events and even various personalities. Such analysis also provides insight into a person’s likes and inclinations in real time independent of the data size. Several techniques have been created to retrieve such data however the most efficient technique is clustering. This paper provides an overview of the algorithms of the various clustering methods as well as looking at their efficiency in determining trending information. The clustered data may be further classified by topics for real time analysis on a large dynamic data set. In this paper, data classification is performed and analyzed for flaws followed by another classification on the same data set.

Unleashing analytics to reduce electricity consumption using incremental clustering algorithm

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Archana Yashodip Chaudhari ◽  
Preeti Mulay
Keyword(s):  
Real Time ◽  
Clustering Algorithm ◽  
Electricity Consumption ◽  
Incremental Clustering ◽  
Science Data ◽  
Load Curve ◽  
Data Set ◽  
Content Type ◽  
Validity Indices ◽  
Reduce Electricity Consumption

Purpose To reduce the electricity consumption in our homes, a first step is to make the user aware of it. Reading a meter once in a month is not enough, instead, it requires real-time meter reading. Smart electricity meter (SEM) is capable of providing a quick and exact meter reading in real-time at regular time intervals. SEM generates a considerable amount of household electricity consumption data in an incremental manner. However, such data has embedded load patterns and hidden information to extract and learn consumer behavior. The extracted load patterns from data clustering should be updated because consumer behaviors may be changed over time. The purpose of this study is to update the new clustering results based on the old data rather than to re-cluster all of the data from scratch. Design/methodology/approach This paper proposes an incremental clustering with nearness factor (ICNF) algorithm to update load patterns without overall daily load curve clustering. Findings Extensive experiments are implemented on real-world SEM data of Irish Social Science Data Archive (Ireland) data set. The results are evaluated by both accuracy measures and clustering validity indices, which indicate that proposed method is useful for using the enormous amount of smart meter data to understand customers’ electricity consumption behaviors. Originality/value ICNF can provide an efficient response for electricity consumption patterns analysis to end consumers via SEMs.

Entropy-Based City Tunnel Real-Time Traffic Incident Detection Clustering Algorithm

2013 ◽  
Vol 380-384 ◽  
pp. 753-756
Author(s):  
Xiao Feng Li ◽  
Wei Wei Gao ◽  
Xue Mei Wang
Keyword(s):  
Real Time ◽  
Clustering Algorithm ◽  
Spatial Clustering ◽  
Entropy Change ◽  
Practical Significance ◽  
Incident Detection ◽  
Data Set ◽  
Traffic Incident ◽  
Real Time Traffic ◽  
The City

The use of spatial clustering technology has important practical significance to obtain useful information. According to the characteristics of city tunnel real-time traffic ,then, put forward ECRT (Entropy-based City Tunnel Real-time), the object associated with the city tunnel as real-time traffic properties to calculate the entropy of information between the city tunnel, based on information entropy change to achieve real-time traffic urban tunnel clustering. Algorithm used in the actual data set ECRT test. The results showed that the algorithm ECRT is effective.

scholarly journals Perceptions of wheat producers towards risk and risk management strategies: A case study from Turkey

2019 ◽  
Vol 49 (12) ◽  
Author(s):  
Seyit Hayran
Keyword(s):  
Risk Management ◽  
Management Strategies ◽  
Policy Instruments ◽  
Data Set ◽  
Wheat Market ◽  
Risk Management Strategies ◽  
Risk Sources ◽  
The Government ◽  
The Impact

ABSTRACT: In this study, risk perception of wheat producers in Turkey was examined based on a case study conducted in Bitlis Province. The data set used in the study was obtained from 157 farmers randomly. Factor analysis was employed to classify risk sources and management strategies, and then multiple regression was used to investigate the relationship between farmers perceptions and some characteristic. Results of this study have shown that economic-based risks were perceived more strongly by farmers. Farmers’ also used more than one risk management strategy to minimize the impact of the risks they face. So, in order to ensure social and economic sustainability and predictability in wheat production and wheat market, the government should be considered preventive policy instruments and interventions to prevent fluctuations in input and output prices.

Text Mining and Real-Time Analytics of Twitter Data: A Case Study of Australian Hay Fever Prediction

2018 ◽  
pp. 134-145 ◽  
Author(s):  
Sudha Subramani ◽  
Sandra Michalska ◽  
Hua Wang ◽  
Frank Whittaker ◽  
Benjamin Heyward
Keyword(s):  
Text Mining ◽  
Real Time ◽  
Hay Fever ◽  
Twitter Data

scholarly journals A real-time big data sentiment analysis for iraqi tweets using spark streaming

2020 ◽  
Vol 9 (4) ◽  
pp. 1411-1419
Author(s):  
Nashwan Dheyaa Zaki ◽  
Nada Yousif Hashim ◽  
Yasmin Makki Mohialden ◽  
Mostafa Abdulghafoor Mohammed ◽  
Tole Sutikno ◽  
...  
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Sentiment Analysis ◽  
Real Time ◽  
Data Science ◽  
Opinion Mining ◽  
Data Streaming ◽  
Machine Learning Applications ◽  
Learning Research

The scale of data streaming in social networks, such as Twitter, is increasing exponentially. Twitter is one of the most important and suitable big data sources for machine learning research in terms of analysis, prediction, extract knowledge, and opinions. People use Twitter platform daily to express their opinion which is a fundamental fact that influence their behaviors. In recent years, the flow of Iraqi dialect has been increased, especially on the Twitter platform. Sentiment analysis for different dialects and opinion mining has become a hot topic in data science researches. In this paper, we will attempt to develop a real-time analytic model for sentiment analysis and opinion mining to Iraqi tweets using spark streaming, also create a dataset for researcher in this field. The Twitter handle Bassam AlRawi is the case study here. The new method is more suitable in the current day machine learning applications and fast online prediction. 

On the identification of Electron Diffusion Regions at the magnetopause with an AI approach

2020 ◽  
Author(s):  
Quentin Lenouvel ◽  
Vincent Génot ◽  
Philippe Garnier ◽  
Sergio Toledo-Redondo ◽  
Benoît Lavraud ◽  
...  
Keyword(s):  
Machine Learning ◽  
Data Science ◽  
Phase 1 ◽  
Diffusion Region ◽  
Data Set ◽  
Electron Diffusion ◽  
Training Techniques ◽  
Research Fields ◽  
Dayside Magnetopause

<div> <div> <div> <div> <div> <div> <div> <div> <div> <div> <p><strong></strong></p> <p>MMS has already been producing a very large dataset with invaluable information about how the solar wind and the Earth's magnetosphere interact. However, it remains challenging to process all these new data and convert it into scientific knowledge, the ultimate goal of the mission. Data science and machine learning are nowadays a very powerful and successful technology that is employed to many applied and research fields. During this presentation, I shall discuss the tentative use of machine learning for the automatic detection and classification of plasma regions, relevant to the study of magnetic reconnection in the MMS data set, with a focus on the critical but poorly understood electron diffusion region (EDR) at the Earth's dayside magnetopause. We make use of the EDR database and the plasma regions nearby that has been identified by the MMS community and compiled by Webster et al. (2018) as well as the Magnetopause crossings database compiled by the ISSI team, to train a neural network using supervised training techniques. I shall present a list of new EDR candidates found during the phase 1 of MMS and do a case study of some of the strong candidates.</p> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div>

scholarly journals Dynamic Analytics and Forecasting Model for Covid-19 Using Machine Learning Algorithms

Webology ◽  
2021 ◽  
Vol 18 (05) ◽  
pp. 1212-1225
Author(s):  
Siva C ◽  
Maheshwari K.G ◽  
Nalinipriya G ◽  
Priscilla Mary J
Keyword(s):  
Machine Learning ◽  
Real Time ◽  
Hierarchical Clustering ◽  
Clustering Algorithm ◽  
Prediction Models ◽  
Vital Role ◽  
Machine Learning Algorithms ◽  
World Health ◽  
Medical Emergency ◽  
Data Set

In our day to day life, the availability of correctly labelled data as well as handling of categorical data are mostly acknowledged as two main challenges in dynamic analysis. Therefore, clustering techniques are applied on unlabelled data to group them in accordance with the homogeneity. There are many prediction methods that are being popularly used in handling forecasting problems in real time environment. The outbreak of coronavirus disease (COVID19)-2019 creates the need for a medical emergency of worldwide concern with a rapidly high danger of open out and strike the entire world. Recently, the ML prediction models were used in many real time applications which necessitate the identification and categorization for real time environment. In medical field Prediction models are vital role to obtain observations of spread and significances of infectious diseases. Machine learning related forecasting mechanisms have showed their importance to develop the decision making on the upcoming course of actions. The K-means algorithm and hierarchy were applied directly on the renewed dataset using R programming language to create the covid patient cluster. Confirmed Covid patients count are passed to Prophet package, then the prophet model has been created. This forecasts model predicts the future covid count, which is essential for the clinical and healthcare leaders to make the appropriate measures in advance. The results of the experiments indicate that the quality of Hierarchical clustering outperforms than the K-Means clustering algorithm in the structured structured dataset. Thus, the prediction model also used to support model predictions help for the officials to take timely actions and make decisions to contain the COVID-19 dilemma. This work concludes Hierarchical clustering algorithm is the best model for clustering the covid data set obtained from world health organization (WHO).

Sign in / Sign up

Export Citation Format

Share Document