Deep Learning for text in limted data settings

2020 ◽  
Author(s):  
Pathikkumar Patel ◽  
Bhargav Lad ◽  
Jinan Fiaidhi
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Transfer Learning ◽  
Text Classification ◽  
State Of The Art ◽  
Time Series Forecasting ◽  
Text Data ◽  
Performance Levels

During the last few years, RNN models have been extensively used and they have proven to be better for sequence and text data. RNNs have achieved state-of-the-art performance levels in several applications such as text classification, sequence to sequence modelling and time series forecasting. In this article we will review different Machine Learning and Deep Learning based approaches for text data and look at the results obtained from these methods. This work also explores the use of transfer learning in NLP and how it affects the performance of models on a specific application of sentiment analysis.

scholarly journals Deep Learning for text in limted data settings

2020 ◽  
Author(s):  
Pathikkumar Patel ◽  
Bhargav Lad ◽  
Jinan Fiaidhi
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Transfer Learning ◽  
Text Classification ◽  
State Of The Art ◽  
Time Series Forecasting ◽  
Text Data ◽  
Performance Levels

During the last few years, RNN models have been extensively used and they have proven to be better for sequence and text data. RNNs have achieved state-of-the-art performance levels in several applications such as text classification, sequence to sequence modelling and time series forecasting. In this article we will review different Machine Learning and Deep Learning based approaches for text data and look at the results obtained from these methods. This work also explores the use of transfer learning in NLP and how it affects the performance of models on a specific application of sentiment analysis.

scholarly journals A Tweet Sentiment Classification Approach Using a Hybrid Stacked Ensemble Technique

Information ◽  
2021 ◽  
Vol 12 (9) ◽  
pp. 374
Author(s):  
Babacar Gaye ◽  
Dezheng Zhang ◽  
Aziguli Wulamu
Keyword(s):  
Machine Learning ◽  
Logistic Regression ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Short Term Memory ◽  
State Of The Art ◽  
Accuracy Score ◽  
Learning Models ◽  
Proposed Model

With the extensive availability of social media platforms, Twitter has become a significant tool for the acquisition of peoples’ views, opinions, attitudes, and emotions towards certain entities. Within this frame of reference, sentiment analysis of tweets has become one of the most fascinating research areas in the field of natural language processing. A variety of techniques have been devised for sentiment analysis, but there is still room for improvement where the accuracy and efficacy of the system are concerned. This study proposes a novel approach that exploits the advantages of the lexical dictionary, machine learning, and deep learning classifiers. We classified the tweets based on the sentiments extracted by TextBlob using a stacked ensemble of three long short-term memory (LSTM) as base classifiers and logistic regression (LR) as a meta classifier. The proposed model proved to be effective and time-saving since it does not require feature extraction, as LSTM extracts features without any human intervention. We also compared our proposed approach with conventional machine learning models such as logistic regression, AdaBoost, and random forest. We also included state-of-the-art deep learning models in comparison with the proposed model. Experiments were conducted on the sentiment140 dataset and were evaluated in terms of accuracy, precision, recall, and F1 Score. Empirical results showed that our proposed approach manifested state-of-the-art results by achieving an accuracy score of 99%.

scholarly journals Comparative Assessment to Predict and Forecast Water-Cooled Chiller Power Consumption Using Machine Learning and Deep Learning Algorithms

2021 ◽  
Vol 13 (2) ◽  
pp. 744
Author(s):  
Elsa Chaerun Nisa ◽  
Yean-Der Kuan
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Power Consumption ◽  
Energy Demand ◽  
Prediction Models ◽  
Short Term Memory ◽  
Time Series Forecasting ◽  
Experimental Result ◽  
Forecasting Model

Over the last few decades, total energy consumption has increased while energy resources remain limited. Energy demand management is crucial for this reason. To solve this problem, predicting and forecasting water-cooled chiller power consumption using machine learning and deep learning are presented. The prediction models adopted are thermodynamic model and multi-layer perceptron (MLP), while the time-series forecasting models adopted are MLP, one-dimensional convolutional neural network (1D-CNN), and long short-term memory (LSTM). Each group of models is compared. The best model in each group is then selected for implementation. The data were collected every minute from an academic building at one of the universities in Taiwan. The experimental result demonstrates that the best prediction model is the MLP with 0.971 of determination (R2), 0.743 kW of mean absolute error (MAE), and 1.157 kW of root mean square error (RMSE). The time-series forecasting model trained every day for three consecutive days using new data to forecast the next minute of power consumption. The best time-series forecasting model is LSTM with 0.994 of R2, 0.233 kW of MAE, and 1.415 kW of RMSE. The models selected for both MLP and LSTM indicated very close predictive and forecasting values to the actual value.

Modern technologies of deep learning for forecasting time series

2021 ◽  
pp. 95-105
Author(s):  
А.И. Сотников
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Time Series ◽  
Big Data ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
High Accuracy ◽  
Time Series Forecasting ◽  
Machine Learning Methods ◽  
Modern Technologies

Прогнозирование временных рядов стало очень интенсивной областью исследований, число которых в последние годы даже увеличивается. Глубокие нейронные сети доказали свою эффективность и достигают высокой точности во многих областях применения. По этим причинам в настоящее время они являются одним из наиболее широко используемых методов машинного обучения для решения проблем, связанных с большими данными. Time series forecasting has become a very intensive area of research, the number of which has even increased in recent years. Deep neural networks have been proven to be effective and achieve high accuracy in many applications. For these reasons, they are currently one of the most widely used machine learning methods for solving big data problems.

Survey on Classic and Latest Textual Sentiment Analysis Articles and Techniques

2019 ◽  
Vol 18 (04) ◽  
pp. 1243-1287 ◽  
Author(s):  
Yong Shi ◽  
Luyao Zhu ◽  
Wei Li ◽  
Kun Guo ◽  
Yuanchun Zheng
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
State Of The Art ◽  
Heterogeneous Data ◽  
Research Trends ◽  
Useful Knowledge ◽  
Entire Process ◽  
Different Types ◽  
Pros And Cons

Text is a typical example of unstructured and heterogeneous data in which massive useful knowledge is embedded. Sentiment analysis is used to analyze and predict sentiment polarities of the text. This paper provides a survey and gives comparative analyses of the latest articles and techniques pertaining to lexicon-based, traditional machine learning-based, deep learning-based, and hybrid sentiment analysis approaches. These approaches have their own superiority and get the state-of-the-art results on diverse sentiment analysis tasks. Besides, a brief sentiment analysis example in the tourism domain is displayed, illustrating the entire process of sentiment analysis. Furthermore, we create a large table to compare the pros and cons of different types of approaches, and discuss some insights with respect to research trends. In addition, a lot of important sentiment analysis datasets are summarized in this survey.

scholarly journals A Deep Learning BiLSTM Encoding-Decoding Model for COVID-19 Pandemic Spread Forecasting

2021 ◽  
Vol 5 (4) ◽  
pp. 175
Author(s):  
Ahmed I. Shahin ◽  
Sultan Almotairi
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Time Series Forecasting ◽  
Machine Learning Algorithms ◽  
Forecasting Model ◽  
Learning Models ◽  
Forecasting Models ◽  
Proposed Model ◽  
Death Cases

The COVID-19 pandemic has widely spread with an increasing infection rate through more than 200 countries. The governments of the world need to record the confirmed infectious, recovered, and death cases for the present state and predict the cases. In favor of future case prediction, governments can impose opening and closing procedures to save human lives by slowing down the pandemic progression spread. There are several forecasting models for pandemic time series based on statistical processing and machine learning algorithms. Deep learning has been proven as an excellent tool for time series forecasting problems. This paper proposes a deep learning time-series prediction model to forecast the confirmed, recovered, and death cases. Our proposed network is based on an encoding–decoding deep learning network. Moreover, we optimize the selection of our proposed network hyper-parameters. Our proposed forecasting model was applied in Saudi Arabia. Then, we applied the proposed model to other countries. Our study covers two categories of countries that have witnessed different spread waves this year. During our experiments, we compared our proposed model and the other time-series forecasting models, which totaled fifteen prediction models: three statistical models, three deep learning models, seven machine learning models, and one prophet model. Our proposed forecasting model accuracy was assessed using several statistical evaluation criteria. It achieved the lowest error values and achieved the highest R-squared value of 0.99. Our proposed model may help policymakers to improve the pandemic spread control, and our method can be generalized for other time series forecasting tasks.

scholarly journals An Analysis Method for Interpretability of CNN Text Classification Model

2020 ◽  
Vol 12 (12) ◽  
pp. 228
Author(s):  
Peng Ce ◽  
Bao Tie
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Text Classification ◽  
Sequence Data ◽  
Classification Model ◽  
Learning Technology ◽  
Analysis Method ◽  
Text Data ◽  
Data Set ◽  
Backtracking Analysis

With continuous development of artificial intelligence, text classification has gradually changed from a knowledge-based method to a method based on statistics and machine learning. Among them, it is a very important and efficient way to classify text based on the convolutional neural network (CNN) model. Text data are a kind of sequence data, while time sequentiality of the general text data is relatively weak, so text classification is usually less relevant to the sequential structure of the full text. Therefore, CNN-based text classification has gradually become a research hotspot when dealing with issues of text classification. For machine learning, especially deep learning, model interpretability has increasingly become the focus of academic research and industrial applications, and also become a key issue for further development and application of deep learning technology. Therefore, we recommend using the backtracking analysis method to conduct in-depth research on deep learning models. This paper proposes an analysis method for interpretability of a CNN text classification model. The method proposed by us can perform multi-angle analysis on the discriminant results of multi-classified text and multi-label classification tasks through backtracking analysis on model prediction results. Finally, the analysis results of the model can be displayed using visualization technology from multiple dimensions based on interpretability. The representative data set IMDB (Internet Movie Database) in text classification is verified by examples, and the results show that the model can be effectively analyzed when using our method.

scholarly journals A Framework for Sentiment Analysis of Telugu Tweets

2020 ◽  
Vol 9 (6) ◽  
pp. 523-525
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Social Media ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Recurrent Neural Network ◽  
English Language ◽  
Research Work ◽  
Learning Approaches ◽  
Text Data

Now a day Social Media like Facebook, twitter and Instagram is major Sources for people to share their emotions based on the current situations in society. By knowing the interesting patterns in it, a government/appropriate person for that situation can take good and useful decisions. Sentiment analysis is a method where people can extract the useful information from the text like the emotions (happy, sad, and neutral) of people. Much research work was been underdoing in the area of sentiment analysis. Among that work the Machine learning and Deep learning approaches plays a maximum role. Existing works on sentiment analysis is going in the English language. In this paper, proposed a novel framework that specifically designed to do sentiment analysis of the text data, that available in the telugu language. The proposed framework was integrated with the word embedding model Word2Vec, language translator and deep learning approaches like Recurrent Neural Network and Navie base algorithms to collect and analyse the sentiment in tweeter data that present in telugu language. The results shows effective in terms of accuracy, precision and specificity.

A new dam structural response estimation paradigm powered by deep learning and transfer learning techniques

2021 ◽  
pp. 147592172110097
Author(s):  
Yangtao Li ◽  
Tengfei Bao ◽  
Zhixin Gao ◽  
Xiaosong Shu ◽  
Kang Zhang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Transfer Learning ◽  
Health Monitoring ◽  
Structural Response ◽  
Data Driven ◽  
Structural Health ◽  
Learning Techniques

With the rapid development of information and communication techniques, dam structural health assessment based on data collected from structural health monitoring systems has become a trend. This allows for applying data-driven methods for dam safety analysis. However, data-driven models in most related literature are statistical and shallow machine learning models, which cannot capture the time series patterns or learn from long-term dependencies of dam structural response time series. Furthermore, the effectiveness and applicability of these models are only validated in a small data set and part of monitoring points in a dam structural health monitoring system. To address the problems, this article proposes a new modeling paradigm based on various deep learning and transfer learning techniques. The paradigm utilizes one-dimensional convolutional neural networks to extract the inherent features from dam structural response–related environmental quantity monitoring data. Then bidirectional gated recurrent unit with a self-attention mechanism is used to learn from long-term dependencies, and transfer learning is utilized to transfer knowledge learned from the typical monitoring point to the others. The proposed paradigm integrates the powerful modeling capability of deep learning networks and the flexible transferability of transfer learning. Rather than traditional models that rely on experience for feature selection, the proposed deep learning–based paradigm directly utilizes environmental monitoring time series as inputs to accurately estimate dam structural response changes. A high arch dam in long-term service is selected as the case study, and three monitoring items, including dam displacement, crack opening displacement, and seepage are used as the research objects. The experimental results show that the proposed paradigm outperforms conventional and shallow machine learning–based methods in all 41 tested monitoring points, which indicates that the proposed paradigm is capable of dealing with dam structural response estimation with high accuracy and robustness.

Sign in / Sign up

Export Citation Format

Share Document