A BERT-based Hybrid Short Text Classification Model Incorporating CNN and Attention-based BiGRU

Short text classification is a research focus for natural language processing (NLP), which is widely used in news classification, sentiment analysis, mail filtering and other fields. In recent years, deep learning techniques are applied to text classification and has made some progress. Different from ordinary text classification, short text has the problem of less vocabulary and feature sparsity, which raise higher request for text semantic feature representation. To address this issue, this paper propose a feature fusion framework based on the Bidirectional Encoder Representations from Transformers (BERT). In this hybrid method, BERT is used to train word vector representation. Convolutional neural network (CNN) capture static features. As a supplement, a bi-gated recurrent neural network (BiGRU) is adopted to capture contextual features. Furthermore, an attention mechanism is introduced to assign the weight of salient words. The experimental results confirmed that the proposed model significantly outperforms the other state-of-the-art baseline methods.

Combining Daily Electricity Forecasts Based on DBSCAN and stacking Fusion

Background: Modern upgrades of power grids and a rapidly expanding economy complexify the uncertainties of electricity demand. Objective: The objective of the study is to have a more precise prediction on the demand side, which is beneficial in affirming the stable operation of the power system. Methods: This paper presents a combined electricity forecasting method based on the users clustering and stacking ensemble learning to mine underlying properties of different individual consumers. The preprocessed electricity consumption profiles are inputted into the DBSCAN clustering algorithm to obtain the clusters. The alternative models are tailored for different clusters in the stacking fusion framework for training and testing. Result: Experimental results on the operating data of Shandong Power Grid show that the proposed method has higher prediction accuracy and better generalization ability. Conclusion: The framework is of great significance for improving the level of power supply service.