Multi-task prediction model based on ConvLSTM and encoder-decoder

2021 ◽  
Vol 25 (2) ◽  
pp. 359-382
Author(s):  
Tao Luo ◽  
Xudong Cao ◽  
Jin Li ◽  
Kun Dong ◽  
Rui Zhang ◽  
...  
Keyword(s):  
Time Series ◽  
Northwest China ◽  
Attention Mechanism ◽  
Test Results ◽  
Nonlinear Characteristics ◽  
Multi Scale ◽  
Model Based ◽  
Index Value ◽  
Fully Connected ◽  
Energy Network

The energy load data in the micro-energy network are a time series with sequential and nonlinear characteristics. This paper proposes a model based on the encode-decode architecture and ConvLSTM for multi-scale prediction of multi-energy loads in the micro-energy network. We apply ConvLSTM, LSTM, attention mechanism and multi-task learning concepts to construct a model specifically for processing the energy load forecasting of the micro-energy network. In this paper, ConvLSTM is used to encode the input time series. The attention mechanism is used to assign different weights to the features, which are subsequently decoded by the decoder LSTM layer. Finally, the fully connected layer interprets the output. This model is applied to forecast the multi-energy load data of the micro-energy network in a certain area of Northwest China. The test results prove that our model is convergent, and the evaluation index value of the model is better than that of the multi-task FC-LSTM and the single-task FC-LSTM. In particular, the application of the attention mechanism makes the model converge faster and with higher precision.

scholarly journals A New Multi-Scale Convolutional Model Based on Multiple Attention for Image Classification

2019 ◽  
Vol 10 (1) ◽  
pp. 101 ◽  
Author(s):  
Yadong Yang ◽  
Chengji Xu ◽  
Feng Dong ◽  
Xiaofeng Wang
Keyword(s):  
Image Classification ◽  
Attention Mechanism ◽  
Natural Scenes ◽  
Significant Information ◽  
Convolutional Networks ◽  
Multi Scale ◽  
Model Based ◽  
Convolution Model ◽  
Grouping Method ◽  
Feature Expression

Computer vision systems are insensitive to the scale of objects in natural scenes, so it is important to study the multi-scale representation of features. Res2Net implements hierarchical multi-scale convolution in residual blocks, but its random grouping method affects the robustness and intuitive interpretability of the network. We propose a new multi-scale convolution model based on multiple attention. It introduces the attention mechanism into the structure of a Res2-block to better guide feature expression. First, we adopt channel attention to score channels and sort them in descending order of the feature’s importance (Channels-Sort). The sorted residual blocks are grouped and intra-block hierarchically convolved to form a single attention and multi-scale block (AMS-block). Then, we implement channel attention on the residual small blocks to constitute a dual attention and multi-scale block (DAMS-block). Introducing spatial attention before sorting the channels to form multi-attention multi-scale blocks(MAMS-block). A MAMS-convolutional neural network (CNN) is a series of multiple MAMS-blocks. It enables significant information to be expressed at more levels, and can also be easily grafted into different convolutional structures. Limited by hardware conditions, we only prove the validity of the proposed ideas through convolutional networks of the same magnitude. The experimental results show that the convolution model with an attention mechanism and multi-scale features is superior in image classification.

An interpretable mechanism for personalized recommendation based on cross feature

2021 ◽  
pp. 1-12
Author(s):  
Lv YE ◽  
Yue Yang ◽  
Jian-Xu Zeng
Keyword(s):  
Recommender System ◽  
Online Shopping ◽  
Side Information ◽  
Attention Mechanism ◽  
Personalized Recommendation ◽  
Model Based ◽  
Embedded Model ◽  
Feature Attention ◽  
Match Score

The existing recommender system provides personalized recommendation service for users in online shopping, entertainment, and other activities. In order to improve the probability of users accepting the system’s recommendation service, compared with the traditional recommender system, the interpretable recommender system will give the recommendation reasons and results at the same time. In this paper, an interpretable recommendation model based on XGBoost tree is proposed to obtain comprehensible and effective cross features from side information. The results are input into the embedded model based on attention mechanism to capture the invisible interaction among user IDs, item IDs and cross features. The captured interactions are used to predict the match score between the user and the recommended item. Cross-feature attention score is used to generate different recommendation reasons for different user-items.Experimental results show that the proposed algorithm can guarantee the quality of recommendation. The transparency and readability of the recommendation process has been improved by providing reference reasons. This method can help users better understand the recommendation behavior of the system and has certain enlightenment to help the recommender system become more personalized and intelligent.

scholarly journals MOOC Dropout Prediction Using FWTS-CNN Model Based on Fused Feature Weighting and Time Series

IEEE Access ◽  
2020 ◽  
pp. 1-1
Author(s):  
Ya-Feng Zheng ◽  
Zhang-Hao Gao ◽  
Yi-Hang Wang ◽  
Qian Fu
Keyword(s):  
Time Series ◽  
Feature Weighting ◽  
Model Based

scholarly journals DeepRePath: Identifying the Prognostic Features of Early-Stage Lung Adenocarcinoma Using Multi-Scale Pathology Images and Deep Convolutional Neural Networks

Cancers ◽  
2021 ◽  
Vol 13 (13) ◽  
pp. 3308
Author(s):  
Won Sang Shim ◽  
Kwangil Yim ◽  
Tae-Jung Kim ◽  
Yeoun Eun Sung ◽  
Gyeongyun Lee ◽  
...  
Keyword(s):  
Lung Adenocarcinoma ◽  
Early Stage ◽  
External Validation ◽  
Area Under The Curve ◽  
The Cancer Genome Atlas ◽  
Deep Convolutional Neural Networks ◽  
Prognostic Features ◽  
Multi Scale ◽  
Model Based ◽  
Number Of Patients

The prognosis of patients with lung adenocarcinoma (LUAD), especially early-stage LUAD, is dependent on clinicopathological features. However, its predictive utility is limited. In this study, we developed and trained a DeepRePath model based on a deep convolutional neural network (CNN) using multi-scale pathology images to predict the prognosis of patients with early-stage LUAD. DeepRePath was pre-trained with 1067 hematoxylin and eosin-stained whole-slide images of LUAD from the Cancer Genome Atlas. DeepRePath was further trained and validated using two separate CNNs and multi-scale pathology images of 393 resected lung cancer specimens from patients with stage I and II LUAD. Of the 393 patients, 95 patients developed recurrence after surgical resection. The DeepRePath model showed average area under the curve (AUC) scores of 0.77 and 0.76 in cohort I and cohort II (external validation set), respectively. Owing to low performance, DeepRePath cannot be used as an automated tool in a clinical setting. When gradient-weighted class activation mapping was used, DeepRePath indicated the association between atypical nuclei, discohesive tumor cells, and tumor necrosis in pathology images showing recurrence. Despite the limitations associated with a relatively small number of patients, the DeepRePath model based on CNNs with transfer learning could predict recurrence after the curative resection of early-stage LUAD using multi-scale pathology images.

scholarly journals Non-Intrusive Load Disaggregation Based on a Multi-Scale Attention Residual Network

2020 ◽  
Vol 10 (24) ◽  
pp. 9132
Author(s):  
Liguo Weng ◽  
Xiaodong Zhang ◽  
Junhao Qian ◽  
Min Xia ◽  
Yiqing Xu ◽  
...  
Keyword(s):  
Smart Grids ◽  
Recognition Rate ◽  
Low Frequency ◽  
Attention Mechanism ◽  
Learning Ability ◽  
Residual Network ◽  
Multi Scale ◽  
Energy Disaggregation ◽  
Benchmark Datasets ◽  
Load Disaggregation

Non-intrusive load disaggregation (NILD) is of great significance to the development of smart grids. Current energy disaggregation methods extract features from sequences, and this process easily leads to a loss of load features and difficulties in detecting, resulting in a low recognition rate of low-use electrical appliances. To solve this problem, a non-intrusive sequential energy disaggregation method based on a multi-scale attention residual network is proposed. Multi-scale convolutions are used to learn features, and the attention mechanism is used to enhance the learning ability of load features. The residual learning further improves the performance of the algorithm, avoids network degradation, and improves the precision of load decomposition. The experimental results on two benchmark datasets show that the proposed algorithm has more advantages than the existing algorithms in terms of load disaggregation accuracy and judgments of the on/off state, and the attention mechanism can further improve the disaggregation accuracy of low-frequency electrical appliances.

Sign in / Sign up

Export Citation Format

Share Document