Selective transfer learning with adversarial training for stock movement prediction

Abstract Introduction Classifying whether concepts in an unstructured clinical text are negated is an important unsolved task. New domain adaptation and transfer learning methods can potentially address this issue. Objective We examine neural unsupervised domain adaptation methods, introducing a novel combination of domain adaptation with transformer-based transfer learning methods to improve negation detection. We also want to better understand the interaction between the widely used bidirectional encoder representations from transformers (BERT) system and domain adaptation methods. Materials and Methods We use 4 clinical text datasets that are annotated with negation status. We evaluate a neural unsupervised domain adaptation algorithm and BERT, a transformer-based model that is pretrained on massive general text datasets. We develop an extension to BERT that uses domain adversarial training, a neural domain adaptation method that adds an objective to the negation task, that the classifier should not be able to distinguish between instances from 2 different domains. Results The domain adaptation methods we describe show positive results, but, on average, the best performance is obtained by plain BERT (without the extension). We provide evidence that the gains from BERT are likely not additive with the gains from domain adaptation. Discussion Our results suggest that, at least for the task of clinical negation detection, BERT subsumes domain adaptation, implying that BERT is already learning very general representations of negation phenomena such that fine-tuning even on a specific corpus does not lead to much overfitting. Conclusion Despite being trained on nonclinical text, the large training sets of models like BERT lead to large gains in performance for the clinical negation detection task.

Download Full-text

Aspect-augmented Adversarial Networks for Domain Adaptation

Transactions of the Association for Computational Linguistics ◽

10.1162/tacl_a_00077 ◽

2017 ◽

Vol 5 ◽

pp. 515-528 ◽

Cited By ~ 17

Author(s):

Yuan Zhang ◽

Regina Barzilay ◽

Tommi Jaakkola

Keyword(s):

Transfer Learning ◽

Domain Adaptation ◽

Experimental Results ◽

Dependent Manner ◽

Target Classification ◽

Adversarial Networks ◽

Adversarial Training ◽

Classification Tasks ◽

Class Labels

We introduce a neural method for transfer learning between two (source and target) classification tasks or aspects over the same domain. Rather than training on target labels, we use a few keywords pertaining to source and target aspects indicating sentence relevance instead of document class labels. Documents are encoded by learning to embed and softly select relevant sentences in an aspect-dependent manner. A shared classifier is trained on the source encoded documents and labels, and applied to target encoded documents. We ensure transfer through aspect-adversarial training so that encoded documents are, as sets, aspect-invariant. Experimental results demonstrate that our approach outperforms different baselines and model variants on two datasets, yielding an improvement of 27% on a pathology dataset and 5% on a review dataset.

Download Full-text

Method of extracting sentences about protein interaction from the literature on protein structure analysis using selective transfer learning

2012 IEEE 12th International Conference on Bioinformatics & Bioengineering (BIBE) ◽

10.1109/bibe.2012.6399705 ◽

2012 ◽

Author(s):

Shun Koyabu ◽

Riku Kyougoku ◽

Takenao Ohkawa

Keyword(s):

Protein Structure ◽

Transfer Learning ◽

Structure Analysis ◽

Protein Interaction ◽

Selective Transfer ◽

Protein Structure Analysis

Download Full-text

Wasserstein Selective Transfer Learning for Cross-domain Text Mining

10.18653/v1/2021.emnlp-main.770 ◽

2021 ◽

Author(s):

Lingyun Feng ◽

Minghui Qiu ◽

Yaliang Li ◽

Haitao Zheng ◽

Ying Shen

Keyword(s):

Text Mining ◽

Transfer Learning ◽

Selective Transfer ◽

Cross Domain

Download Full-text

Enhancing Stock Movement Prediction with Adversarial Training

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/810 ◽

2019 ◽

Cited By ~ 8

Author(s):

Fuli Feng ◽

Huimin Chen ◽

Xiangnan He ◽

Ji Ding ◽

Maosong Sun ◽

...

Keyword(s):

Stock Price ◽

State Of The Art ◽

Stochastic Variable ◽

Stock Prediction ◽

Movement Prediction ◽

Neural Network Prediction ◽

Adversarial Training ◽

Network Prediction ◽

Near Future ◽

New Machine

This paper contributes a new machine learning solution for stock movement prediction, which aims to predict whether the price of a stock will be up or down in the near future. The key novelty is that we propose to employ adversarial training to improve the generalization of a neural network prediction model. The rationality of adversarial training here is that the input features to stock prediction are typically based on stock price, which is essentially a stochastic variable and continuously changed with time by nature. As such, normal training with static price-based features (e.g. the close price) can easily overfit the data, being insufficient to obtain reliable models. To address this problem, we propose to add perturbations to simulate the stochasticity of price variable, and train the model to work well under small yet intentional perturbations. Extensive experiments on two real-world stock data show that our method outperforms the state-of-the-art solution [Xu and Cohen, 2018] with 3.11% relative improvements on average w.r.t. accuracy, validating the usefulness of adversarial training for stock prediction task.

Download Full-text