A Practical Approach towards Causality Mining in Clinical Text using Active Transfer Learning

In recent years, various studies have begun to use deep learning models to conduct research in the field of human activity recognition (HAR). However, there has been a severe lag in the absolute development of such models since training deep learning models require a lot of labeled data. In fields such as HAR, it is difficult to collect data and there are high costs and efforts involved in manual labeling. The existing methods rely heavily on manual data collection and proper labeling of the data, which is done by human administrators. This often results in the data gathering process often being slow and prone to human-biased labeling. To address these problems, we proposed a new solution for the existing data gathering methods by reducing the labeling tasks conducted on new data based by using the data learned through the semi-supervised active transfer learning method. This method achieved 95.9% performance while also reducing labeling compared to the random sampling or active transfer learning methods.

Download Full-text

Deep learning framework for material design space exploration using active transfer learning and data augmentation

npj Computational Materials ◽

10.1038/s41524-021-00609-2 ◽

2021 ◽

Vol 7 (1) ◽

Author(s):

Yongtae Kim ◽

Youngsoo Kim ◽

Charles Yang ◽

Kundo Park ◽

Grace X. Gu ◽

...

Keyword(s):

Neural Network ◽

Transfer Learning ◽

Design Space Exploration ◽

Predictive Power ◽

Design Space ◽

Data Augmentation ◽

Generative Models ◽

Training Dataset ◽

Initial Training ◽

Active Transfer

AbstractNeural network-based generative models have been actively investigated as an inverse design method for finding novel materials in a vast design space. However, the applicability of conventional generative models is limited because they cannot access data outside the range of training sets. Advanced generative models that were devised to overcome the limitation also suffer from the weak predictive power on the unseen domain. In this study, we propose a deep neural network-based forward design approach that enables an efficient search for superior materials far beyond the domain of the initial training set. This approach compensates for the weak predictive power of neural networks on an unseen domain through gradual updates of the neural network with active transfer learning and data augmentation methods. We demonstrate the potential of our framework with a grid composite optimization problem that has an astronomical number of possible design configurations. Results show that our proposed framework can provide excellent designs close to the global optima, even with the addition of a very small dataset corresponding to less than 0.5% of the initial training dataset size.

Download Full-text

Active Transfer Learning with Zero-Shot Priors: Reusing Past Datasets for Future Tasks

2015 IEEE International Conference on Computer Vision (ICCV) ◽

10.1109/iccv.2015.313 ◽

2015 ◽

Cited By ~ 30

Author(s):

E. Gavves ◽

T. Mensink ◽

T. Tommasi ◽

C. G. M. Snoek ◽

T. Tuytelaars

Keyword(s):

Transfer Learning ◽

Active Transfer

Download Full-text

Does BERT need domain adaptation for clinical negation detection?

Journal of the American Medical Informatics Association ◽

10.1093/jamia/ocaa001 ◽

2020 ◽

Vol 27 (4) ◽

pp. 584-591 ◽

Cited By ~ 2

Author(s):

Chen Lin ◽

Steven Bethard ◽

Dmitriy Dligach ◽

Farig Sadeque ◽

Guergana Savova ◽

...

Keyword(s):

Transfer Learning ◽

Domain Adaptation ◽

Fine Tuning ◽

Adaptation Algorithm ◽

Learning Methods ◽

Clinical Text ◽

Unsupervised Domain Adaptation ◽

Adversarial Training ◽

Negation Detection ◽

Adaptation Method

Abstract Introduction Classifying whether concepts in an unstructured clinical text are negated is an important unsolved task. New domain adaptation and transfer learning methods can potentially address this issue. Objective We examine neural unsupervised domain adaptation methods, introducing a novel combination of domain adaptation with transformer-based transfer learning methods to improve negation detection. We also want to better understand the interaction between the widely used bidirectional encoder representations from transformers (BERT) system and domain adaptation methods. Materials and Methods We use 4 clinical text datasets that are annotated with negation status. We evaluate a neural unsupervised domain adaptation algorithm and BERT, a transformer-based model that is pretrained on massive general text datasets. We develop an extension to BERT that uses domain adversarial training, a neural domain adaptation method that adds an objective to the negation task, that the classifier should not be able to distinguish between instances from 2 different domains. Results The domain adaptation methods we describe show positive results, but, on average, the best performance is obtained by plain BERT (without the extension). We provide evidence that the gains from BERT are likely not additive with the gains from domain adaptation. Discussion Our results suggest that, at least for the task of clinical negation detection, BERT subsumes domain adaptation, implying that BERT is already learning very general representations of negation phenomena such that fine-tuning even on a specific corpus does not lead to much overfitting. Conclusion Despite being trained on nonclinical text, the large training sets of models like BERT lead to large gains in performance for the clinical negation detection task.

Download Full-text