scholarly journals Intra-domain and cross-domain transfer learning for time series data – How transferable are the features?

2021 ◽  
pp. 107976
Author(s):  
Erik Otović ◽  
Marko Njirjak ◽  
Dario Jozinović ◽  
Goran Mauša ◽  
Alberto Michelini ◽  
...  
Keyword(s):  
Time Series ◽  
Transfer Learning ◽  
Time Series Data ◽  
Series Data ◽  
Cross Domain ◽  
Domain Transfer ◽  
Domain Transfer Learning

Tri-training for Dependency Parsing Domain Adaptation

2022 ◽  
Vol 21 (3) ◽  
pp. 1-17
Author(s):  
Shu Jiang ◽  
Zuchao Li ◽  
Hai Zhao ◽  
Bao-Liang Lu ◽  
Rui Wang
Keyword(s):  
Transfer Learning ◽  
High Performance ◽  
Domain Adaptation ◽  
Language Model ◽  
Training Methods ◽  
Dependency Parsing ◽  
Cross Domain ◽  
Cross Lingual ◽  
Domain Transfer ◽  
Domain Transfer Learning

In recent years, the research on dependency parsing focuses on improving the accuracy of the domain-specific (in-domain) test datasets and has made remarkable progress. However, there are innumerable scenarios in the real world that are not covered by the dataset, namely, the out-of-domain dataset. As a result, parsers that perform well on the in-domain data usually suffer from significant performance degradation on the out-of-domain data. Therefore, to adapt the existing in-domain parsers with high performance to a new domain scenario, cross-domain transfer learning methods are essential to solve the domain problem in parsing. This paper examines two scenarios for cross-domain transfer learning: semi-supervised and unsupervised cross-domain transfer learning. Specifically, we adopt a pre-trained language model BERT for training on the source domain (in-domain) data at the subword level and introduce self-training methods varied from tri-training for these two scenarios. The evaluation results on the NLPCC-2019 shared task and universal dependency parsing task indicate the effectiveness of the adopted approaches on cross-domain transfer learning and show the potential of self-learning to cross-lingual transfer learning.

Intra-domain and cross-domain transfer learning for time series

2021 ◽  
Author(s):  
Erik Otović ◽  
Marko Njirjak ◽  
Dario Jozinović ◽  
Goran Mauša ◽  
Alberto Michelini ◽  
...  
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Transfer Learning ◽  
Time Series Data ◽  
The Other ◽  
Series Data ◽  
Sound Recognition ◽  
Transfer Of Knowledge ◽  
Learning Models ◽  
Machine Learning Models

<p>In this study, we compared the performance of machine learning models trained using transfer learning and those that were trained from scratch - on time series data. Four machine learning models were used for the experiment. Two models were taken from the field of seismology, and the other two are general-purpose models for working with time series data. The accuracy of selected models was systematically observed and analyzed when switching within the same domain of application (seismology), as well as between mutually different domains of application (seismology, speech, medicine, finance). In seismology, we used two databases of local earthquakes (one in counts, and the other with the instrument response removed) and a database of global earthquakes for predicting earthquake magnitude; other datasets targeted classifying spoken words (speech), predicting stock prices (finance) and classifying muscle movement from EMG signals (medicine).<br>In practice, it is very demanding and sometimes impossible to collect datasets of tagged data large enough to successfully train a machine learning model. Therefore, in our experiment, we use reduced data sets of 1,500 and 9,000 data instances to mimic such conditions. Using the same scaled-down datasets, we trained two sets of machine learning models: those that used transfer learning for training and those that were trained from scratch. We compared the performances between pairs of models in order to draw conclusions about the utility of transfer learning. In order to confirm the validity of the obtained results, we repeated the experiments several times and applied statistical tests to confirm the significance of the results. The study shows when, within the set experimental framework, the transfer of knowledge brought improvements in terms of model accuracy and in terms of model convergence rate.<br><br>Our results show that it is possible to achieve better performance and faster convergence by transferring knowledge from the domain of global earthquakes to the domain of local earthquakes; sometimes also vice versa. However, improvements in seismology can sometimes also be achieved by transferring knowledge from medical and audio domains. The results show that the transfer of knowledge between other domains brought even more significant improvements, compared to those within the field of seismology. For example, it has been shown that models in the field of sound recognition have achieved much better performance compared to classical models and that the domain of sound recognition is very compatible with knowledge from other domains. We came to similar conclusions for the domains of medicine and finance. Ultimately, the paper offers suggestions when transfer learning is useful, and the explanations offered can provide a good starting point for knowledge transfer using time series data.</p>

scholarly journals Automatic Microsurgical Skill Assessment Based on Cross-Domain Transfer Learning

2020 ◽  
Vol 5 (3) ◽  
pp. 4148-4155
Author(s):  
Dandan Zhang ◽  
Zicong Wu ◽  
Junhong Chen ◽  
Anzhu Gao ◽  
Xu Chen ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Skill Assessment ◽  
Cross Domain ◽  
Domain Transfer ◽  
Domain Transfer Learning

scholarly journals Deep Transfer Learning for Time Series Data Based on Sensor Modality Classification

Sensors ◽  
2020 ◽  
Vol 20 (15) ◽  
pp. 4271 ◽  
Author(s):  
Frédéric Li ◽  
Kimiaki Shirahama ◽  
Muhammad Adeel Nisar ◽  
Xinyu Huang ◽  
Marcin Grzegorzek
Keyword(s):  
Time Series ◽  
Ubiquitous Computing ◽  
Transfer Learning ◽  
Time Series Data ◽  
Wearable Devices ◽  
Series Data ◽  
Target Field ◽  
Proper Training ◽  
Application Fields ◽  
General Architecture

The scarcity of labelled time-series data can hinder a proper training of deep learning models. This is especially relevant for the growing field of ubiquitous computing, where data coming from wearable devices have to be analysed using pattern recognition techniques to provide meaningful applications. To address this problem, we propose a transfer learning method based on attributing sensor modality labels to a large amount of time-series data collected from various application fields. Using these data, our method firstly trains a Deep Neural Network (DNN) that can learn general characteristics of time-series data, then transfers it to another DNN designed to solve a specific target problem. In addition, we propose a general architecture that can adapt the transferred DNN regardless of the sensors used in the target field making our approach in particular suitable for multichannel data. We test our method for two ubiquitous computing problems—Human Activity Recognition (HAR) and Emotion Recognition (ER)—and compare it a baseline training the DNN without using transfer learning. For HAR, we also introduce a new dataset, Cognitive Village-MSBand (CogAge), which contains data for 61 atomic activities acquired from three wearable devices (smartphone, smartwatch, and smartglasses). Our results show that our transfer learning approach outperforms the baseline for both HAR and ER.

scholarly journals Damage detection using in-domain and cross-domain transfer learning

2021 ◽  
Author(s):  
Zaharah A. Bukhsh ◽  
Nils Jansen ◽  
Aaqib Saeed
Keyword(s):  
Damage Detection ◽  
Learning Strategies ◽  
Transfer Learning ◽  
Large Scale ◽  
Visual Inspection ◽  
Superior Performance ◽  
Cross Domain ◽  
Domain Transfer ◽  
Domain Transfer Learning ◽  
The Impact

AbstractWe investigate the capabilities of transfer learning in the area of structural health monitoring. In particular, we are interested in damage detection for concrete structures. Typical image datasets for such problems are relatively small, calling for the transfer of learned representation from a related large-scale dataset. Past efforts of damage detection using images have mainly considered cross-domain transfer learning approaches using pre-trained ImageNet models that are subsequently fine-tuned for the target task. However, there are rising concerns about the generalizability of ImageNet representations for specific target domains, such as for visual inspection and medical imaging. We, therefore, evaluate a combination of in-domain and cross-domain transfer learning strategies for damage detection in bridges. We perform comprehensive comparisons to study the impact of cross-domain and in-domain transfer, with various initialization strategies, using six publicly available visual inspection datasets. The pre-trained models are also evaluated for their ability to cope with the extremely low-data regime. We show that the combination of cross-domain and in-domain transfer persistently shows superior performance specially with tiny datasets. Likewise, we also provide visual explanations of predictive models to enable algorithmic transparency and provide insights to experts about the intrinsic decision logic of typically black-box deep models.

scholarly journals Fuzzy Transfer Learning in Time Series Forecasting for Stock Market Prices

2021 ◽  
Author(s):  
Shanoli Samui Pal ◽  
Samarjit Kar
Keyword(s):  
Time Series ◽  
Stock Market ◽  
Prior Knowledge ◽  
Transfer Learning ◽  
Prediction Accuracy ◽  
Time Series Data ◽  
Time Series Prediction ◽  
Series Data ◽  
Data Sets ◽  
Market Data

Abstract Transfer learning involves transferring prior knowledge of solving similar problems in order to achieve quick and efficient solution. The aim of fuzzy transfer learning is to transfer prior knowledge in an imprecise environment. Time series like stock market data are non-linear in nature and movement of stock is uncertain, so it is quite difficult following the stock market and in decision making. In this study, we propose a method to forecast stock market time series in the situation when we can use prior experience to make decisions. Fuzzy transfer learning (FuzzyTL) is based on knowledge transfer in that and adapting rules obtained domain. Three different stock market time series data sets are used for comparative study. It is observed that the effect of knowledge transferring works well together with smoothing of dependent attributes as the stock market data fluctuate with time. Finally, we give an empirical application in Shenzhen stock market with larger data sets to demonstrate the performance of the model. We have explored FuzzyTL in time series prediction to unerstand the essence of FuzzyTL. We were working on the question of the capability of FuzzyTL in improving prediction accuracy. From the comparisons, it can be said fuzzy transfer learning with smoothing improves prediction accuracy efficiently.

Sign in / Sign up

Export Citation Format

Share Document