scholarly journals Transfer Learning for Tabular Data

2021 ◽  
Author(s):  
Leonid Joffe
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Deep Learning ◽  
Transfer Learning ◽  
Internal Representation ◽  
The Other ◽  
Learning Models ◽  
Tabular Data ◽  
Convolutional Network ◽  
Feature Interactions

Deep learning models for tabular data are restricted to a specific table format. Computer vision models, on the other hand, have a broader applicability; they work on all images and can learn universal features. This allows them to be trained on enormous corpora and have very wide transferability and applicability. Inspired by these properties, this work presents an architecture that aims to capture useful patterns across arbitrary tables. The model is trained on randomly sampled subsets of features from a table, processed by a convolutional network. This internal representation captures feature interactions that appear in the table. Experimental results show that the embeddings produced by this model are useful and transferable across many commonly used machine learning benchmarks datasets. Specifically, that using the embeddings produced by the network as additional features, improves the performance of a number of classifiers.

scholarly journals Transfer Learning for Tabular Data

2021 ◽  
Author(s):  
Leonid Joffe
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Deep Learning ◽  
Transfer Learning ◽  
Internal Representation ◽  
The Other ◽  
Learning Models ◽  
Tabular Data ◽  
Convolutional Network ◽  
Feature Interactions

Deep learning models for tabular data are restricted to a specific table format. Computer vision models, on the other hand, have a broader applicability; they work on all images and can learn universal features. This allows them to be trained on enormous corpora and have very wide transferability and applicability. Inspired by these properties, this work presents an architecture that aims to capture useful patterns across arbitrary tables. The model is trained on randomly sampled subsets of features from a table, processed by a convolutional network. This internal representation captures feature interactions that appear in the table. Experimental results show that the embeddings produced by this model are useful and transferable across many commonly used machine learning benchmarks datasets. Specifically, that using the embeddings produced by the network as additional features, improves the performance of a number of classifiers.

Intra-domain and cross-domain transfer learning for time series

2021 ◽  
Author(s):  
Erik Otović ◽  
Marko Njirjak ◽  
Dario Jozinović ◽  
Goran Mauša ◽  
Alberto Michelini ◽  
...  
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Transfer Learning ◽  
Time Series Data ◽  
The Other ◽  
Series Data ◽  
Sound Recognition ◽  
Transfer Of Knowledge ◽  
Learning Models ◽  
Machine Learning Models

<p>In this study, we compared the performance of machine learning models trained using transfer learning and those that were trained from scratch - on time series data. Four machine learning models were used for the experiment. Two models were taken from the field of seismology, and the other two are general-purpose models for working with time series data. The accuracy of selected models was systematically observed and analyzed when switching within the same domain of application (seismology), as well as between mutually different domains of application (seismology, speech, medicine, finance). In seismology, we used two databases of local earthquakes (one in counts, and the other with the instrument response removed) and a database of global earthquakes for predicting earthquake magnitude; other datasets targeted classifying spoken words (speech), predicting stock prices (finance) and classifying muscle movement from EMG signals (medicine).<br>In practice, it is very demanding and sometimes impossible to collect datasets of tagged data large enough to successfully train a machine learning model. Therefore, in our experiment, we use reduced data sets of 1,500 and 9,000 data instances to mimic such conditions. Using the same scaled-down datasets, we trained two sets of machine learning models: those that used transfer learning for training and those that were trained from scratch. We compared the performances between pairs of models in order to draw conclusions about the utility of transfer learning. In order to confirm the validity of the obtained results, we repeated the experiments several times and applied statistical tests to confirm the significance of the results. The study shows when, within the set experimental framework, the transfer of knowledge brought improvements in terms of model accuracy and in terms of model convergence rate.<br><br>Our results show that it is possible to achieve better performance and faster convergence by transferring knowledge from the domain of global earthquakes to the domain of local earthquakes; sometimes also vice versa. However, improvements in seismology can sometimes also be achieved by transferring knowledge from medical and audio domains. The results show that the transfer of knowledge between other domains brought even more significant improvements, compared to those within the field of seismology. For example, it has been shown that models in the field of sound recognition have achieved much better performance compared to classical models and that the domain of sound recognition is very compatible with knowledge from other domains. We came to similar conclusions for the domains of medicine and finance. Ultimately, the paper offers suggestions when transfer learning is useful, and the explanations offered can provide a good starting point for knowledge transfer using time series data.</p>

Data science in economics: comprehensive review of advanced machine learning and deep learning methods

2020 ◽  
Author(s):  
Saeed Nosratabadi ◽  
Amir Mosavi ◽  
Puhong Duan ◽  
Pedram Ghamisi ◽  
Filip Ferdinand ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Prediction Accuracy ◽  
Data Science ◽  
State Of The Art ◽  
Hybrid Models ◽  
The Other ◽  
Learning Models ◽  
Comprehensive Review

Abstract This paper provides the state of the art of data science in economics. Through a novel taxonomy of applications and methods advances in data science are investigated. The data science advances are investigated in three individual classes of deep learning models, ensemble models, and hybrid models. Application domains include stock market, marketing, E-commerce, corporate banking, and cryptocurrency. Prisma method, a systematic literature review methodology is used to ensure the quality of the survey. The findings revealed that the trends are on advancement of hybrid models as more than 51% of the reviewed articles applied hybrid model. On the other hand, it is found that based on the RMSE accuracy metric, hybrid models had higher prediction accuracy than other algorithms. While it is expected the trends go toward the advancements of deep learning models.

scholarly journals Machine Learning Models for Abnormality Detection in Musculoskeletal Radiographs

Reports ◽  
2019 ◽  
Vol 2 (4) ◽  
pp. 26 ◽  
Author(s):  
Govind Chada
Keyword(s):  
Machine Learning ◽  
Primary Care ◽  
Deep Learning ◽  
Transfer Learning ◽  
Primary Care Physicians ◽  
Screening Tools ◽  
Learning Approaches ◽  
Limited Data ◽  
Learning Models ◽  
High Recognition Accuracy

Increasing radiologist workloads and increasing primary care radiology services make it relevant to explore the use of artificial intelligence (AI) and particularly deep learning to provide diagnostic assistance to radiologists and primary care physicians in improving the quality of patient care. This study investigates new model architectures and deep transfer learning to improve the performance in detecting abnormalities of upper extremities while training with limited data. DenseNet-169, DenseNet-201, and InceptionResNetV2 deep learning models were implemented and evaluated on the humerus and finger radiographs from MURA, a large public dataset of musculoskeletal radiographs. These architectures were selected because of their high recognition accuracy in a benchmark study. The DenseNet-201 and InceptionResNetV2 models, employing deep transfer learning to optimize training on limited data, detected abnormalities in the humerus radiographs with 95% CI accuracies of 83–92% and high sensitivities greater than 0.9, allowing for these models to serve as useful initial screening tools to prioritize studies for expedited review. The performance in the case of finger radiographs was not as promising, possibly due to the limitations of large inter-radiologist variation. It is suggested that the causes of this variation be further explored using machine learning approaches, which may lead to appropriate remediation.

scholarly journals Data Science in Economics: Comprehensive Review of Advanced Machine Learning and Deep Learning Methods

2020 ◽  
Author(s):  
Saeed Nosratabadi ◽  
Amir Mosavi ◽  
Puhong Duan ◽  
Pedram Ghamisi ◽  
Filip Ferdinand ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Prediction Accuracy ◽  
Data Science ◽  
State Of The Art ◽  
Hybrid Models ◽  
The Other ◽  
Learning Models ◽  
Comprehensive Review

This paper provides the state of the art of data science in economics. Through a novel taxonomy of applications and methods advances in data science are investigated. The data science advances are investigated in three individual classes of deep learning models, ensemble models, and hybrid models. Application domains include stock market, marketing, E-commerce, corporate banking, and cryptocurrency. Prisma method, a systematic literature review methodology is used to ensure the quality of the survey. The findings revealed that the trends are on advancement of hybrid models as more than 51% of the reviewed articles applied hybrid model. On the other hand, it is found that based on the RMSE accuracy metric, hybrid models had higher prediction accuracy than other algorithms. While it is expected the trends go toward the advancements of deep learning models.

scholarly journals Detection of Unilateral Arm Paresis after Stroke by Wearable Accelerometers and Machine Learning

Sensors ◽  
2021 ◽  
Vol 21 (23) ◽  
pp. 7784
Author(s):  
Johan Wasselius ◽  
Eric Lyckegård Finn ◽  
Emma Persson ◽  
Petter Ericson ◽  
Christina Brogårdh ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Accelerometer Data ◽  
Learning Models ◽  
Stroke Patients ◽  
Convolutional Network ◽  
Stroke Treatment ◽  
Data Windows

Recent advances in stroke treatment have provided effective tools to successfully treat ischemic stroke, but still a majority of patients are not treated due to late arrival to hospital. With modern stroke treatment, earlier arrival would greatly improve the overall treatment results. This prospective study was performed to asses the capability of bilateral accelerometers worn in bracelets 24/7 to detect unilateral arm paralysis, a hallmark symptom of stroke, early enough to receive treatment. Classical machine learning algorithms as well as state-of-the-art deep neural networks were evaluated on detection times between 15 min and 120 min. Motion data were collected using triaxial accelerometer bracelets worn on both arms for 24 h. Eighty-four stroke patients with unilateral arm motor impairment and 101 healthy subjects participated in the study. Accelerometer data were divided into data windows of different lengths and analyzed using multiple machine learning algorithms. The results show that all algorithms performed well in separating the two groups early enough to be clinically relevant, based on wrist-worn accelerometers. The two evaluated deep learning models, fully convolutional network and InceptionTime, performed better than the classical machine learning models with an AUC score between 0.947–0.957 on 15 min data windows and up to 0.993–0.994 on 120 min data windows. Window lengths longer than 90 min only marginally improved performance. The difference in performance between the deep learning models and the classical models was statistically significant according to a non-parametric Friedman test followed by a post-hoc Nemenyi test. Introduction of wearable stroke detection devices may dramatically increase the portion of stroke patients eligible for revascularization and shorten the time to treatment. Since the treatment effect is highly time-dependent, early stroke detection may dramatically improve stroke outcomes.

Efficient Deep Learning Models for Categorizing Chenopodiaceae in the Wild

2021 ◽  
pp. 2152015
Author(s):  
Ahmad Heidary-Sharifabad ◽  
Mohsen Sardari Zarchi ◽  
Sima Emadi ◽  
Gholamreza Zarei
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Species Recognition ◽  
The Other ◽  
Fine Tuning ◽  
Learning Models ◽  
Natural Habitats ◽  
Biodiversity Protection ◽  
Species Analysis ◽  
In The Wild

The Chenopodiaceae species are ecologically and financially important, and play a significant role in biodiversity around the world. Biodiversity protection is critical for the survival and sustainability of each ecosystem and since plant species recognition in their natural habitats is the first process in plant diversity protection, an automatic species classification in the wild would greatly help the species analysis and consequently biodiversity protection on earth. Computer vision approaches can be used for automatic species analysis. Modern computer vision approaches are based on deep learning techniques. A standard dataset is essential in order to perform a deep learning algorithm. Hence, the main goal of this research is to provide a standard dataset of Chenopodiaceae images. This dataset is called ACHENY and contains 27030 images of 30 Chenopodiaceae species in their natural habitats. The other goal of this study is to investigate the applicability of ACHENY dataset by using deep learning models. Therefore, two novel deep learning models based on ACHENY dataset are introduced: First, a lightweight deep model which is trained from scratch and is designed innovatively to be agile and fast. Second, a model based on the EfficientNet-B1 architecture, which is pre-trained on ImageNet and is fine-tuned on ACHENY. The experimental results show that the two proposed models can do Chenopodiaceae fine-grained species recognition with promising accuracy. To evaluate our models, their performance was compared with the well-known VGG-16 model after fine-tuning it on ACHENY. Both VGG-16 and our first model achieved about 80% accuracy while the size of VGG-16 is about 16[Formula: see text] larger than the first model. Our second model has an accuracy of about 90% and outperforms the other models where its number of parameters is 5[Formula: see text] than the first model but it is still about one-third of the VGG-16 parameters.

scholarly journals Comparison of Deep Transfer Learning Techniques in Human Skin Burns Discrimination

2020 ◽  
Vol 3 (2) ◽  
pp. 20 ◽  
Author(s):  
Aliyu Abubakar ◽  
Mohammed Ajuji ◽  
Ibrahim Usman Yahya
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Transfer Learning ◽  
Fine Tuning ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Learning Models ◽  
Skin Injuries ◽  
Learning Techniques ◽  
Injured Skin

While visual assessment is the standard technique for burn evaluation, computer-aided diagnosis is increasingly sought due to high number of incidences globally. Patients are increasingly facing challenges which are not limited to shortage of experienced clinicians, lack of accessibility to healthcare facilities and high diagnostic cost. Certain number of studies were proposed in discriminating burn and healthy skin using machine learning leaving a huge and important gap unaddressed; whether burns and related skin injuries can be effectively discriminated using machine learning techniques. Therefore, we specifically use transfer learning by leveraging pre-trained deep learning models due to deficient dataset in this paper, to discriminate two classes of skin injuries—burnt skin and injured skin. Experiments were extensively conducted using three state-of-the-art pre-trained deep learning models that includes ResNet50, ResNet101 and ResNet152 for image patterns extraction via two transfer learning strategies—fine-tuning approach where dense and classification layers were modified and trained with features extracted by base layers and in the second approach support vector machine (SVM) was used to replace top-layers of the pre-trained models, trained using off-the-shelf features from the base layers. Our proposed approach records near perfect classification accuracy in categorizing burnt skin ad injured skin of approximately 99.9%.

scholarly journals Comparison of Deep Transfer Learning Techniques in Human Skin Burns Discrimination

2020 ◽  
Author(s):  
Aliyu Abubakar ◽  
Mohammed Ajuji ◽  
Ibrahim Usman Yahya
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Learning Strategies ◽  
Transfer Learning ◽  
Standard Technique ◽  
Fine Tuning ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Learning Models ◽  
Learning Techniques

While visual assessment is the standard technique for burn evaluation, computer-aided diagnosis is increasingly sought due to high number of incidences globally. Patients are increasingly facing challenges which are not limited to shortage of experienced clinicians, lack of accessibility to healthcare facilities, and high diagnostic cost. Certain number of studies were proposed in discriminating burn and healthy skin using machine learning leaving a huge and important gap unaddressed; whether burns and related skin injuries can be effectively discriminated using machine learning techniques. Therefore, we specifically use pre-trained deep learning models due to deficient dataset to train a new model from scratch. Experiments were extensively conducted using three state-of-the-art pre-trained deep learning models that includes ResNet50, ResNet101 and ResNet152 for image patterns extraction via two transfer learning strategies: fine-tuning approach where dense and classification layers were modified and trained with features extracted by base layers, and in the second approach support vector machine (SVM) was used to replace top-layers of the pre-trained models, trained using off-the-shelf features from the base layers. Our proposed approach records near perfect classification accuracy of approximately 99.9%.

Large-Scale Land Cover Mapping on Sentinel-1 SAR Imagery Using Deep Transfer Learning

2019 ◽  
Author(s):  
Sanja Šćepanović ◽  
Oleg Antropov ◽  
Pekka Laurila ◽  
Vladimir Ignatenko ◽  
Jaan Praks
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Land Cover ◽  
Transfer Learning ◽  
Large Scale ◽  
Geographical Area ◽  
Wide Area ◽  
Land Cover Mapping ◽  
Learning Models ◽  
Sar Imagery

Land cover mapping and monitoring are essential for understanding the environment and the effects of human activities on the environment. The automatic approaches to land cover mapping are predominantly based on the traditional machine learning that requires heuristic feature design. Such approaches are relatively slow and they are often suitable only for a particular type of satellite sensor or geographical area. Recently, deep learning has outperformed traditional machine learning approaches on a range of image processing tasks including image classification and segmentation. In this study, we demonstrated the suitability of deep learning models to land cover mapping on a large scale using satellite C-band SAR images. We used a set of 14 ESA Sentinel-1 scenes acquired during the summer season over a wide area in Finland representative of the land cover in the country. These imagery were used as an input to seven state-of-the-art deep-learning models for semantic segmentation, namely U-Net, DeepLabV3+, PSPNet, BiSeNet, SegNet, FC-DenseNet, and FRRN-B. These models were pre-trained on the ImageNet dataset and further fine-tuned in this study. To the best of our knowledge, this is the first successful demonstration of transfer learning for SAR imagery in the context of wide-area land-cover mapping. CORINE land cover map produced by the Finnish Environment Institute was used as a reference, and the models were trained to distinguish between 5 Level-1 CORINE classes. Upon the evaluation and benchmarking, we found that all the models demonstrated solid performance, with the top FC-DenseNet model achieving an overall accuracy of 90.66%. These results indicate the suitability of deep learning methods to support efficient wide-area mapping using satellite SAR imagery.

Sign in / Sign up

Export Citation Format

Share Document