Transfer Learning and Textual Analysis of Accounting Disclosures: Applying Big Data Methods to Small(er) Data Sets

2021 ◽  
Author(s):  
Federico Siano ◽  
Peter Wysocki
Keyword(s):  
Big Data ◽  
Transfer Learning ◽  
Textual Analysis ◽  
Low Cost ◽  
Language Model ◽  
Fine Tuning ◽  
Superior Performance ◽  
Data Sets ◽  
Quarterly Earnings ◽  
Accounting Disclosures

We introduce and apply machine transfer learning methods to analyze accounting disclosures. We use the examples of the new BERT language model and sentiment analysis of quarterly earnings disclosures to demonstrate the key transfer learning concepts of: (i) pre-training on generic "Big Data", (ii) fine-tuning on small accounting datasets, and (iii) using a language model that captures context rather than stand-alone words. Overall, we show that this new approach is easy to implement, uses widely-available and low-cost computing resources, and has superior performance relative to existing textual analysis tools in accounting. We conclude with suggestions for opportunities to apply transfer learning to address important accounting research questions.

scholarly journals Big data; sensor networks and remotely-sensed data for mapping; feature extraction from lidar

2018 ◽  
Vol 1 ◽  
pp. 1-4
Author(s):  
Lorato Tlhabano
Keyword(s):  
Feature Extraction ◽  
Big Data ◽  
Low Cost ◽  
High Tech ◽  
Test Case ◽  
Data Sets ◽  
Remotely Sensed Data ◽  
Dramatic Rise ◽  
Close Range ◽  
Laws And Policies

Unmanned aerial vehicles (UAVs) can be used for mapping in the close range domain, combining aerial and terrestrial photogrammetry and now the emergence of affordable platforms to carry these technologies has opened up new opportunities for mapping and modeling cadastral boundaries. At the current state mainly low cost UAVs fitted with sensors are used in mapping projects with low budgets, the amount of data produced by the UAVs can be enormous hence the need for big data techniques’ and concepts. The past couple of years have witnessed the dramatic rise of low-cost UAVs fitted with high tech Lidar sensors and as such the UAVS have now reached a level of practical reliability and professionalism which allow the use of these systems as mapping platforms. UAV based mapping provides not only the required accuracy with respect to cadastral laws and policies as well as requirements for feature extraction from the data sets and maps produced, UAVs are also competitive to other measurement technologies in terms of economic aspects. In the following an overview on how the various technologies of UAVs, big data concepts and lidar sensor technologies can work together to revolutionize cadastral mapping particularly in Africa and as a test case Botswana in particular will be used to investigate these technologies. These technologies can be combined to efficiently provide cadastral mapping in difficult to reach areas and over large areas of land similar to the Land Administration Procedures, Capacity and Systems (LAPCAS) exercise which was recently undertaken by the Botswana government, we will show how the uses of UAVS fitted with lidar sensor and utilizing big data concepts could have reduced not only costs and time for our government but also how UAVS could have provided more detailed cadastral maps.

scholarly journals Transfer learning using AlexNet Convolutional Neural Network for Face Recognition

2020 ◽  
Vol 9 (11) ◽  
pp. 285-294
Keyword(s):  
Neural Network ◽  
Face Recognition ◽  
Transfer Learning ◽  
Data Augmentation ◽  
Recognition System ◽  
Training Data ◽  
Fine Tuning ◽  
Data Sets ◽  
Learning Method ◽  
Face Recognition System

This research is aimed to achieve high-precision accuracy and for face recognition system. Convolution Neural Network is one of the Deep Learning approaches and has demonstrated excellent performance in many fields, including image recognition of a large amount of training data (such as ImageNet). In fact, hardware limitations and insufficient training data-sets are the challenges of getting high performance. Therefore, in this work the Deep Transfer Learning method using AlexNet pre-trained CNN is proposed to improve the performance of the face-recognition system even for a smaller number of images. The transfer learning method is used to fine-tuning on the last layer of AlexNet CNN model for new classification tasks. The data augmentation (DA) technique also proposed to minimize the over-fitting problem during Deep transfer learning training and to improve accuracy. The results proved the improvement in over-fitting and in performance after using the data augmentation technique. All the experiments were tested on UTeMFD, GTFD, and CASIA-Face V5 small data-sets. As a result, the proposed system achieved a high accuracy as 100% on UTeMFD, 96.67% on GTFD, and 95.60% on CASIA-Face V5 in less than 0.05 seconds of recognition time.

scholarly journals TranScreen: Transfer Learning on Graph-Based Anti-Cancer Virtual Screening Model

2020 ◽  
Vol 4 (3) ◽  
pp. 16
Author(s):  
Milad Salem ◽  
Aminollah Khormali ◽  
Arash Keshavarzi Arshadi ◽  
Julia Webb ◽  
Jiann-Shiun Yuan
Keyword(s):  
Virtual Screening ◽  
Transfer Learning ◽  
Fine Tuning ◽  
Superior Performance ◽  
Receiver Operating Curve ◽  
Target Task ◽  
Automatic Feature Extraction ◽  
Unseen Data ◽  
Anti Cancer ◽  
Multiple Challenges

Deep learning’s automatic feature extraction has proven its superior performance over traditional fingerprint-based features in the implementation of virtual screening models. However, these models face multiple challenges in the field of early drug discovery, such as over-training and generalization to unseen data, due to the inherently unbalanced and small datasets. In this work, the TranScreen pipeline is proposed, which utilizes transfer learning and a collection of weight initializations to overcome these challenges. An amount of 182 graph convolutional neural networks are trained on molecular source datasets and the learned knowledge is transferred to the target task for fine-tuning. The target task of p53-based bioactivity prediction, an important factor for anti-cancer discovery, is chosen to showcase the capability of the pipeline. Having trained a collection of source models, three different approaches are implemented to compare and rank them for a given task before fine-tuning. The results show improvement in performance of the model in multiple cases, with the best model increasing the area under receiver operating curve ROC-AUC from 0.75 to 0.91 and the recall from 0.25 to 1. This improvement is vital for practical virtual screening via lowering the false negatives and demonstrates the potential of transfer learning. The code and pre-trained models are made accessible online.

Weighted area/angle distortion minimization for Mesh Parameterization

2017 ◽  
Vol 34 (6) ◽  
pp. 1874-1895 ◽  
Author(s):  
Daniel Mejia ◽  
Diego A. Acosta ◽  
Oscar Ruiz-Salguero
Keyword(s):  
Sensitivity Analysis ◽  
Tool Path ◽  
Complexity Analysis ◽  
Fine Tuning ◽  
Superior Performance ◽  
Data Sets ◽  
Content Type ◽  
Mesh Parameterization ◽  
Distortion Minimization ◽  
Area Distortion

Purpose Mesh Parameterization is central to reverse engineering, tool path planning, etc. This work synthesizes parameterizations with un-constrained borders, overall minimum angle plus area distortion. This study aims to present an assessment of the sensitivity of the minimized distortion with respect to weighed area and angle distortions. Design/methodology/approach A Mesh Parameterization which does not constrain borders is implemented by performing: isometry maps for each triangle to the plane Z = 0; an affine transform within the plane Z = 0 to glue the triangles back together; and a Levenberg–Marquardt minimization algorithm of a nonlinear F penalty function that modifies the parameters of the first two transformations to discourage triangle flips, angle or area distortions. F is a convex weighed combination of area distortion (weight: α with 0 ≤ α ≤ 1) and angle distortion (weight: 1 − α). Findings The present study parameterization algorithm has linear complexity [𝒪(n), n = number of mesh vertices]. The sensitivity analysis permits a fine-tuning of the weight parameter which achieves overall bijective parameterizations in the studied cases. No theoretical guarantee is given in this manuscript for the bijectivity. This algorithm has equal or superior performance compared with the ABF, LSCM and ARAP algorithms for the Ball, Cow and Gargoyle data sets. Additional correct results of this algorithm alone are presented for the Foot, Fandisk and Sliced-Glove data sets. Originality/value The devised free boundary nonlinear Mesh Parameterization method does not require a valid initial parameterization and produces locally bijective parameterizations in all of our tests. A formal sensitivity analysis shows that the resulting parameterization is more stable, i.e. the UV mapping changes very little when the algorithm tries to preserve angles than when it tries to preserve areas. The algorithm presented in this study belongs to the class that parameterizes meshes with holes. This study presents the results of a complexity analysis comparing the present study algorithm with 12 competing ones.

scholarly journals Active Learning for Effectively Fine-Tuning Transfer Learning to Downstream Task

2021 ◽  
Vol 12 (2) ◽  
pp. 1-24
Author(s):  
Md Abul Bashar ◽  
Richi Nayak
Keyword(s):  
Active Learning ◽  
Transfer Learning ◽  
Language Processing ◽  
State Of The Art ◽  
Language Model ◽  
Ensemble Classifier ◽  
Classification Performance ◽  
Fine Tuning ◽  
Linguistic Features ◽  
Better Than

Language model (LM) has become a common method of transfer learning in Natural Language Processing (NLP) tasks when working with small labeled datasets. An LM is pretrained using an easily available large unlabelled text corpus and is fine-tuned with the labelled data to apply to the target (i.e., downstream) task. As an LM is designed to capture the linguistic aspects of semantics, it can be biased to linguistic features. We argue that exposing an LM model during fine-tuning to instances that capture diverse semantic aspects (e.g., topical, linguistic, semantic relations) present in the dataset will improve its performance on the underlying task. We propose a Mixed Aspect Sampling (MAS) framework to sample instances that capture different semantic aspects of the dataset and use the ensemble classifier to improve the classification performance. Experimental results show that MAS performs better than random sampling as well as the state-of-the-art active learning models to abuse detection tasks where it is hard to collect the labelled data for building an accurate classifier.

scholarly journals Enhancing Mouth-based Emotion Recognition using Transfer Learning

2020 ◽  
Author(s):  
Valentina Franzoni ◽  
Giulio Biondi ◽  
Damiano Perri ◽  
Osvaldo Gervasi
Keyword(s):  
Emotion Recognition ◽  
Transfer Learning ◽  
Large Data ◽  
Healthcare Management ◽  
Training Data ◽  
Fine Tuning ◽  
Data Sets ◽  
Critical Patients ◽  
Few Data ◽  
Full Face

The paper concludes the first research on mouth-based Emotion Recognition (ER), adopting a Transfer Learning (TL) approach. Transfer Learning results paramount for mouth-based emotion ER, because a few data sets are available, and most of them include emotional expressions simulated by actors, instead of adopting a real-world categorization. Using TL we can use fewer training data than training a whole network from scratch, thus more efficiently fine-tuning the network with emotional data and improving the convolutional neural network accuracy in the desired domain. The proposed approach aims at improving the Emotion Recognition dynamically, taking into account not only new scenarios but also modified situations with respect to the initial training phase, because the image of the mouth can be available even when the whole face is visible only in an unfavourable perspective. Typical applications include automated supervision of bedridden critical patients in an healthcare management environment, or portable applications supporting disabled users having difficulties in seeing or recognizing facial emotions. This work takes advantage from previous preliminary works on mouth-based emotion recognition using CNN deep-learning, and has the further benefit of testing and comparing a set of networks on large data sets for face-based emotion recognition well known in literature. The final result is not directly comparable with works on full-face ER, but valorizes the significance of mouth in emotion recognition, obtaining consistent performances on the visual emotion recognition domain.

scholarly journals Utilizing Indonesian Universal Language Model Fine-tuning for Text Classification

2021 ◽  
Vol 5 (3) ◽  
pp. 325
Author(s):  
Hendra Bunyamin
Keyword(s):  
Computer Vision ◽  
Natural Language Processing ◽  
Transfer Learning ◽  
Language Processing ◽  
Text Classification ◽  
Language Model ◽  
Fine Tuning ◽  
Classification Task ◽  
Universal Language ◽  
Learning Technique

Inductive transfer learning technique has made a huge impact on the computer vision field. Particularly, computer vision  applications including object detection, classification, and segmentation, are rarely trained from scratch; instead, they are fine-tuned from pretrained models, which are products of learning from huge datasets. In contrast to computer vision, state-of-the-art natural language processing models are still generally trained from the ground up. Accordingly, this research attempts to investigate an adoption of the transfer learning technique for natural language processing. Specifically, we utilize a transfer learning technique called Universal Language Model Fine-tuning (ULMFiT) for doing an Indonesian news text classification task. The dataset for constructing the language model is collected from several news providers from January to December 2017 whereas the dataset employed for text classification task comes from news articles provided by the Agency for the Assessment and Application of Technology (BPPT). To examine the impact of ULMFiT, we provide a baseline that is a vanilla neural network with two hidden layers. Although the performance of ULMFiT on validation set is lower than the one of our baseline, we find that the benefits of ULMFiT for the classification task significantly reduce the overfitting, that is the difference between train and validation accuracies from 4% to nearly zero.

scholarly journals A review of transfer learning for medical image classification

2021 ◽  
Author(s):  
Hee E. Kim ◽  
Alejandro Cosa-Linan ◽  
Mate E. Maros ◽  
Nandhini Santhanam ◽  
Mahboubeh Jannesari ◽  
...  
Keyword(s):  
Image Classification ◽  
Transfer Learning ◽  
Medical Image ◽  
Medical Image Analysis ◽  
Fine Tuning ◽  
Data Sets ◽  
Learning Approaches ◽  
Limited Data ◽  
Feature Extractor ◽  
Learning Types

Abstract This review paper provides an overview of the peer-reviewed articles using transfer learning for medical image analysis, while also providing guidelines for selecting a convolutional neural network model and its configurations for the image classification task. The data characteristics and the trend of models and transfer learning types in the medical domain are additionally analyzed. Publications were retrieved from the databases PubMed and Web of Science of peer-reviewed articles published in English until December 31, 2020. We followed the PRISMA guidelines for the paper selection and 121 studies were regarded as eligible for the scope of this review. With respect to the model, the majority of studies (n = 57) empirically evaluated numerous models followed by deep (n = 33) and shallow (n = 24) models. With respect to the transfer learning approaches, the majority of studies (n = 46) empirically searched for the optimal transfer learning configuration followed by feature extractor (n = 38) and fine-tuning scratch (n = 27), feature extractor hybrid (n = 7) and fine-tuning (n = 3). The investigated studies showed that transfer learning demonstrates either a better or at least a similar performance compared to medical experts despite the limited data sets. We hence encourage data scientists and practitioners to use models such as ResNet or Inception with a feature extractor approach, which saves computational costs and time without degrading the predictive power.

The importance of transfer learning in seismic modeling and imaging

Geophysics ◽  
2019 ◽  
Vol 84 (6) ◽  
pp. A47-A52 ◽  
Author(s):  
Ali Siahkoohi ◽  
Mathias Louboutin ◽  
Felix J. Herrmann
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Low Cost ◽  
Fine Tuning ◽  
Numerical Dispersion ◽  
High Fidelity ◽  
Velocity Models ◽  
Current Survey ◽  
Exploration Seismology ◽  
Fine Tune

Accurate forward modeling is essential for solving inverse problems in exploration seismology. Unfortunately, it is often not possible to afford being physically or numerically accurate. To overcome this conundrum, we make use of raw and processed data from nearby surveys. We have used these data, consisting of shot records or velocity models, to pretrain a neural network to correct for the effects of, for instance, the free surface or numerical dispersion, both of which can be considered as proxies for incomplete or inaccurate physics. Given this pretrained neural network, we apply transfer learning to fine-tune this pretrained neural network so it performs well on its task of mapping low-cost, but low-fidelity, solutions to high-fidelity solutions for the current survey. As long as we can limit ourselves during fine-tuning to using only a small fraction of high-fidelity data, we gain processing the current survey while using information from nearby surveys. We examined this principle by removing surface-related multiples and ghosts from shot records and the effects of numerical dispersion from migrated images and wave simulations.

Sign in / Sign up

Export Citation Format

Share Document