scholarly journals Investigation of text data augmentation for transformer training via translation technique

2021 ◽  
pp. 97-105
Author(s):  
Dominykas Šeputis
Keyword(s):  
Foreign Language ◽  
Data Augmentation ◽  
Experimental Results ◽  
Text Data ◽  
Translation Technique

Data augmentation can improve model’s final accuracy by introducing new data samples to the dataset. In this paper, text data augmentation using translation technique is investigated. Synthetic translations, generated by Opus-MT model are compared to the unique foreign data samples in terms of an impact to the trans- former network-based models’ performance. The experimental results showed that multilingual models like DistilBERT in some cases benefit from the introduction of the addition artificially created data samples presented in a foreign language.

scholarly journals Text Data Augmentation for Deep Learning

2021 ◽  
Author(s):  
Connor Shorten ◽  
Taghi M. Khoshgoftaar ◽  
Borko Furht
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Data Augmentation ◽  
Early Stage ◽  
Practical Implementation ◽  
Text Data ◽  
Training Strategy ◽  
Local Decision ◽  
Decision Boundaries

Abstract Natural Language Processing (NLP) is one of the most captivating applications of Deep Learning. In this survey, we consider how the Data Augmentation training strategy can aid in its development. We begin with the major motifs of Data Augmentation summarized into strengthening local decision boundaries, brute force training, causality and counterfactual examples, and the distinction between meaning and form. We follow these motifs with a concrete list of augmentation frameworks that have been developed for text data. Deep Learning generally struggles with the measurement of generalization and characterization of overfitting. We highlight studies that cover how augmentations can construct test sets for generalization. NLP is at an early stage in applying Data Augmentation compared to Computer Vision. We highlight the key differences and promising ideas that have yet to be tested in NLP. For the sake of practical implementation, we describe tools that facilitate Data Augmentation such as the use of consistency regularization, controllers, and offline and online augmentation pipelines, to preview a few. Finally, we discuss interesting topics around Data Augmentation in NLP such as task-specific augmentations, the use of prior knowledge in self-supervised learning versus Data Augmentation, intersections with transfer and multi-task learning, and ideas for AI-GAs (AI-Generating Algorithms). We hope this paper inspires further research interest in Text Data Augmentation.

Toward a Better Text Data Augmentation via Filtering and Transforming Augmented Instances

2021 ◽  
pp. 198-210
Author(s):  
Fei Xia ◽  
Shizhu He ◽  
Kang Liu ◽  
Shengping Liu ◽  
Jun Zhao
Keyword(s):  
Data Augmentation ◽  
Text Data

scholarly journals Parkinson’s Disease EMG Data Augmentation and Simulation with DCGANs and Style Transfer

Sensors ◽  
2020 ◽  
Vol 20 (9) ◽  
pp. 2605 ◽  
Author(s):  
Rafael Anicet Zanini ◽  
Esther Luna Colombini
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Data Augmentation ◽  
Experimental Results ◽  
Generative Adversarial Networks ◽  
Style Transfer ◽  
Treatment Approaches ◽  
Adversarial Networks

This paper proposes two new data augmentation approaches based on Deep Convolutional Generative Adversarial Networks (DCGANs) and Style Transfer for augmenting Parkinson’s Disease (PD) electromyography (EMG) signals. The experimental results indicate that the proposed models can adapt to different frequencies and amplitudes of tremor, simulating each patient’s tremor patterns and extending them to different sets of movement protocols. Therefore, one could use these models for extending the existing patient dataset and generating tremor simulations for validating treatment approaches on different movement scenarios.

scholarly journals Vision-Based Classification of Mosquito Species: Comparison of Conventional and Deep Learning Methods

2019 ◽  
Vol 9 (18) ◽  
pp. 3935 ◽  
Author(s):  
Kazushige Okayasu ◽  
Kota Yoshida ◽  
Masataka Fuchida ◽  
Akio Nakamura
Keyword(s):  
Deep Learning ◽  
Conventional Method ◽  
Data Augmentation ◽  
Mosquito Species ◽  
Experimental Results ◽  
Support Vector ◽  
Learning Method ◽  
Species Classification ◽  
Scale Invariant ◽  
Machine Method

This study aims to propose a vision-based method to classify mosquito species. To investigate the efficiency of the method, we compared two different classification methods: The handcraft feature-based conventional method and the convolutional neural network-based deep learning method. For the conventional method, 12 types of features were adopted for handcraft feature extraction, while a support vector machine method was adopted for classification. For the deep learning method, three types of architectures were adopted for classification. We built a mosquito image dataset, which included 14,400 images with three types of mosquito species. The dataset comprised 12,000 images for training, 1500 images for testing, and 900 images for validating. Experimental results revealed that the accuracy of the conventional method using the scale-invariant feature transform algorithm was 82.4% at maximum, whereas the accuracy of the deep learning method was 95.5% in a residual network using data augmentation. From the experimental results, deep learning can be considered to be effective for classifying the mosquito species of the proposed dataset. Furthermore, data augmentation improves the accuracy of mosquito species’ classification.

scholarly journals Two Birds with One Stone: Series Saliency for Accurate and Interpretable Multivariate Time Series Forecasting

2021 ◽  
Author(s):  
Qingyi Pan ◽  
Wenbo Hu ◽  
Ning Chen
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Data Augmentation ◽  
Multivariate Time Series ◽  
Time Series Forecasting ◽  
Experimental Results ◽  
Learning Methods ◽  
Forecasting Accuracy ◽  
Sliding Windows ◽  
Scheme Of Series

It is important yet challenging to perform accurate and interpretable time series forecasting. Though deep learning methods can boost forecasting accuracy, they often sacrifice interpretability. In this paper, we present a new scheme of series saliency to boost both accuracy and interpretability. By extracting series images from sliding windows of the time series, we design series saliency as a mixup strategy with a learnable mask between the series images and their perturbed versions. Series saliency is model agnostic and performs as an adaptive data augmentation method for training deep models. Moreover, by slightly changing the objective, we optimize series saliency to find a mask for interpretable forecasting in both feature and time dimensions. Experimental results on several real datasets demonstrate that series saliency is effective to produce accurate time-series forecasting results as well as generate temporal interpretations.

scholarly journals Efficient Deep Learning Architecture for Detection and Recognition of Thyroid Nodules

2020 ◽  
Vol 2020 ◽  
pp. 1-15
Author(s):  
Jingzhe Ma ◽  
Shaobo Duan ◽  
Ye Zhang ◽  
Jing Wang ◽  
Zongmin Wang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Thyroid Nodules ◽  
Data Augmentation ◽  
Experimental Results ◽  
Local Health ◽  
Ultrasound Images ◽  
Detection Time ◽  
Average Precision ◽  
Medical Image Diagnosis ◽  
Detection And Recognition

Ultrasonography is widely used in the clinical diagnosis of thyroid nodules. Ultrasound images of thyroid nodules have different appearances, interior features, and blurred borders that are difficult for a physician to diagnose into malignant or benign types merely through visual recognition. The development of artificial intelligence, especially deep learning, has led to great advances in the field of medical image diagnosis. However, there are some challenges to achieve precision and efficiency in the recognition of thyroid nodules. In this work, we propose a deep learning architecture, you only look once v3 dense multireceptive fields convolutional neural network (YOLOv3-DMRF), based on YOLOv3. It comprises a DMRF-CNN and multiscale detection layers. In DMRF-CNN, we integrate dilated convolution with different dilation rates to continue passing the edge and the texture features to deeper layers. Two different scale detection layers are deployed to recognize the different sizes of the thyroid nodules. We used two datasets to train and evaluate the YOLOv3-DMRF during the experiments. One dataset includes 699 original ultrasound images of thyroid nodules collected from a local health physical center. We obtained 10,485 images after data augmentation. Another dataset is an open-access dataset that includes ultrasound images of 111 malignant and 41 benign thyroid nodules. Average precision (AP) and mean average precision (mAP) are used as the metrics for quantitative and qualitative evaluations. We compared the proposed YOLOv3-DMRF with some state-of-the-art deep learning networks. The experimental results show that YOLOv3-DMRF outperforms others on mAP and detection time on both the datasets. Specifically, the values of mAP and detection time were 90.05 and 95.23% and 3.7 and 2.2 s, respectively, on the two test datasets. Experimental results demonstrate that the proposed YOLOv3-DMRF is efficient for detection and recognition of thyroid nodules for ultrasound images.

scholarly journals The Automated VSMs to Categorize Arabic Text Data Sets

2014 ◽  
Vol 13 (1) ◽  
pp. 4074-4081
Author(s):  
Mamoun Suleiman Al Rababaa ◽  
Essam Said Hanandeh
Keyword(s):  
Data Mining ◽  
Information Retrieval ◽  
Vector Space ◽  
Text Categorization ◽  
Experimental Results ◽  
Data Sets ◽  
Arabic Text ◽  
Text Data ◽  
Evaluation Measures ◽  
Vector Space Models

Text Categorization is one of the most important tasks in information retrieval and data mining. This paper aims at investigating different variations of vector space models (VSMs) using KNN algorithm. we used 242 Arabic abstract documents that were used by (Hmeidi & Kanaan, 1997). The bases of our comparison are the most popular text evaluation measures; we use Recall measure, Precision measure, and F1 measure. The Experimental results against the Saudi data sets reveal that Cosine outperformed over of the Dice and Jaccard coefficients.

Sign in / Sign up

Export Citation Format

Share Document