BeautyNet: Joint Multiscale CNN and Transfer Learning Method for Unconstrained Facial Beauty Prediction

Because of the lack of discriminative face representations and scarcity of labeled training data, facial beauty prediction (FBP), which aims at assessing facial attractiveness automatically, has become a challenging pattern recognition problem. Inspired by recent promising work on fine-grained image classification using the multiscale architecture to extend the diversity of deep features, BeautyNet for unconstrained facial beauty prediction is proposed in this paper. Firstly, a multiscale network is adopted to improve the discriminative of face features. Secondly, to alleviate the computational burden of the multiscale architecture, MFM (max-feature-map) is utilized as an activation function which can not only lighten the network and speed network convergence but also benefit the performance. Finally, transfer learning strategy is introduced here to mitigate the overfitting phenomenon which is caused by the scarcity of labeled facial beauty samples and improves the proposed BeautyNet’s performance. Extensive experiments performed on LSFBD demonstrate that the proposed scheme outperforms the state-of-the-art methods, which can achieve 67.48% classification accuracy.

Download Full-text

Improving Semi-Supervised Learning for Audio Classification with FixMatch

Electronics ◽

10.3390/electronics10151807 ◽

2021 ◽

Vol 10 (15) ◽

pp. 1807

Author(s):

Sascha Grollmisch ◽

Estefanía Cano

Keyword(s):

Neural Networks ◽

Supervised Learning ◽

Transfer Learning ◽

Data Transfer ◽

State Of The Art ◽

Training Data ◽

Audio Classification ◽

Image Domain ◽

Full Dataset ◽

Audio Data

Including unlabeled data in the training process of neural networks using Semi-Supervised Learning (SSL) has shown impressive results in the image domain, where state-of-the-art results were obtained with only a fraction of the labeled data. The commonality between recent SSL methods is that they strongly rely on the augmentation of unannotated data. This is vastly unexplored for audio data. In this work, SSL using the state-of-the-art FixMatch approach is evaluated on three audio classification tasks, including music, industrial sounds, and acoustic scenes. The performance of FixMatch is compared to Convolutional Neural Networks (CNN) trained from scratch, Transfer Learning, and SSL using the Mean Teacher approach. Additionally, a simple yet effective approach for selecting suitable augmentation methods for FixMatch is introduced. FixMatch with the proposed modifications always outperformed Mean Teacher and the CNNs trained from scratch. For the industrial sounds and music datasets, the CNN baseline performance using the full dataset was reached with less than 5% of the initial training data, demonstrating the potential of recent SSL methods for audio data. Transfer Learning outperformed FixMatch only for the most challenging dataset from acoustic scene classification, showing that there is still room for improvement.

Download Full-text

New polyp image classification technique using transfer learning of network-in-network structure in endoscopic images

Scientific Reports ◽

10.1038/s41598-021-83199-9 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Young Jae Kim ◽

Jang Pyo Bae ◽

Jun-Won Chung ◽

Dong Kyun Park ◽

Kwang Gi Kim ◽

...

Keyword(s):

Colorectal Cancer ◽

Transfer Learning ◽

Test Data ◽

State Of The Art ◽

Early Stage ◽

Statistical Significance ◽

Recall Rate ◽

Training Data ◽

Fine Tuning ◽

Accuracy Evaluation

AbstractWhile colorectal cancer is known to occur in the gastrointestinal tract. It is the third most common form of cancer of 27 major types of cancer in South Korea and worldwide. Colorectal polyps are known to increase the potential of developing colorectal cancer. Detected polyps need to be resected to reduce the risk of developing cancer. This research improved the performance of polyp classification through the fine-tuning of Network-in-Network (NIN) after applying a pre-trained model of the ImageNet database. Random shuffling is performed 20 times on 1000 colonoscopy images. Each set of data are divided into 800 images of training data and 200 images of test data. An accuracy evaluation is performed on 200 images of test data in 20 experiments. Three compared methods were constructed from AlexNet by transferring the weights trained by three different state-of-the-art databases. A normal AlexNet based method without transfer learning was also compared. The accuracy of the proposed method was higher in statistical significance than the accuracy of four other state-of-the-art methods, and showed an 18.9% improvement over the normal AlexNet based method. The area under the curve was approximately 0.930 ± 0.020, and the recall rate was 0.929 ± 0.029. An automatic algorithm can assist endoscopists in identifying polyps that are adenomatous by considering a high recall rate and accuracy. This system can enable the timely resection of polyps at an early stage.

Download Full-text

Relevance-guided Supervision for OpenQA with ColBERT

Transactions of the Association for Computational Linguistics ◽

10.1162/tacl_a_00405 ◽

2021 ◽

Vol 9 ◽

pp. 929-944

Author(s):

Omar Khattab ◽

Christopher Potts ◽

Matei Zaharia

Keyword(s):

Question Answering ◽

State Of The Art ◽

Training Data ◽

Coarse Grained ◽

Retrieval Model ◽

Open Domain ◽

Weak Supervision ◽

Fine Grained ◽

Vector Representations ◽

Large Corpus

Abstract Systems for Open-Domain Question Answering (OpenQA) generally depend on a retriever for finding candidate passages in a large corpus and a reader for extracting answers from those passages. In much recent work, the retriever is a learned component that uses coarse-grained vector representations of questions and passages. We argue that this modeling choice is insufficiently expressive for dealing with the complexity of natural language questions. To address this, we define ColBERT-QA, which adapts the scalable neural retrieval model ColBERT to OpenQA. ColBERT creates fine-grained interactions between questions and passages. We propose an efficient weak supervision strategy that iteratively uses ColBERT to create its own training data. This greatly improves OpenQA retrieval on Natural Questions, SQuAD, and TriviaQA, and the resulting system attains state-of-the-art extractive OpenQA performance on all three datasets.

Download Full-text

Iterative Metric Learning for Imbalance Data Classification

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2018/389 ◽

2018 ◽

Cited By ~ 1

Author(s):

Nan Wang ◽

Xibin Zhao ◽

Yu Jiang ◽

Yue Gao

Keyword(s):

Learning Strategy ◽

State Of The Art ◽

Metric Learning ◽

Training Data ◽

Imbalance Data ◽

Data Space ◽

Software Defect ◽

Training Samples ◽

Testing Data ◽

Data Program

In many classification applications, the amount of data from different categories usually vary significantly, such as software defect predication and medical diagnosis. Under such circumstances, it is essential to propose a proper method to solve the imbalance issue among the data. However, most of the existing methods mainly focus on improving the performance of classifiers rather than searching for an appropriate way to find an effective data space for classification. In this paper, we propose a method named Iterative Metric Learning (IML) to explore the correlations among imbalance data and construct an effective data space for classification. Given the imbalance training data, it is important to select a subset of training samples for each testing data. Thus, we aim to find a more stable neighborhood for testing data using the iterative metric learning strategy. To evaluate the effectiveness of the proposed method, we have conducted experiments on two groups of dataset, i.e., the NASA Metrics Data Program (NASA) dataset and UCI Machine Learning Repository (UCI) dataset. Experimental results and comparisons with state-of-the-art methods have exhibited better performance of our proposed method.

Download Full-text

Seismic trace interpolation for irregularly spatial sampled data using convolutional autoencoder

Geophysics ◽

10.1190/geo2018-0699.1 ◽

2020 ◽

Vol 85 (2) ◽

pp. V119-V130 ◽

Cited By ~ 2

Author(s):

Yingying Wang ◽

Benfeng Wang ◽

Ning Tu ◽

Jianhua Geng

Keyword(s):

Transfer Learning ◽

Field Data ◽

Learning Strategy ◽

Complete Data ◽

Training Data ◽

Sampled Data ◽

Data Set ◽

Network Training ◽

Convolutional Autoencoder ◽

Irregularly Sampled Data

Seismic trace interpolation is an important technique because irregular or insufficient sampling data along the spatial direction may lead to inevitable errors in multiple suppression, imaging, and inversion. Many interpolation methods have been studied for irregularly sampled data. Inspired by the working idea of the autoencoder and convolutional neural network, we have performed seismic trace interpolation by using the convolutional autoencoder (CAE). The irregularly sampled data are taken as corrupted data. By using a training data set including pairs of the corrupted and complete data, CAE can automatically learn to extract features from the corrupted data and reconstruct the complete data from the extracted features. It can avoid some assumptions in the traditional trace interpolation method such as the linearity of events, low-rankness, or sparsity. In addition, once the CAE network training is completed, the corrupted seismic data can be interpolated immediately with very low computational cost. A CAE network composed of three convolutional layers and three deconvolutional layers is designed to explore the capabilities of CAE-based seismic trace interpolation for an irregularly sampled data set. To solve the problem of rare complete shot gathers in field data applications, the trained network on synthetic data is used as an initialization of the network training on field data, called the transfer learning strategy. Experiments on synthetic and field data sets indicate the validity and flexibility of the trained CAE. Compared with the curvelet-transform-based method, CAE can lead to comparable or better interpolation performances efficiently. The transfer learning strategy enhances the training efficiency on field data and improves the interpolation performance of CAE with limited training data.

Download Full-text

Learning from Web Data Using Adversarial Discriminative Neural Networks for Fine-Grained Classification

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.3301273 ◽

2019 ◽

Vol 33 ◽

pp. 273-280 ◽

Cited By ~ 2

Author(s):

Xiaoxiao Sun ◽

Liyi Chen ◽

Jufeng Yang

Keyword(s):

Neural Networks ◽

Large Scale ◽

State Of The Art ◽

Training Data ◽

Web Data ◽

Fine Grained ◽

Learning Framework ◽

Attractive Option ◽

Public Datasets ◽

Noisy Labels

Fine-grained classification is absorbed in recognizing the subordinate categories of one field, which need a large number of labeled images, while it is expensive to label these images. Utilizing web data has been an attractive option to meet the demands of training data for convolutional neural networks (CNNs), especially when the well-labeled data is not enough. However, directly training on such easily obtained images often leads to unsatisfactory performance due to factors such as noisy labels. This has been conventionally addressed by reducing the noise level of web data. In this paper, we take a fundamentally different view and propose an adversarial discriminative loss to advocate representation coherence between standard and web data. This is further encapsulated in a simple, scalable and end-to-end trainable multi-task learning framework. We experiment on three public datasets using large-scale web data to evaluate the effectiveness and generalizability of the proposed approach. Extensive experiments demonstrate that our approach performs favorably against the state-of-the-art methods.

Download Full-text

Dialog State Tracking with Reinforced Data Augmentation

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6491 ◽

2020 ◽

Vol 34 (05) ◽

pp. 9474-9481

Author(s):

Yichun Yin ◽

Lifeng Shang ◽

Xin Jiang ◽

Xiao Chen ◽

Qun Liu

Keyword(s):

Data Augmentation ◽

State Of The Art ◽

The State ◽

Training Data ◽

Quality Data ◽

Specific Context ◽

High Quality ◽

High Quality Data ◽

Fine Grained ◽

State Tracking

Neural dialog state trackers are generally limited due to the lack of quantity and diversity of annotated training data. In this paper, we address this difficulty by proposing a reinforcement learning (RL) based framework for data augmentation that can generate high-quality data to improve the neural state tracker. Specifically, we introduce a novel contextual bandit generator to learn fine-grained augmentation policies that can generate new effective instances by choosing suitable replacements for specific context. Moreover, by alternately learning between the generator and the state tracker, we can keep refining the generative policies to generate more high-quality training data for neural state tracker. Experimental results on the WoZ and MultiWoZ (restaurant) datasets demonstrate that the proposed framework significantly improves the performance over the state-of-the-art models, especially with limited training data.

Download Full-text

Recognizing More Emotions with Less Data Using Self-supervised Transfer Learning

10.20944/preprints202008.0645.v1 ◽

2020 ◽

Author(s):

Jonathan Boigne ◽

Biman Liyanage ◽

Ted Östrem

Keyword(s):

Neural Network ◽

Emotion Recognition ◽

Transfer Learning ◽

Network Performance ◽

State Of The Art ◽

Research Community ◽

Training Data ◽

Linguistic Knowledge ◽

Speech Emotion Recognition ◽

Learning Method

We propose a novel transfer learning method for speech emotion recognition allowing us to obtain promising results when only few training data is available. With as low as 125 examples per emotion class, we were able to reach a higher accuracy than a strong baseline trained on 8 times more data. Our method leverages knowledge contained in pre-trained speech representations extracted from models trained on a more general self-supervised task which doesn’t require human annotations, such as the wav2vec model. We provide detailed insights on the benefits of our approach by varying the training data size, which can help labeling teams to work more efficiently. We compare performance with other popular methods on the IEMOCAP dataset, a well-benchmarked dataset among the Speech Emotion Recognition (SER) research community. Furthermore, we demonstrate that results can be greatly improved by combining acoustic and linguistic knowledge from transfer learning. We align acoustic pre-trained representations with semantic representations from the BERT model through an attention-based recurrent neural network. Performance improves significantly when combining both modalities and scales with the amount of data. When trained on the full IEMOCAP dataset, we reach a new state-of-the-art of 73.9% unweighted accuracy (UA).

Download Full-text

Transfer Optimization in Accelerating the Design of Turbomachinery Cascades

Volume 2D: Turbomachinery ◽

10.1115/gt2020-16321 ◽

2020 ◽

Author(s):

Qineng Wang ◽

Liming Song ◽

Zhendong Guo ◽

Jun Li

Keyword(s):

Transfer Learning ◽

Negative Transfer ◽

Learning Strategy ◽

State Of The Art ◽

Computational Cost ◽

Expected Improvement ◽

Optimization Framework ◽

The Past ◽

Past Experiences ◽

Turbomachinery Cascades

Abstract This paper draws motivation from the fact that engineering optimizations were mostly carried out from scratch. In contrast, however, humans routinely take advantage of the knowledge from past experiences whenever a new task is met. Such a transfer learning process by leveraging knowledge from already completed tasks can be promising to significantly improve the performance of current state-of-the-art algorithms, particularly in solving expensive black-box problems. In light of the above, we propose a Cokriging based transfer optimization framework for the design of turbomachinery cascades, which is demonstrated by optimization to re-design the first-stage vane of GEE3. Specifically, when building Cokriging surrogate in such a transfer optimization context, the samples from already completed tasks are treated as low-fidelity (LF) data. The acquisition function of expected improvement is adopted to guide the search for high-fidelity (HF) data. In order to make full use of the “past experiences”, one of our efforts was drawn to designing selection strategies of initial HF samples. In addition, as the “past experiences” may do harm to the optimization of the target task, the correlation coefficients between source and target tasks in each optimization process were calculated to avoid “negative transfer”. The test results show that, by learning from the past, the transfer optimization framework can reduce the computational cost by much as 50%. More importantly, our proposed transfer learning strategy can effectively avoid “negative transfer” and thus always achieve better solutions than the compared state-of-the-art algorithms.

Download Full-text

Data Augmentation and Transfer Learning Strategies for Reaction Prediction in Low Chemical Data Regimes

10.26434/chemrxiv.13383275 ◽

2020 ◽

Author(s):

Yun Zhang ◽

Ling Wang ◽

Xinqiao Wang ◽

Chengyun Zhang ◽

Jiamin Ge ◽

...

Keyword(s):

Deep Learning ◽

Learning Strategies ◽

Transfer Learning ◽

Data Augmentation ◽

Learning Strategy ◽

Predictive Performance ◽

Learning Model ◽

Training Data ◽

Reaction Prediction ◽

Transformer Model

<p><b>Abstract:</b> Effective and rapid deep learning method to predict chemical reactions contributes to the research and development of organic chemistry and drug discovery. Despite the outstanding capability of deep learning in retrosynthesis and forward synthesis, predictions based on small chemical datasets generally result in low accuracy due to an insufficiency of reaction examples. Here, we introduce a new state art of method, which integrates transfer learning with transformer model to predict the outcomes of the Baeyer-Villiger reaction which is a representative small dataset reaction. The results demonstrate that introducing transfer learning strategy markedly improves the top-1 accuracy of the transformer-transfer learning model (81.8%) over that of the transformer-baseline model (58.4%). Moreover, we further introduce data augmentation to the input reaction SMILES, which allows for better performance and improves the accuracy of the transformer-transfer learning model (86.7%). In summary, both transfer learning and data augmentation methods significantly improve the predictive performance of transformer model, which are powerful methods used in chemistry field to eliminate the restriction of limited training data.</p>

Download Full-text