Cross-Domain Text Sentiment Analysis Based on CNN_FT Method

Transfer learning is one of the popular methods for solving the problem that the models built on the source domain cannot be directly applied to the target domain in the cross-domain sentiment classification. This paper proposes a transfer learning method based on the multi-layer convolutional neural network (CNN). Interestingly, we construct a convolutional neural network model to extract features from the source domain and share the weights in the convolutional layer and the pooling layer between the source and target domain samples. Next, we fine-tune the weights in the last layer, named the fully connected layer, and transfer the models from the source domain to the target domain. Comparing with the classical transfer learning methods, the method proposed in this paper does not need to retrain the network for the target domain. The experimental evaluation of the cross-domain data set shows that the proposed method achieves a relatively good performance.

Download Full-text

Multi-Source Deep Transfer Neural Network Algorithm

Sensors ◽

10.3390/s19183992 ◽

2019 ◽

Vol 19 (18) ◽

pp. 3992 ◽

Cited By ~ 2

Author(s):

Jingmei Li ◽

Weifei Wu ◽

Di Xue ◽

Peng Gao

Keyword(s):

Neural Network ◽

Probability Distribution ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Classification Performance ◽

Classification Error ◽

Target Domain ◽

Source Domain ◽

Network Algorithm ◽

Neural Network Algorithm

Transfer learning can enhance classification performance of a target domain with insufficient training data by utilizing knowledge relating to the target domain from source domain. Nowadays, it is common to see two or more source domains available for knowledge transfer, which can improve performance of learning tasks in the target domain. However, the classification performance of the target domain decreases due to mismatching of probability distribution. Recent studies have shown that deep learning can build deep structures by extracting more effective features to resist the mismatching. In this paper, we propose a new multi-source deep transfer neural network algorithm, MultiDTNN, based on convolutional neural network and multi-source transfer learning. In MultiDTNN, joint probability distribution adaptation (JPDA) is used for reducing the mismatching between source and target domains to enhance features transferability of the source domain in deep neural networks. Then, the convolutional neural network is trained by utilizing the datasets of each source and target domain to obtain a set of classifiers. Finally, the designed selection strategy selects classifier with the smallest classification error on the target domain from the set to assemble the MultiDTNN framework. The effectiveness of the proposed MultiDTNN is verified by comparing it with other state-of-the-art deep transfer learning on three datasets.

Download Full-text

Sentiment Classification for Financial Texts Based on Deep Learning

Computational Intelligence and Neuroscience ◽

10.1155/2021/9524705 ◽

2021 ◽

Vol 2021 ◽

pp. 1-9

Author(s):

Shanshan Dong ◽

Chang Liu

Keyword(s):

Neural Network ◽

Deep Learning ◽

Transfer Learning ◽

Loss Function ◽

Classification Accuracy ◽

Sentiment Classification ◽

Target Domain ◽

Text Data ◽

Source Domain ◽

Cross Domain

Sentiment classification for financial texts is of great importance for predicting stock markets and financial crises. At present, with the popularity of applications in the field of natural language processing (NLP) adopting deep learning, the application of automatic text classification and text-based sentiment classification has become more and more extensive. However, in the field of financial text-based sentiment classification, due to a lack of labeled samples, such applications are limited. A domain-adaptation-based financial text sentiment classification method is proposed in this paper, which can adopt source domain (SD) text data with sentiment labels and a large amount of unlabeled target domain (TD) financial text data as training samples for the proposed neural network. The proposed method is a cross-domain transfer-learning-based method. The domain classification subnetwork is added to the original neural network, and the domain classification loss function is also added to the original training loss function. Therefore, the network can simultaneously adapt to the target domain and then accomplish the classification task. The experiment of the proposed sentiment classification transfer learning method is carried out through an open-source dataset. The proposed method in this paper uses the reviews of Amazon Books, DVDs, electronics, and kitchen appliances as the source domain for cross-domain learning, and the classification accuracy rates can reach 65.0%, 61.2%, 61.6%, and 66.3%, respectively. Compared with nontransfer learning, the classification accuracy rate has improved by 11.0%, 7.6%, 11.4%, and 13.4%, respectively.

Download Full-text

Convolutional Neural Network Coupled with a Transfer-Learning Approach for Time-Series Flood Predictions

Water ◽

10.3390/w12010096 ◽

2019 ◽

Vol 12 (1) ◽

pp. 96 ◽

Cited By ~ 5

Author(s):

Nobuaki Kimura ◽

Ikuo Yoshinaga ◽

Kenji Sekijima ◽

Issaku Azechi ◽

Daichi Baba

Keyword(s):

Neural Network ◽

Neural Networks ◽

Time Series ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Learning Approach ◽

Target Domain ◽

Training Time ◽

Source Domain ◽

Flood Model

East Asian regions in the North Pacific have recently experienced severe riverine flood disasters. State-of-the-art neural networks are currently utilized as a quick-response flood model. Neural networks typically require ample time in the training process because of the use of numerous datasets. To reduce the computational costs, we introduced a transfer-learning approach to a neural-network-based flood model. For a concept of transfer leaning, once the model is pretrained in a source domain with large datasets, it can be reused in other target domains. After retraining parts of the model with the target domain datasets, the training time can be reduced due to reuse. A convolutional neural network (CNN) was employed because the CNN with transfer learning has numerous successful applications in two-dimensional image classification. However, our flood model predicts time-series variables (e.g., water level). The CNN with transfer learning requires a conversion tool from time-series datasets to image datasets in preprocessing. First, the CNN time-series classification was verified in the source domain with less than 10% errors for the variation in water level. Second, the CNN with transfer learning in the target domain efficiently reduced the training time by 1/5 of and a mean error difference by 15% of those obtained by the CNN without transfer learning, respectively. Our method can provide another novel flood model in addition to physical-based models.

Download Full-text

Cross-domain and multi-task transfer learning of deep convolutional neural network for breast cancer diagnosis in digital breast tomosynthesis

Medical Imaging 2018: Computer-Aided Diagnosis ◽

10.1117/12.2293412 ◽

2018 ◽

Cited By ~ 5

Author(s):

Ravi K. Samala ◽

Heang-Ping Chan ◽

Lubomir M. Hadjiiski ◽

Mark A. Helvie ◽

Caleb Richter ◽

...

Keyword(s):

Breast Cancer ◽

Neural Network ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Cancer Diagnosis ◽

Digital Breast Tomosynthesis ◽

Breast Cancer Diagnosis ◽

Breast Tomosynthesis ◽

Cross Domain ◽

Task Transfer

Download Full-text

Detection of Epileptic Seizure Using Pretrained Deep Convolutional Neural Network and Transfer Learning

European Neurology ◽

10.1159/000512985 ◽

2020 ◽

Vol 83 (6) ◽

pp. 602-614

Author(s):

Hidir Selcuk Nogay ◽

Hojjat Adeli

Keyword(s):

Neural Network ◽

Machine Learning ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Epileptic Seizure ◽

Binary Classification ◽

Classification Model ◽

Automatic Identification ◽

Eeg Signal ◽

Data Set

Introduction: The diagnosis of epilepsy takes a certain process, depending entirely on the attending physician. However, the human factor may cause erroneous diagnosis in the analysis of the EEG signal. In the past 2 decades, many advanced signal processing and machine learning methods have been developed for the detection of epileptic seizures. However, many of these methods require large data sets and complex operations. Methods: In this study, an end-to-end machine learning model is presented for detection of epileptic seizure using the pretrained deep two-dimensional convolutional neural network (CNN) and the concept of transfer learning. The EEG signal is converted directly into visual data with a spectrogram and used directly as input data. Results: The authors analyzed the results of the training of the proposed pretrained AlexNet CNN model. Both binary and ternary classifications were performed without any extra procedure such as feature extraction. By performing data set creation from short-term spectrogram graphic images, the authors were able to achieve 100% accuracy for binary classification for epileptic seizure detection and 100% for ternary classification. Discussion/Conclusion: The proposed automatic identification and classification model can help in the early diagnosis of epilepsy, thus providing the opportunity for effective early treatment.

Download Full-text

Classification of Plant Leaf Diseases Based on Improved Convolutional Neural Network

Sensors ◽

10.3390/s19194161 ◽

2019 ◽

Vol 19 (19) ◽

pp. 4161 ◽

Cited By ~ 11

Author(s):

Hang ◽

Zhang ◽

Chen ◽

Zhang ◽

Wang

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Convergence Time ◽

Model Parameters ◽

Data Set ◽

Training Time ◽

Plant Leaf ◽

The Neural Network ◽

Target Disease ◽

Fully Connected

Plant leaf diseases are closely related to people's daily life. Due to the wide variety of diseases, it is not only time-consuming and labor-intensive to identify and classify diseases by artificial eyes, but also easy to be misidentified with having a high error rate. Therefore, we proposed a deep learning-based method to identify and classify plant leaf diseases. The proposed method can take the advantages of the neural network to extract the characteristics of diseased parts, and thus to classify target disease areas. To address the issues of long training convergence time and too-large model parameters, the traditional convolutional neural network was improved by combining a structure of inception module, a squeeze-and-excitation (SE) module and a global pooling layer to identify diseases. Through the Inception structure, the feature data of the convolutional layer were fused in multi-scales to improve the accuracy on the leaf disease dataset. Finally, the global average pooling layer was used instead of the fully connected layer to reduce the number of model parameters. Compared with some traditional convolutional neural networks, our model yielded better performance and achieved an accuracy of 91.7% on the test data set. At the same time, the number of model parameters and training time have also been greatly reduced. The experimental classification on plant leaf diseases indicated that our method is feasible and effective.

Download Full-text

A novel method for peanut variety identification and classification by Improved VGG16

Scientific Reports ◽

10.1038/s41598-021-95240-y ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Haoyan Yang ◽

Jiangong Ni ◽

Jiyue Gao ◽

Zhongzhi Han ◽

Tao Luan

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Fine Tuning ◽

Variety Identification ◽

Learning Technology ◽

Data Set ◽

Average Accuracy ◽

Classical Models ◽

Crop Variety ◽

Fully Connected

AbstractCrop variety identification is an essential link in seed detection, phenotype collection and scientific breeding. This paper takes peanut as an example to explore a new method for crop variety identification. Peanut is a crucial oil crop and cash crop. The yield and quality of different peanut varieties are different, so it is necessary to identify and classify different peanut varieties. The traditional image processing method of peanut variety identification needs to extract many features, which has defects such as intense subjectivity and insufficient generalization ability. Based on the deep learning technology, this paper improved the deep convolutional neural network VGG16 and applied the improved VGG16 to the identification and classification task of 12 varieties of peanuts. Firstly, the peanut pod images of 12 varieties obtained by the scanner were preprocessed with gray-scale, binarization, and ROI extraction to form a peanut pod data set with a total of 3365 images of 12 varieties. A series of improvements have been made to VGG16. Remove the F6 and F7 fully connected layers of VGG16. Add Conv6 and Global Average Pooling Layer. The three convolutional layers of conv5 have changed into Depth Concatenation and add the Batch Normalization(BN) layers to the model. Besides, fine-tuning is carried out based on the improved VGG16. We adjusted the location of the BN layers. Adjust the number of filters for Conv6. Finally, the improved VGG16 model's training test results were compared with the other classic models, AlexNet, VGG16, GoogLeNet, ResNet18, ResNet50, SqueezeNet, DenseNet201 and MobileNetv2 verify its superiority. The average accuracy of the improved VGG16 model on the peanut pods test set was 96.7%, which was 8.9% higher than that of VGG16, and 1.6–12.3% higher than that of other classical models. Besides, supplementary experiments were carried out to prove the robustness and generality of the improved VGG16. The improved VGG16 was applied to the identification and classification of seven corn grain varieties with the same method and an average accuracy of 90.1% was achieved. The experimental results show that the improved VGG16 proposed in this paper can identify and classify peanut pods of different varieties, proving the feasibility of a convolutional neural network in variety identification and classification. The model proposed in this experiment has a positive significance for exploring other Crop variety identification and classification.

Download Full-text

Airborne Network Traffic Identification Method under Small Training Samples

Xibei Gongye Daxue Xuebao/Journal of Northwestern Polytechnical University ◽

10.1051/jnwpu/20203851129 ◽

2020 ◽

Vol 38 (5) ◽

pp. 1129-1137

Author(s):

Na Lyu ◽

Jiaxin Zhou ◽

Zhuo Chen ◽

Wu Chen

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Network Traffic ◽

Learning Algorithm ◽

Small Sample ◽

Fine Tuning ◽

Target Domain ◽

Data Set ◽

Traffic Identification ◽

Identification Method

Due to the high cost and difficulty of traffic data set acquisition and the high time sensitivity of traffic distribution, the machine learning-based traffic identification method is difficult to be applied in airborne network environment. Aiming at this problem, a method for airborne network traffic identification based on the convolutional neural network under small traffic samples is proposed. Firstly, the pre-training of the initial model for the convolutional neural network is implemented based on the complete data set in source domain, and then the retraining of the convolutional neural network is realized through the layer frozen based fine-tuning learning algorithm of convolutional neural network on the incomplete dataset in target domain, and the convolutional neural network model based feature representing transferring(FRT-CNN) is constructed to realize online traffic identification. The experiment results on the actual airborne network traffic dataset show that the proposed method can guarantee the accuracy of traffic identification under limited traffic samples, and the classification performance is significantly improved comparing with the existing small-sample learning methods.

Download Full-text

Deep Transfer Learning for Biology Cross-Domain Image Classification

Journal of Control Science and Engineering ◽

10.1155/2021/2518837 ◽

2021 ◽

Vol 2021 ◽

pp. 1-19

Author(s):

Chunfeng Guo ◽

Bin Wei ◽

Kun Yu

Keyword(s):

Image Classification ◽

Transfer Learning ◽

Poor Performance ◽

Target Domain ◽

Deep Convolutional Neural Networks ◽

Specific Domain ◽

Source Domain ◽

Cross Domain ◽

Domain Transfer ◽

Domain Transfer Learning

Automatic biology image classification is essential for biodiversity conservation and ecological study. Recently, due to the record-shattering performance, deep convolutional neural networks (DCNNs) have been used more often in biology image classification. However, training DCNNs requires a large amount of labeled data, which may be difficult to collect for some organisms. This study was carried out to exploit cross-domain transfer learning for DCNNs with limited data. According to the literature, previous studies mainly focus on transferring from ImageNet to a specific domain or transferring between two closely related domains. While this study explores deep transfer learning between species from different domains and analyzes the situation when there is a huge difference between the source domain and the target domain. Inspired by the analysis of previous studies, the effect of biology cross-domain image classification in transfer learning is proposed. In this work, the multiple transfer learning scheme is designed to exploit deep transfer learning on several biology image datasets from different domains. There may be a huge difference between the source domain and the target domain, causing poor performance on transfer learning. To address this problem, multistage transfer learning is proposed by introducing an intermediate domain. The experimental results show the effectiveness of cross-domain transfer learning and the importance of data amount and validate the potential of multistage transfer learning.

Download Full-text

Automatic velocity analysis using convolutional neural network and transfer learning

Geophysics ◽

10.1190/geo2018-0870.1 ◽

2019 ◽

Vol 85 (1) ◽

pp. V33-V43 ◽

Cited By ~ 4

Author(s):

Min Jun Park ◽

Mauricio D. Sacchi

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Input Data ◽

Synthetic Data ◽

Velocity Model ◽

Training Data ◽

Velocity Analysis ◽

Time Step ◽

Data Set

Velocity analysis can be a time-consuming task when performed manually. Methods have been proposed to automate the process of velocity analysis, which, however, typically requires significant manual effort. We have developed a convolutional neural network (CNN) to estimate stacking velocities directly from the semblance. Our CNN model uses two images as one input data for training. One is an entire semblance (guide image), and the other is a small patch (target image) extracted from the semblance at a specific time step. Labels for each input data set are the root mean square velocities. We generate the training data set using synthetic data. After training the CNN model with synthetic data, we test the trained model with another synthetic data that were not used in the training step. The results indicate that the model can predict a consistent velocity model. We also noticed that when the input data are extremely different from those used for the training, the CNN model will hardly pick the correct velocities. In this case, we adopt transfer learning to update the trained model (base model) with a small portion of the target data to improve the accuracy of the predicted velocity model. A marine data set from the Gulf of Mexico is used for validating our new model. The updated model performed a reasonable velocity analysis in seconds.

Download Full-text