scholarly journals Physically-informed Data-driven Deep Learning and Prospect for Transfer Learning in Materials Informatics

2021 ◽  
Vol 28 (1) ◽  
pp. 28-55
Author(s):  
Masashi Tsubaki
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Data Driven ◽  
Materials Informatics

scholarly journals A Novel Hybrid Signal Decomposition Technique for Transfer Learning Based Industrial Fault Diagnosis

2021 ◽  
Vol 5 (4) ◽  
pp. 37-53
Author(s):  
Zurana Mehrin Ruhi ◽  
Sigma Jahan ◽  
Jia Uddin
Keyword(s):  
Deep Learning ◽  
Fault Diagnosis ◽  
Transfer Learning ◽  
Industrial Revolution ◽  
State Of The Art ◽  
Signal Decomposition ◽  
Data Driven ◽  
Detection Model ◽  
Mode Decomposition ◽  
To Come

In the fourth industrial revolution, data-driven intelligent fault diagnosis for industrial purposes serves a crucial role. In contemporary times, although deep learning is a popular approach for fault diagnosis, it requires massive amounts of labelled samples for training, which is arduous to come by in the real world. Our contribution to introduce a novel comprehensive intelligent fault detection model using the Case Western Reserve University dataset is divided into two steps. Firstly, a new hybrid signal decomposition methodology is developed comprising Empirical Mode Decomposition and Variational Mode Decomposition to leverage signal information from both processes for effective feature extraction. Secondly, transfer learning with DenseNet121 is employed to alleviate the constraints of deep learning models. Finally, our proposed novel technique surpassed not only previous outcomes but also generated state-of-the-art outcomes represented via the F1 score.

A new dam structural response estimation paradigm powered by deep learning and transfer learning techniques

2021 ◽  
pp. 147592172110097
Author(s):  
Yangtao Li ◽  
Tengfei Bao ◽  
Zhixin Gao ◽  
Xiaosong Shu ◽  
Kang Zhang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Transfer Learning ◽  
Health Monitoring ◽  
Structural Response ◽  
Data Driven ◽  
Structural Health ◽  
Learning Techniques

With the rapid development of information and communication techniques, dam structural health assessment based on data collected from structural health monitoring systems has become a trend. This allows for applying data-driven methods for dam safety analysis. However, data-driven models in most related literature are statistical and shallow machine learning models, which cannot capture the time series patterns or learn from long-term dependencies of dam structural response time series. Furthermore, the effectiveness and applicability of these models are only validated in a small data set and part of monitoring points in a dam structural health monitoring system. To address the problems, this article proposes a new modeling paradigm based on various deep learning and transfer learning techniques. The paradigm utilizes one-dimensional convolutional neural networks to extract the inherent features from dam structural response–related environmental quantity monitoring data. Then bidirectional gated recurrent unit with a self-attention mechanism is used to learn from long-term dependencies, and transfer learning is utilized to transfer knowledge learned from the typical monitoring point to the others. The proposed paradigm integrates the powerful modeling capability of deep learning networks and the flexible transferability of transfer learning. Rather than traditional models that rely on experience for feature selection, the proposed deep learning–based paradigm directly utilizes environmental monitoring time series as inputs to accurately estimate dam structural response changes. A high arch dam in long-term service is selected as the case study, and three monitoring items, including dam displacement, crack opening displacement, and seepage are used as the research objects. The experimental results show that the proposed paradigm outperforms conventional and shallow machine learning–based methods in all 41 tested monitoring points, which indicates that the proposed paradigm is capable of dealing with dam structural response estimation with high accuracy and robustness.

Deep Learning for text in limted data settings

2020 ◽  
Author(s):  
Pathikkumar Patel ◽  
Bhargav Lad ◽  
Jinan Fiaidhi
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Transfer Learning ◽  
Text Classification ◽  
State Of The Art ◽  
Time Series Forecasting ◽  
Text Data ◽  
Performance Levels

During the last few years, RNN models have been extensively used and they have proven to be better for sequence and text data. RNNs have achieved state-of-the-art performance levels in several applications such as text classification, sequence to sequence modelling and time series forecasting. In this article we will review different Machine Learning and Deep Learning based approaches for text data and look at the results obtained from these methods. This work also explores the use of transfer learning in NLP and how it affects the performance of models on a specific application of sentiment analysis.

scholarly journals Deep Transfer Learning Based Intrusion Detection System for Electric Vehicular Networks

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4736
Author(s):  
Sk. Tanzir Mehedi ◽  
Adnan Anwar ◽  
Ziaur Rahman ◽  
Kawsar Ahmed
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Intrusion Detection ◽  
Real Time ◽  
Transfer Learning ◽  
Security Requirements ◽  
Detection Accuracy ◽  
Area Network ◽  
Complex Data ◽  
Network Intrusion

The Controller Area Network (CAN) bus works as an important protocol in the real-time In-Vehicle Network (IVN) systems for its simple, suitable, and robust architecture. The risk of IVN devices has still been insecure and vulnerable due to the complex data-intensive architectures which greatly increase the accessibility to unauthorized networks and the possibility of various types of cyberattacks. Therefore, the detection of cyberattacks in IVN devices has become a growing interest. With the rapid development of IVNs and evolving threat types, the traditional machine learning-based IDS has to update to cope with the security requirements of the current environment. Nowadays, the progression of deep learning, deep transfer learning, and its impactful outcome in several areas has guided as an effective solution for network intrusion detection. This manuscript proposes a deep transfer learning-based IDS model for IVN along with improved performance in comparison to several other existing models. The unique contributions include effective attribute selection which is best suited to identify malicious CAN messages and accurately detect the normal and abnormal activities, designing a deep transfer learning-based LeNet model, and evaluating considering real-world data. To this end, an extensive experimental performance evaluation has been conducted. The architecture along with empirical analyses shows that the proposed IDS greatly improves the detection accuracy over the mainstream machine learning, deep learning, and benchmark deep transfer learning models and has demonstrated better performance for real-time IVN security.

scholarly journals Effectiveness of transfer learning for enhancing tumor classification with a convolutional neural network on frozen sections

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Young-Gon Kim ◽  
Sungchul Kim ◽  
Cristina Eunbee Cho ◽  
In Hye Song ◽  
Hee Jin Lee ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Frozen Section ◽  
Medical Center ◽  
External Validation ◽  
Model Performance ◽  
Classification Model ◽  
Training Dataset

AbstractFast and accurate confirmation of metastasis on the frozen tissue section of intraoperative sentinel lymph node biopsy is an essential tool for critical surgical decisions. However, accurate diagnosis by pathologists is difficult within the time limitations. Training a robust and accurate deep learning model is also difficult owing to the limited number of frozen datasets with high quality labels. To overcome these issues, we validated the effectiveness of transfer learning from CAMELYON16 to improve performance of the convolutional neural network (CNN)-based classification model on our frozen dataset (N = 297) from Asan Medical Center (AMC). Among the 297 whole slide images (WSIs), 157 and 40 WSIs were used to train deep learning models with different dataset ratios at 2, 4, 8, 20, 40, and 100%. The remaining, i.e., 100 WSIs, were used to validate model performance in terms of patch- and slide-level classification. An additional 228 WSIs from Seoul National University Bundang Hospital (SNUBH) were used as an external validation. Three initial weights, i.e., scratch-based (random initialization), ImageNet-based, and CAMELYON16-based models were used to validate their effectiveness in external validation. In the patch-level classification results on the AMC dataset, CAMELYON16-based models trained with a small dataset (up to 40%, i.e., 62 WSIs) showed a significantly higher area under the curve (AUC) of 0.929 than those of the scratch- and ImageNet-based models at 0.897 and 0.919, respectively, while CAMELYON16-based and ImageNet-based models trained with 100% of the training dataset showed comparable AUCs at 0.944 and 0.943, respectively. For the external validation, CAMELYON16-based models showed higher AUCs than those of the scratch- and ImageNet-based models. Model performance for slide feasibility of the transfer learning to enhance model performance was validated in the case of frozen section datasets with limited numbers.

Sign in / Sign up

Export Citation Format

Share Document