An Online Transfer Learning Approach for Identification and Predictive Control Design With Application to RCCI Engines

2020 ◽  
Author(s):  
Yajie Bao ◽  
Javad Mohammadpour Velni ◽  
Mahdi Shahbakhti
Keyword(s):  
Transfer Learning ◽  
Closed Loop ◽  
Fine Tuning ◽  
Control Performance ◽  
Controlled System ◽  
Reactivity Controlled Compression Ignition ◽  
Engine Model ◽  
Learning Technique ◽  
Predictive Controller ◽  
Batch Data

Abstract This paper presents a framework to refine identified artificial neural networks (ANN) based state-space linear parameter-varying (LPV-SS) models with closed-loop data using online transfer learning. An LPV-SS model is assumed to be first identified offline using inputs/outputs data and a model predictive controller (MPC) designed based on this model. Using collected closed-loop batch data, the model is further refined using online transfer learning and thus the control performance is improved. Specifically, fine-tuning, a transfer learning technique, is employed to improve the model. Furthermore, the scenario where the offline identified model and the online controlled system are “similar but not identitical” is discussed. The proposed method is verified by testing on an experimentally validated high-fidelity reactivity controlled compression ignition (RCCI) engine model. The verification results show that the new online transfer learning technique combined with an adaptive MPC law improves the engine control performance to track requested engine loads and desired combustion phasing with minimum errors.

scholarly journals Automatic Pharyngeal Phase Recognition in Untrimmed Videofluoroscopic Swallowing Study Using Transfer Learning with Deep Convolutional Neural Networks

Diagnostics ◽  
2021 ◽  
Vol 11 (2) ◽  
pp. 300
Author(s):  
Ki-Sun Lee ◽  
Eunyoung Lee ◽  
Bareun Choi ◽  
Sung-Bom Pyun
Keyword(s):  
Transfer Learning ◽  
Fine Tuning ◽  
Deep Convolutional Neural Networks ◽  
Image Frame ◽  
Pharyngeal Phase ◽  
Single Frame ◽  
Learning Techniques ◽  
Learning Technique ◽  
Deep Cnn ◽  
Phase Recognition

Background: Video fluoroscopic swallowing study (VFSS) is considered as the gold standard diagnostic tool for evaluating dysphagia. However, it is time consuming and labor intensive for the clinician to manually search the recorded long video image frame by frame to identify the instantaneous swallowing abnormality in VFSS images. Therefore, this study aims to present a deep leaning-based approach using transfer learning with a convolutional neural network (CNN) that automatically annotates pharyngeal phase frames in untrimmed VFSS videos such that frames need not be searched manually. Methods: To determine whether the image frame in the VFSS video is in the pharyngeal phase, a single-frame baseline architecture based the deep CNN framework is used and a transfer learning technique with fine-tuning is applied. Results: Compared with all experimental CNN models, that fine-tuned with two blocks of the VGG-16 (VGG16-FT5) model achieved the highest performance in terms of recognizing the frame of pharyngeal phase, that is, the accuracy of 93.20 (±1.25)%, sensitivity of 84.57 (±5.19)%, specificity of 94.36 (±1.21)%, AUC of 0.8947 (±0.0269) and Kappa of 0.7093 (±0.0488). Conclusions: Using appropriate and fine-tuning techniques and explainable deep learning techniques such as grad CAM, this study shows that the proposed single-frame-baseline-architecture-based deep CNN framework can yield high performances in the full automation of VFSS video analysis.

scholarly journals Utilizing Indonesian Universal Language Model Fine-tuning for Text Classification

2021 ◽  
Vol 5 (3) ◽  
pp. 325
Author(s):  
Hendra Bunyamin
Keyword(s):  
Computer Vision ◽  
Natural Language Processing ◽  
Transfer Learning ◽  
Language Processing ◽  
Text Classification ◽  
Language Model ◽  
Fine Tuning ◽  
Classification Task ◽  
Universal Language ◽  
Learning Technique

Inductive transfer learning technique has made a huge impact on the computer vision field. Particularly, computer vision  applications including object detection, classification, and segmentation, are rarely trained from scratch; instead, they are fine-tuned from pretrained models, which are products of learning from huge datasets. In contrast to computer vision, state-of-the-art natural language processing models are still generally trained from the ground up. Accordingly, this research attempts to investigate an adoption of the transfer learning technique for natural language processing. Specifically, we utilize a transfer learning technique called Universal Language Model Fine-tuning (ULMFiT) for doing an Indonesian news text classification task. The dataset for constructing the language model is collected from several news providers from January to December 2017 whereas the dataset employed for text classification task comes from news articles provided by the Agency for the Assessment and Application of Technology (BPPT). To examine the impact of ULMFiT, we provide a baseline that is a vanilla neural network with two hidden layers. Although the performance of ULMFiT on validation set is lower than the one of our baseline, we find that the benefits of ULMFiT for the classification task significantly reduce the overfitting, that is the difference between train and validation accuracies from 4% to nearly zero.

scholarly journals AdaFilter: Adaptive Filter Fine-Tuning for Deep Transfer Learning

2020 ◽  
Vol 34 (04) ◽  
pp. 4060-4066
Author(s):  
Yunhui Guo ◽  
Yandong Li ◽  
Liqiang Wang ◽  
Tajana Rosing
Keyword(s):  
Transfer Learning ◽  
Deep Neural Networks ◽  
Network Models ◽  
Fine Tuning ◽  
Public Image ◽  
Classification Error ◽  
Neural Network Models ◽  
Learning Technique ◽  
Task Transfer ◽  
Fine Tune

There is an increasing number of pre-trained deep neural network models. However, it is still unclear how to effectively use these models for a new task. Transfer learning, which aims to transfer knowledge from source tasks to a target task, is an effective solution to this problem. Fine-tuning is a popular transfer learning technique for deep neural networks where a few rounds of training are applied to the parameters of a pre-trained model to adapt them to a new task. Despite its popularity, in this paper we show that fine-tuning suffers from several drawbacks. We propose an adaptive fine-tuning approach, called AdaFilter, which selects only a part of the convolutional filters in the pre-trained model to optimize on a per-example basis. We use a recurrent gated network to selectively fine-tune convolutional filters based on the activations of the previous layer. We experiment with 7 public image classification datasets and the results show that AdaFilter can reduce the average classification error of the standard fine-tuning by 2.54%.

scholarly journals Evaluation of Optimal Vibrotactile Feedback for Force-Controlled Upper Limb Myoelectric Prostheses

Sensors ◽  
2019 ◽  
Vol 19 (23) ◽  
pp. 5209 ◽  
Author(s):  
Andrea Gonzalez-Rodriguez ◽  
Jose L. Ramon ◽  
Vicente Morell ◽  
Gabriel J. Garcia ◽  
Jorge Pomares ◽  
...  
Keyword(s):  
Real Time ◽  
Upper Limb ◽  
Closed Loop ◽  
Fine Tuning ◽  
Vibrotactile Feedback ◽  
Loop Control ◽  
Myoelectric Prostheses ◽  
Amplitude Level ◽  
Real Time Application ◽  
Selection Of

The main goal of this study is to evaluate how to optimally select the best vibrotactile pattern to be used in a closed loop control of upper limb myoelectric prostheses as a feedback of the exerted force. To that end, we assessed both the selection of actuation patterns and the effects of the selection of frequency and amplitude parameters to discriminate between different feedback levels. A single vibrotactile actuator has been used to deliver the vibrations to subjects participating in the experiments. The results show no difference between pattern shapes in terms of feedback perception. Similarly, changes in amplitude level do not reflect significant improvement compared to changes in frequency. However, decreasing the number of feedback levels increases the accuracy of feedback perception and subject-specific variations are high for particular participants, showing that a fine-tuning of the parameters is necessary in a real-time application to upper limb prosthetics. In future works, the effects of training, location, and number of actuators will be assessed. This optimized selection will be tested in a real-time proportional myocontrol of a prosthetic hand.

scholarly journals Deep Learning Techniques for Grape Plant Species Identification in Natural Images

Sensors ◽  
2019 ◽  
Vol 19 (22) ◽  
pp. 4850 ◽  
Author(s):  
Carlos S. Pereira ◽  
Raul Morais ◽  
Manuel J. C. S. Reis
Keyword(s):  
Transfer Learning ◽  
Climatic Conditions ◽  
Fine Tuning ◽  
Variety Identification ◽  
Test Accuracy ◽  
Accuracy Score ◽  
Learning Techniques ◽  
Four Corners ◽  
Integrated Software ◽  
Grape Varieties

Frequently, the vineyards in the Douro Region present multiple grape varieties per parcel and even per row. An automatic algorithm for grape variety identification as an integrated software component was proposed that can be applied, for example, to a robotic harvesting system. However, some issues and constraints in its development were highlighted, namely, the images captured in natural environment, low volume of images, high similarity of the images among different grape varieties, leaf senescence, and significant changes on the grapevine leaf and bunch images in the harvest seasons, mainly due to adverse climatic conditions, diseases, and the presence of pesticides. In this paper, the performance of the transfer learning and fine-tuning techniques based on AlexNet architecture were evaluated when applied to the identification of grape varieties. Two natural vineyard image datasets were captured in different geographical locations and harvest seasons. To generate different datasets for training and classification, some image processing methods, including a proposed four-corners-in-one image warping algorithm, were used. The experimental results, obtained from the application of an AlexNet-based transfer learning scheme and trained on the image dataset pre-processed through the four-corners-in-one method, achieved a test accuracy score of 77.30%. Applying this classifier model, an accuracy of 89.75% on the popular Flavia leaf dataset was reached. The results obtained by the proposed approach are promising and encouraging in helping Douro wine growers in the automatic task of identifying grape varieties.

Reliability of nonlinear stochastic controlled systems considering the dynamics of sensors and actuators

2021 ◽  
pp. 107754632110037
Author(s):  
Sun Jiaojiao ◽  
Xia Lei ◽  
Ying Zuguang ◽  
Huan Ronghua ◽  
Zhu Weiqiu
Keyword(s):  
Closed Loop ◽  
First Passage Time ◽  
Hamiltonian Function ◽  
Small Time ◽  
Kolmogorov Equation ◽  
Sensors And Actuators ◽  
First Passage ◽  
Controlled System ◽  
Main Structure ◽  
Controlled Systems

A closed-loop controlled system usually consists of the main structure, sensors, and actuators. The dynamics of sensors and actuators may influence the motion of the main structure. This article presents an analytical study on the first-passage reliability of a nonlinear stochastic controlled system under the consideration of the dynamics of sensors and actuators. The coupled dynamic equations of the controlled systems with sensors and actuators are first given, which are further integrated into a controlled, randomly excited, dissipated Hamiltonian system. By applying the stochastic averaging method for quasi-Hamiltonian systems, a one-dimensional averaged differential equation for the Hamiltonian function is obtained. The backward Kolmogorov equation associated with the averaged equation is then derived for the first-passage reliability analysis, from which the approximate reliability function and probability density of first-passage time are obtained. The accuracy of the proposed procedure is demonstrated by an example. A comparative analysis of the reliability of the system with/without sensors and actuators is carried out, which indicates that ignoring sensors and actuators will make underestimation of the reliability of the closed-loop system with small time. However, when time increases, there appears the opposite trend. Our findings provide a reference for control strategy design.

scholarly journals New polyp image classification technique using transfer learning of network-in-network structure in endoscopic images

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Young Jae Kim ◽  
Jang Pyo Bae ◽  
Jun-Won Chung ◽  
Dong Kyun Park ◽  
Kwang Gi Kim ◽  
...  
Keyword(s):  
Colorectal Cancer ◽  
Transfer Learning ◽  
Test Data ◽  
State Of The Art ◽  
Early Stage ◽  
Statistical Significance ◽  
Recall Rate ◽  
Training Data ◽  
Fine Tuning ◽  
Accuracy Evaluation

AbstractWhile colorectal cancer is known to occur in the gastrointestinal tract. It is the third most common form of cancer of 27 major types of cancer in South Korea and worldwide. Colorectal polyps are known to increase the potential of developing colorectal cancer. Detected polyps need to be resected to reduce the risk of developing cancer. This research improved the performance of polyp classification through the fine-tuning of Network-in-Network (NIN) after applying a pre-trained model of the ImageNet database. Random shuffling is performed 20 times on 1000 colonoscopy images. Each set of data are divided into 800 images of training data and 200 images of test data. An accuracy evaluation is performed on 200 images of test data in 20 experiments. Three compared methods were constructed from AlexNet by transferring the weights trained by three different state-of-the-art databases. A normal AlexNet based method without transfer learning was also compared. The accuracy of the proposed method was higher in statistical significance than the accuracy of four other state-of-the-art methods, and showed an 18.9% improvement over the normal AlexNet based method. The area under the curve was approximately 0.930 ± 0.020, and the recall rate was 0.929 ± 0.029. An automatic algorithm can assist endoscopists in identifying polyps that are adenomatous by considering a high recall rate and accuracy. This system can enable the timely resection of polyps at an early stage.

scholarly journals COVID-19 diagnosis from chest X-ray images using transfer learning: Enhanced performance by debiasing dataloader

2021 ◽  
Vol 29 (1) ◽  
pp. 19-36
Author(s):  
Çağín Polat ◽  
Onur Karaman ◽  
Ceren Karaman ◽  
Güney Korkmaz ◽  
Mehmet Can Balcı ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Critical Time ◽  
Image Data ◽  
Mapping Method ◽  
Fine Tuning ◽  
Learning Stage ◽  
X Ray ◽  
Testing Dataset ◽  
X Ray Imaging ◽  
Chest X Ray

BACKGROUND: Chest X-ray imaging has been proved as a powerful diagnostic method to detect and diagnose COVID-19 cases due to its easy accessibility, lower cost and rapid imaging time. OBJECTIVE: This study aims to improve efficacy of screening COVID-19 infected patients using chest X-ray images with the help of a developed deep convolutional neural network model (CNN) entitled nCoV-NET. METHODS: To train and to evaluate the performance of the developed model, three datasets were collected from resources of “ChestX-ray14”, “COVID-19 image data collection”, and “Chest X-ray collection from Indiana University,” respectively. Overall, 299 COVID-19 pneumonia cases and 1,522 non-COVID 19 cases are involved in this study. To overcome the probable bias due to the unbalanced cases in two classes of the datasets, ResNet, DenseNet, and VGG architectures were re-trained in the fine-tuning stage of the process to distinguish COVID-19 classes using a transfer learning method. Lastly, the optimized final nCoV-NET model was applied to the testing dataset to verify the performance of the proposed model. RESULTS: Although the performance parameters of all re-trained architectures were determined close to each other, the final nCOV-NET model optimized by using DenseNet-161 architecture in the transfer learning stage exhibits the highest performance for classification of COVID-19 cases with the accuracy of 97.1 %. The Activation Mapping method was used to create activation maps that highlights the crucial areas of the radiograph to improve causality and intelligibility. CONCLUSION: This study demonstrated that the proposed CNN model called nCoV-NET can be utilized for reliably detecting COVID-19 cases using chest X-ray images to accelerate the triaging and save critical time for disease control as well as assisting the radiologist to validate their initial diagnosis.

Sign in / Sign up

Export Citation Format

Share Document