Auto-sharing parameters for transfer learning based on multi-objective optimization

2021 ◽  
pp. 1-13
Author(s):  
Hailin Liu ◽  
Fangqing Gu ◽  
Zixian Lin
Keyword(s):  
Transfer Learning ◽  
Optimization Problem ◽  
Data Sets ◽  
Multi Objective Optimization ◽  
Particle Swarm Optimizer ◽  
Real World Data ◽  
Data Set ◽  
Target Task ◽  
Main Research ◽  
Multi Objective

Transfer learning methods exploit similarities between different datasets to improve the performance of the target task by transferring knowledge from source tasks to the target task. “What to transfer” is a main research issue in transfer learning. The existing transfer learning method generally needs to acquire the shared parameters by integrating human knowledge. However, in many real applications, an understanding of which parameters can be shared is unknown beforehand. Transfer learning model is essentially a special multi-objective optimization problem. Consequently, this paper proposes a novel auto-sharing parameter technique for transfer learning based on multi-objective optimization and solves the optimization problem by using a multi-swarm particle swarm optimizer. Each task objective is simultaneously optimized by a sub-swarm. The current best particle from the sub-swarm of the target task is used to guide the search of particles of the source tasks and vice versa. The target task and source task are jointly solved by sharing the information of the best particle, which works as an inductive bias. Experiments are carried out to evaluate the proposed algorithm on several synthetic data sets and two real-world data sets of a school data set and a landmine data set, which show that the proposed algorithm is effective.

Development of Fractional Genetic PSO Algorithm for Multi Objective Data Clustering

2016 ◽  
Vol 7 (3) ◽  
pp. 1-16 ◽  
Author(s):  
Aparna K. ◽  
Mydhili K. Nair
Keyword(s):  
Data Clustering ◽  
Fitness Function ◽  
Pso Algorithm ◽  
Data Sets ◽  
Multi Objective Optimization ◽  
Data Set ◽  
Multi Objective ◽  
Stability Measure ◽  
Benchmark Data ◽  
Hybrid Optimization Algorithm

Clustering is the task of finding natural partitioning within a data set such that data items within the same group are more similar than those within different groups. The performance of the traditional K-Means and Bisecting K-Means algorithm degrades as the dimensionality of the data increases. In order to find better clustering results, it is important to enhance the traditional algorithms by incorporating various constraints. Hence it is planned to develop a Multi-Objective Optimization (MOO) technique by including different objectives, like MSE, Stability measure, DB index, XB-index and sym-index. These five objectives will be used as fitness function for the proposed Fractional Genetic PSO algorithm (FGPSO) which is the hybrid optimization algorithm to do the clustering process. The performance of the proposed multi objective FGPSO algorithm will be evaluated based on clustering accuracy. Finally, the applicability of the proposed algorithm will be checked for some benchmark data sets available in the UCI machine learning repository.

scholarly journals Classification of jujube defects in small data sets based on transfer learning

2021 ◽  
Author(s):  
Jianping Ju ◽  
Hong Zheng ◽  
Xiaohang Xu ◽  
Zhongyuan Guo ◽  
Zhaohui Zheng ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Loss Function ◽  
Training Model ◽  
Parameter Distribution ◽  
Test Accuracy ◽  
Small Data ◽  
Data Sets ◽  
Data Set ◽  
Small Data Sets

AbstractAlthough convolutional neural networks have achieved success in the field of image classification, there are still challenges in the field of agricultural product quality sorting such as machine vision-based jujube defects detection. The performance of jujube defect detection mainly depends on the feature extraction and the classifier used. Due to the diversity of the jujube materials and the variability of the testing environment, the traditional method of manually extracting the features often fails to meet the requirements of practical application. In this paper, a jujube sorting model in small data sets based on convolutional neural network and transfer learning is proposed to meet the actual demand of jujube defects detection. Firstly, the original images collected from the actual jujube sorting production line were pre-processed, and the data were augmented to establish a data set of five categories of jujube defects. The original CNN model is then improved by embedding the SE module and using the triplet loss function and the center loss function to replace the softmax loss function. Finally, the depth pre-training model on the ImageNet image data set was used to conduct training on the jujube defects data set, so that the parameters of the pre-training model could fit the parameter distribution of the jujube defects image, and the parameter distribution was transferred to the jujube defects data set to complete the transfer of the model and realize the detection and classification of the jujube defects. The classification results are visualized by heatmap through the analysis of classification accuracy and confusion matrix compared with the comparison models. The experimental results show that the SE-ResNet50-CL model optimizes the fine-grained classification problem of jujube defect recognition, and the test accuracy reaches 94.15%. The model has good stability and high recognition accuracy in complex environments.

scholarly journals Braille Block Detection via Multi-Objective Optimization from an Egocentric Viewpoint

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2775
Author(s):  
Tsubasa Takano ◽  
Takumi Nakane ◽  
Takuya Akashi ◽  
Chao Zhang
Keyword(s):  
Optimization Problem ◽  
Line Pair ◽  
Objective Functions ◽  
Multi Objective Optimization ◽  
Multi Objective ◽  
Optimization Framework ◽  
Visually Impaired People ◽  
Comprehensive Comparison ◽  
Basic Characteristics ◽  
Support Devices

In this paper, we propose a method to detect Braille blocks from an egocentric viewpoint, which is a key part of many walking support devices for visually impaired people. Our main contribution is to cast this task as a multi-objective optimization problem and exploits both the geometric and the appearance features for detection. Specifically, two objective functions were designed under an evolutionary optimization framework with a line pair modeled as an individual (i.e., solution). Both of the objectives follow the basic characteristics of the Braille blocks, which aim to clarify the boundaries and estimate the likelihood of the Braille block surface. Our proposed method was assessed by an originally collected and annotated dataset under real scenarios. Both quantitative and qualitative experimental results show that the proposed method can detect Braille blocks under various environments. We also provide a comprehensive comparison of the detection performance with respect to different multi-objective optimization algorithms.

scholarly journals Research on urban public bicycle dispatching optimization method

2019 ◽  
Author(s):  
Lin Fei ◽  
Yang Yang ◽  
Wang Shihua ◽  
Xu Yudi ◽  
Ma Hong
Keyword(s):  
Clustering Algorithm ◽  
Optimization Method ◽  
Multi Objective Optimization ◽  
Data Set ◽  
Community Discovery ◽  
The Public ◽  
Multi Objective ◽  
Scheduling Scheme ◽  
Theoretical Support ◽  
The Matrix

Unreasonable public bicycle dispatching area division seriously affects the operational efficiency of the public bicycle system. To solve this problem, this paper innovatively proposes an improved community discovery algorithm based on multi-objective optimization (CDoMO). The data set is preprocessed into a lease/return relationship, thereby it calculated a similarity matrix, and the community discovery algorithm Fast Unfolding is executed on the matrix to obtain a scheduling scheme. For the results obtained by the algorithm, the workload indicators (scheduled distance, number of sites, and number of scheduling bicycles) should be adjusted to maximize the overall benefits, and the entire process is continuously optimized by a multi-objective optimization algorithm NSGA2. The experimental results show that compared with the clustering algorithm and the community discovery algorithm, the method can shorten the estimated scheduling distance by 20%-50%, and can effectively balance the scheduling workload of each area. The method can provide theoretical support for the public bicycle dispatching department, and improve the efficiency of public bicycle dispatching system.

scholarly journals Learning Discriminative Correlation Subspace for Heterogeneous Domain Adaptation

2017 ◽  
Author(s):  
Yuguang Yan ◽  
Wen Li ◽  
Michael Ng ◽  
Mingkui Tan ◽  
Hanrui Wu ◽  
...  
Keyword(s):  
Optimization Problem ◽  
Domain Adaptation ◽  
Data Sets ◽  
Target Domain ◽  
Real World Data ◽  
Discriminative Ability ◽  
Convex Optimization Problem ◽  
Alternating Direction ◽  
Feature Spaces ◽  
Target Data

Domain adaptation aims to reduce the effort on collecting and annotating target data by leveraging knowledge from a different source domain. The domain adaptation problem will become extremely challenging when the feature spaces of the source and target domains are different, which is also known as the heterogeneous domain adaptation (HDA) problem. In this paper, we propose a novel HDA method to find the optimal discriminative correlation subspace for the source and target data. The discriminative correlation subspace is inherited from the canonical correlation subspace between the source and target data, and is further optimized to maximize the discriminative ability for the target domain classifier. We formulate a joint objective in order to simultaneously learn the discriminative correlation subspace and the target domain classifier. We then apply an alternating direction method of multiplier (ADMM) algorithm to address the resulting non-convex optimization problem. Comprehensive experiments on two real-world data sets demonstrate the effectiveness of the proposed method compared to the state-of-the-art methods.

Toward the Use of Pareto Performance Solutions and Pareto Robustness Solutions for Multi-Objective Robust Optimization Problems

2012 ◽  
Author(s):  
Weijun Wang ◽  
Stéphane Caro ◽  
Fouad Bennis ◽  
Oscar Brito Augusto
Keyword(s):  
Robust Optimization ◽  
Pareto Front ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Multi Objective Optimization ◽  
Robust Solutions ◽  
Multi Objective ◽  
Robust Optimization Problem ◽  
Design Solutions ◽  
Further Development

For Multi-Objective Robust Optimization Problem (MOROP), it is important to obtain design solutions that are both optimal and robust. To find these solutions, usually, the designer need to set a threshold of the variation of Performance Functions (PFs) before optimization, or add the effects of uncertainties on the original PFs to generate a new Pareto robust front. In this paper, we divide a MOROP into two Multi-Objective Optimization Problems (MOOPs). One is the original MOOP, another one is that we take the Robustness Functions (RFs), robust counterparts of the original PFs, as optimization objectives. After solving these two MOOPs separately, two sets of solutions come out, namely the Pareto Performance Solutions (PP) and the Pareto Robustness Solutions (PR). Make a further development on these two sets, we can get two types of solutions, namely the Pareto Robustness Solutions among the Pareto Performance Solutions (PR(PP)), and the Pareto Performance Solutions among the Pareto Robustness Solutions (PP(PR)). Further more, the intersection of PR(PP) and PP(PR) can represent the intersection of PR and PP well. Then the designer can choose good solutions by comparing the results of PR(PP) and PP(PR). Thanks to this method, we can find out the optimal and robust solutions without setting the threshold of the variation of PFs nor losing the initial Pareto front. Finally, an illustrative example highlights the contributions of the paper.

Sign in / Sign up

Export Citation Format

Share Document