scholarly journals Variable-Length Particle Swarm Optimization for Feature Selection on High-Dimensional Classification

2021 ◽  
Author(s):  
B Tran ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Feature Selection ◽  
Particle Swarm Optimization ◽  
Computational Cost ◽  
Particle Swarm ◽  
Search Space ◽  
Classification Performance ◽  
Variable Length ◽  
High Dimensional ◽  
Swarm Optimization ◽  
Local Optima

© 1997-2012 IEEE. With a global search mechanism, particle swarm optimization (PSO) has shown promise in feature selection (FS). However, most of the current PSO-based FS methods use a fix-length representation, which is inflexible and limits the performance of PSO for FS. When applying these methods to high-dimensional data, it not only consumes a significant amount of memory but also requires a high computational cost. Overcoming this limitation enables PSO to work on data with much higher dimensionality which has become more and more popular with the advance of data collection technologies. In this paper, we propose the first variable-length PSO representation for FS, enabling particles to have different and shorter lengths, which defines smaller search space and therefore, improves the performance of PSO. By rearranging features in a descending order of their relevance, we facilitate particles with shorter lengths to achieve better classification performance. Furthermore, using the proposed length changing mechanism, PSO can jump out of local optima, further narrow the search space and focus its search on smaller and more fruitful area. These strategies enable PSO to reach better solutions in a shorter time. Results on ten high-dimensional datasets with varying difficulties show that the proposed variable-length PSO can achieve much smaller feature subsets with significantly higher classification performance in much shorter time than the fixed-length PSO methods. The proposed method also outperformed the compared non-PSO FS methods in most cases. © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

scholarly journals Variable-Length Particle Swarm Optimization for Feature Selection on High-Dimensional Classification

2021 ◽  
Author(s):  
B Tran ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Feature Selection ◽  
Particle Swarm Optimization ◽  
Computational Cost ◽  
Particle Swarm ◽  
Search Space ◽  
Classification Performance ◽  
Variable Length ◽  
High Dimensional ◽  
Swarm Optimization ◽  
Local Optima

© 1997-2012 IEEE. With a global search mechanism, particle swarm optimization (PSO) has shown promise in feature selection (FS). However, most of the current PSO-based FS methods use a fix-length representation, which is inflexible and limits the performance of PSO for FS. When applying these methods to high-dimensional data, it not only consumes a significant amount of memory but also requires a high computational cost. Overcoming this limitation enables PSO to work on data with much higher dimensionality which has become more and more popular with the advance of data collection technologies. In this paper, we propose the first variable-length PSO representation for FS, enabling particles to have different and shorter lengths, which defines smaller search space and therefore, improves the performance of PSO. By rearranging features in a descending order of their relevance, we facilitate particles with shorter lengths to achieve better classification performance. Furthermore, using the proposed length changing mechanism, PSO can jump out of local optima, further narrow the search space and focus its search on smaller and more fruitful area. These strategies enable PSO to reach better solutions in a shorter time. Results on ten high-dimensional datasets with varying difficulties show that the proposed variable-length PSO can achieve much smaller feature subsets with significantly higher classification performance in much shorter time than the fixed-length PSO methods. The proposed method also outperformed the compared non-PSO FS methods in most cases. © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

scholarly journals A New Representation in PSO for Discretization-Based Feature Selection

2021 ◽  
Author(s):  
B Tran ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Feature Selection ◽  
Particle Swarm Optimization ◽  
Fitness Function ◽  
Particle Swarm ◽  
Search Space ◽  
Classification Performance ◽  
High Dimensional ◽  
Swarm Optimization ◽  
Learning Capacity ◽  
Feature Interactions

In machine learning, discretization and feature selection (FS) are important techniques for preprocessing data to improve the performance of an algorithm on high-dimensional data. Since many FS methods require discrete data, a common practice is to apply discretization before FS. In addition, for the sake of efficiency, features are usually discretized individually (or univariate). This scheme works based on the assumption that each feature independently influences the task, which may not hold in cases where feature interactions exist. Therefore, univariate discretization may degrade the performance of the FS stage since information showing feature interactions may be lost during the discretization process. Initial results of our previous proposed method [evolve particle swarm optimization (EPSO)] showed that combining discretization and FS in a single stage using bare-bones particle swarm optimization (BBPSO) can lead to a better performance than applying them in two separate stages. In this paper, we propose a new method called potential particle swarm optimization (PPSO) which employs a new representation that can reduce the search space of the problem and a new fitness function to better evaluate candidate solutions to guide the search. The results on ten high-dimensional datasets show that PPSO select less than 5% of the number of features for all datasets. Compared with the two-stage approach which uses BBPSO for FS on the discretized data, PPSO achieves significantly higher accuracy on seven datasets. In addition, PPSO obtains better (or similar) classification performance than EPSO on eight datasets with a smaller number of selected features on six datasets. Furthermore, PPSO also outperforms the three compared (traditional) methods and performs similar to one method on most datasets in terms of both generalization ability and learning capacity. © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

scholarly journals A New Representation in PSO for Discretization-Based Feature Selection

2021 ◽  
Author(s):  
B Tran ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Feature Selection ◽  
Particle Swarm Optimization ◽  
Fitness Function ◽  
Particle Swarm ◽  
Search Space ◽  
Classification Performance ◽  
High Dimensional ◽  
Swarm Optimization ◽  
Learning Capacity ◽  
Feature Interactions

In machine learning, discretization and feature selection (FS) are important techniques for preprocessing data to improve the performance of an algorithm on high-dimensional data. Since many FS methods require discrete data, a common practice is to apply discretization before FS. In addition, for the sake of efficiency, features are usually discretized individually (or univariate). This scheme works based on the assumption that each feature independently influences the task, which may not hold in cases where feature interactions exist. Therefore, univariate discretization may degrade the performance of the FS stage since information showing feature interactions may be lost during the discretization process. Initial results of our previous proposed method [evolve particle swarm optimization (EPSO)] showed that combining discretization and FS in a single stage using bare-bones particle swarm optimization (BBPSO) can lead to a better performance than applying them in two separate stages. In this paper, we propose a new method called potential particle swarm optimization (PPSO) which employs a new representation that can reduce the search space of the problem and a new fitness function to better evaluate candidate solutions to guide the search. The results on ten high-dimensional datasets show that PPSO select less than 5% of the number of features for all datasets. Compared with the two-stage approach which uses BBPSO for FS on the discretized data, PPSO achieves significantly higher accuracy on seven datasets. In addition, PPSO obtains better (or similar) classification performance than EPSO on eight datasets with a smaller number of selected features on six datasets. Furthermore, PPSO also outperforms the three compared (traditional) methods and performs similar to one method on most datasets in terms of both generalization ability and learning capacity. © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

A hybrid feature selection algorithm combining ReliefF and Particle swarm optimization for high-dimensional medical data

2021 ◽  
pp. 1-15
Author(s):  
Zhaozhao Xu ◽  
Derong Shen ◽  
Yue Kou ◽  
Tiezheng Nie
Keyword(s):  
Feature Selection ◽  
Particle Swarm Optimization ◽  
Random Forest ◽  
Classification Accuracy ◽  
Particle Swarm ◽  
Medical Data ◽  
High Dimensional ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Swarm Optimization

Due to high-dimensional feature and strong correlation of features, the classification accuracy of medical data is not as good enough as expected. feature selection is a common algorithm to solve this problem, and selects effective features by reducing the dimensionality of high-dimensional data. However, traditional feature selection algorithms have the blindness of threshold setting and the search algorithms are liable to fall into a local optimal solution. Based on it, this paper proposes a hybrid feature selection algorithm combining ReliefF and Particle swarm optimization. The algorithm is mainly divided into three parts: Firstly, the ReliefF is used to calculate the feature weight, and the features are ranked by the weight. Then ranking feature is grouped according to the density equalization, where the density of features in each group is the same. Finally, the Particle Swarm Optimization algorithm is used to search the ranking feature groups, and the feature selection is performed according to a new fitness function. Experimental results show that the random forest has the highest classification accuracy on the features selected. More importantly, it has the least number of features. In addition, experimental results on 2 medical datasets show that the average accuracy of random forest reaches 90.20%, which proves that the hybrid algorithm has a certain application value.

scholarly journals A Review on Particle Swarm Optimization Algorithm and Its Variants to Human Motion Tracking

2014 ◽  
Vol 2014 ◽  
pp. 1-16 ◽  
Author(s):  
Sanjay Saini ◽  
Dayang Rohaya Bt Awang Rambli ◽  
M. Nordin B. Zakaria ◽  
Suziah Bt Sulaiman
Keyword(s):  
Computer Vision ◽  
Particle Swarm Optimization ◽  
Motion Tracking ◽  
Particle Swarm ◽  
Search Space ◽  
Human Motion ◽  
Video Sequences ◽  
High Dimensional ◽  
Swarm Optimization ◽  
Human Motion Tracking

Automatic human motion tracking in video sequences is one of the most frequently tackled tasks in computer vision community. The goal of human motion capture is to estimate the joints angles of human body at any time. However, this is one of the most challenging problem in computer vision and pattern recognition due to the high-dimensional search space, self-occlusion, and high variability in human appearance. Several approaches have been proposed in the literature using different techniques. However, conventional approaches such as stochastic particle filtering have shortcomings in computational cost, slowness of convergence, suffers from the curse of dimensionality and demand a high number of evaluations to achieve accurate results. Particle swarm optimization (PSO) is a population-based globalized search algorithm which has been successfully applied to address human motion tracking problem and produced better results in high-dimensional search space. This paper presents a systematic literature survey on the PSO algorithm and its variants to human motion tracking. An attempt is made to provide a guide for the researchers working in the field of PSO based human motion tracking from video sequences. Additionally, the paper also presents the performance of various model evaluation search strategies within PSO tracking framework for 3D pose tracking.

scholarly journals Cooperative Quantum-Behaved Particle Swarm Optimization with Dynamic Varying Search Areas and Lévy Flight Disturbance

2014 ◽  
Vol 2014 ◽  
pp. 1-11
Author(s):  
Desheng Li
Keyword(s):  
Particle Swarm Optimization ◽  
Job Shop ◽  
Job Shop Scheduling ◽  
Particle Swarm ◽  
Search Space ◽  
Swarm Optimization ◽  
Local Optima ◽  
Stochastic Disturbance ◽  
Novel Variant ◽  
Benchmark Test Functions

This paper proposes a novel variant of cooperative quantum-behaved particle swarm optimization (CQPSO) algorithm with two mechanisms to reduce the search space and avoid the stagnation, called CQPSO-DVSA-LFD. One mechanism is called Dynamic Varying Search Area (DVSA), which takes charge of limiting the ranges of particles’ activity into a reduced area. On the other hand, in order to escape the local optima, Lévy flights are used to generate the stochastic disturbance in the movement of particles. To test the performance of CQPSO-DVSA-LFD, numerical experiments are conducted to compare the proposed algorithm with different variants of PSO. According to the experimental results, the proposed method performs better than other variants of PSO on both benchmark test functions and the combinatorial optimization issue, that is, the job-shop scheduling problem.

scholarly journals Design of Two-Dimensional Recursive Digital Filter Using Multi Particle Swarm Optimization Algorithm

2020 ◽  
Vol 53 (4) ◽  
pp. 559-566
Author(s):  
Lakhdar Kaddouri ◽  
Amel B.H. Adamou-Mitiche ◽  
Lahcene Mitiche
Keyword(s):  
Particle Swarm Optimization ◽  
Optimization Problems ◽  
Particle Swarm ◽  
Pso Algorithm ◽  
Search Space ◽  
Fast Convergence ◽  
Infinite Impulse Response ◽  
Two Dimensional ◽  
Swarm Optimization ◽  
Local Optima

Particle Swarm Optimization (PSO) is an evolutionary algorithm widely used in optimization problems. It is characterized by a fast convergence, which can lead the algorithm to stagnate in local optima. In the present paper, a new Multi-PSO algorithm for the design of two-dimensional infinite impulse response (IIR) filters is built. It is based on the standard PSO and uses a new initialization strategy. This strategy is relayed to two types of swarms: a principal and auxiliaries. To improve the performance of the algorithm, the search space is divided into several areas, which allows a best covering and leading to a better exploration in each zone separately. This solved the problem of fast convergence in standard PSO. The results obtained demonstrate the effectiveness of the Multi-PSO algorithm in the filter coefficients optimization.

Sign in / Sign up

Export Citation Format

Share Document