Scalable Differential Evolutionary Clustering Algorithm for Big Data Using Map-Reduce Paradigm

2017 ◽  
Vol 8 (1) ◽  
pp. 45-60 ◽  
Author(s):  
Zakaria Benmounah ◽  
Souham Meshoul ◽  
Mohamed Batouche
Keyword(s):  
Differential Evolution ◽  
Clustering Algorithm ◽  
Programming Model ◽  
Differential Evolution Algorithm ◽  
Map Reduce ◽  
Data Sets ◽  
Traditional Use ◽  
Evolutionary Clustering ◽  
De Algorithm ◽  
Large Sets

One of the remarkable results of the rapid advances in information technology is the production of tremendous amounts of data sets, so large or complex that available processing methods are inadequate, among these methods cluster analysis. Clustering becomes more challenging and complex. In this paper, the authors describe a highly scalable Differential Evolution (DE) algorithm based on map-reduce programming model. The traditional use of DE to deal with clustering of large sets of data is so time-consuming that it is not feasible. On the other hand, map-reduce is a programming model emerged lately to allow the design of parallel and distributed approaches. In this paper, four stages map-reduce differential evolution algorithm termed as DE-MRC is presented; each of these four phases is a map-reduce process and dedicated to a particular DE operation. DE-MRC has been tested on a real parallel platform of 128 computers connected with each other and more than 30 GB of data. Experimental results show the high scalability and robustness of DE-MRC.

scholarly journals Differential evolution algorithm of soft island model based on K-means clustering

2020 ◽  
Vol 19 (3) ◽  
pp. 1548
Author(s):  
Xujie Tan ◽  
Seong-Yoon Shin
Keyword(s):  
Differential Evolution ◽  
Clustering Algorithm ◽  
Differential Evolution Algorithm ◽  
Population Based ◽  
Island Model ◽  
Benchmark Problems ◽  
De Algorithm ◽  
Mutation Strategies ◽  
The Individual ◽  
And Control

<p>Differential evolution (DE) is a highly effective evolutionary algorithm. However, the performance of DE depends on strategies and control parameters. The combination of many strategies helps balance the exploitation and exploration of DE. In this study, a multi-population based on<em> k</em>-means clustering is proposed to realize an ensemble of multiple strategies, thereby resulting in a new DE variant, namely KSDE, where similar individuals in the population implement the same mutation strategies, and dissimilar subpopulations migrate information through the soft island model (SIM). Firstly, the population is virtually divided into <em>k</em> subpopulations by the <em>k</em>-means clustering algorithm. Secondly, the individual specific mutation scheme is selected from a strategy pool by random method. Finally, the migration of subpopulation information is done using soft island model. The performance of the KSDE algorithm is evaluated on 13 benchmark problems. The experiments show that KSDE algorithm improves the performance of the DE algorithm.<strong></strong></p>

scholarly journals A Generalized National Planning Approach for Admission Capacity in Higher Education: A Nonlinear Integer Goal Programming Model with a Novel Differential Evolution Algorithm

2016 ◽  
Vol 2016 ◽  
pp. 1-14 ◽  
Author(s):  
Said Ali El-Qulity ◽  
Ali Wagdy Mohamed
Keyword(s):  
Higher Education ◽  
Differential Evolution ◽  
Goal Programming ◽  
Capacity Planning ◽  
Programming Model ◽  
Differential Evolution Algorithm ◽  
Solution Quality ◽  
De Algorithm ◽  
Planning Approach ◽  
Goal Programming Model

This paper proposes a nonlinear integer goal programming model (NIGPM) for solving the general problem of admission capacity planning in a country as a whole. The work aims to satisfy most of the required key objectives of a country related to the enrollment problem for higher education. The system general outlines are developed along with the solution methodology for application to the time horizon in a given plan. The up-to-date data for Saudi Arabia is used as a case study and a novel evolutionary algorithm based on modified differential evolution (DE) algorithm is used to solve the complexity of the NIGPM generated for different goal priorities. The experimental results presented in this paper show their effectiveness in solving the admission capacity for higher education in terms of final solution quality and robustness.

scholarly journals Research on the inverse kinematics of manipulator using an improved self-adaptive mutation differential evolution algorithm

2021 ◽  
Vol 18 (3) ◽  
pp. 172988142110144
Author(s):  
Qianqian Zhang ◽  
Daqing Wang ◽  
Lifu Gao
Keyword(s):  
Differential Evolution ◽  
Objective Function ◽  
Inverse Kinematics ◽  
Differential Evolution Algorithm ◽  
Weight Coefficient ◽  
Adaptive Mutation ◽  
Feasible Region ◽  
Serial Manipulators ◽  
De Algorithm ◽  
Self Adaptive

To assess the inverse kinematics (IK) of multiple degree-of-freedom (DOF) serial manipulators, this article proposes a method for solving the IK of manipulators using an improved self-adaptive mutation differential evolution (DE) algorithm. First, based on the self-adaptive DE algorithm, a new adaptive mutation operator and adaptive scaling factor are proposed to change the control parameters and differential strategy of the DE algorithm. Then, an error-related weight coefficient of the objective function is proposed to balance the weight of the position error and orientation error in the objective function. Finally, the proposed method is verified by the benchmark function, the 6-DOF and 7-DOF serial manipulator model. Experimental results show that the improvement of the algorithm and improved objective function can significantly improve the accuracy of the IK. For the specified points and random points in the feasible region, the proportion of accuracy meeting the specified requirements is increased by 22.5% and 28.7%, respectively.

USING AN IMPROVED BEE MEMORY DIFFERENTIAL EVOLUTION ALGORITHM FOR PARAMETER ESTIMATION TO SIMULATE BIOCHEMICAL PATHWAYS

2014 ◽  
Vol 22 (01) ◽  
pp. 101-121 ◽  
Author(s):  
CHUII KHIM CHONG ◽  
MOHD SABERI MOHAMAD ◽  
SAFAAI DERIS ◽  
MOHD SHAHIR SHAMSIR ◽  
LIAN EN CHAI ◽  
...  
Keyword(s):  
Differential Evolution ◽  
Differential Evolution Algorithm ◽  
Estimation Algorithm ◽  
Convergence Time ◽  
Unknown Parameters ◽  
Estimation Algorithms ◽  
Bee Colony ◽  
De Algorithm ◽  
Cell Cycle Pathway ◽  
Parameter Values

When analyzing a metabolic pathway in a mathematical model, it is important that the essential parameters are estimated correctly. However, this process often faces few problems like when the number of unknown parameters increase, trapping of data in the local minima, repeated exposure to bad results during the search process and occurrence of noisy data. Thus, this paper intends to present an improved bee memory differential evolution (IBMDE) algorithm to solve the mentioned problems. This is a hybrid algorithm that combines the differential evolution (DE) algorithm, the Kalman filter, artificial bee colony (ABC) algorithm, and a memory feature. The aspartate and threonine biosynthesis pathway, and cell cycle pathway are the metabolic pathways used in this paper. For three production simulation pathways, the IBMDE managed to robustly produce the estimated optimal kinetic parameter values with significantly reduced errors. Besides, it also demonstrated faster convergence time compared to the Nelder–Mead (NM), simulated annealing (SA), the genetic algorithm (GA) and DE, respectively. Most importantly, the kinetic parameters that were generated by the IBMDE have improved the production rates of desired metabolites better than other estimation algorithms. Meanwhile, the results proved that the IBMDE is a reliable estimation algorithm.

Research and Improvement on K-Means Clustering Algorithm

2013 ◽  
Vol 756-759 ◽  
pp. 3231-3235
Author(s):  
Xue Mei Wang ◽  
Jin Bo Wang
Keyword(s):  
Differential Evolution ◽  
Clustering Algorithm ◽  
Differential Evolution Algorithm ◽  
Optimal Solution ◽  
Clustering Quality ◽  
Local Optimal Solution ◽  
Evolution Algorithm ◽  
Clustering Center ◽  
Searching Ability ◽  
Improved Differential Evolution Algorithm

According to the defects of classical k-means clustering algorithm such as sensitive to the initial clustering center selection, the poor global search ability, falling into the local optimal solution. A differential evolution algorithm which was a kind of a heuristic global optimization algorithm based on population was introduced in this article, then put forward an improved differential evolution algorithm combined with k-means clustering algorithm at the same time. The experiments showed that the method has solved initial centers optimization problem of k-means clustering algorithm well, had a better searching ability,and more effectively improved clustering quality and convergence speed.

scholarly journals WSNs Compressed Sensing Signal Reconstruction Based on Improved Kernel Fuzzy Clustering and Discrete Differential Evolution Algorithm

2019 ◽  
Vol 2019 ◽  
pp. 1-9
Author(s):  
Zhou-zhou Liu ◽  
Shi-ning Li
Keyword(s):  
Compressed Sensing ◽  
Differential Evolution ◽  
Fuzzy Clustering ◽  
Clustering Algorithm ◽  
Signal Reconstruction ◽  
Differential Evolution Algorithm ◽  
Reconstruction Algorithm ◽  
Reconstruction Algorithms ◽  
Evolution Algorithm ◽  
Discrete Differential Evolution

To reconstruct compressed sensing (CS) signal fast and accurately, this paper proposes an improved discrete differential evolution (IDDE) algorithm based on fuzzy clustering for CS reconstruction. Aiming to overcome the shortcomings of traditional CS reconstruction algorithm, such as heavy dependence on sparsity and low precision of reconstruction, a discrete differential evolution (DDE) algorithm based on improved kernel fuzzy clustering is designed. In this algorithm, fuzzy clustering algorithm is used to analyze the evolutionary population, which improves the pertinence and scientificity of population learning evolution while realizing effective clustering. The differential evolutionary particle coding method and evolutionary mechanism are redefined. And the improved fuzzy clustering discrete differential evolution algorithm is applied to CS reconstruction algorithm, in which signal with unknown sparsity is considered as particle coding. Then the wireless sensor networks (WSNs) sparse signal is accurately reconstructed through the iterative evolution of population. Finally, simulations are carried out in the WSNs data acquisition environment. Results show that compared with traditional reconstruction algorithms such as StOMP, the reconstruction accuracy of the algorithm proposed in this paper is improved by 36.4-51.9%, and the reconstruction time is reduced by 15.1-31.3%.

scholarly journals Optimization Of Raw Material Inventory Costs In The Food Supply Chain Using Differential Evolution Algorithm

2018 ◽  
Vol 73 ◽  
pp. 13016
Author(s):  
Mara Huriga Priymasiwi ◽  
Mustafid
Keyword(s):  
Supply Chain ◽  
Differential Evolution ◽  
Food Industry ◽  
Differential Evolution Algorithm ◽  
Optimization Method ◽  
Raw Material ◽  
Inventory Cost ◽  
De Algorithm ◽  
Material Requirement ◽  
Total Inventory

The management of raw material inventory is used to overcome the problems occuring especially in the food industry to achieve effectiveness, timeliness, and high service levels which are contrary to the problem of effectiveness and cost efficiency. The inventory control system is built to achieve the optimization of raw material inventory cost in the supply chain in food industry. This research represents Differential Evolution (DE) algorithm as optimization method by minimizing total inventory based on amount of raw material requirement, purchasing cost, saefty stock and reorder time. With the population size, the parameters of mutation control, crossover parameters and the number of iterations respectively 80, 0.8, 0.5, 200. With the amount of safety stock at the company 7213.95 obtained a total inventory cost decrease of 39.95%. Result indicate that the use of DE algorithm help providein efficient amount, time and cost.

Differential Evolution Algorithm with Triangular Adaptive Control Parameter for SHEPWM Switching Pattern Optimization

2017 ◽  
Vol 8 (3) ◽  
pp. 1381
Author(s):  
Ismail Yusuf ◽  
Ayong Hiendro ◽  
F. Trias Pontia Wigyarianto ◽  
Kho Hie Khwee
Keyword(s):  
Adaptive Control ◽  
Differential Evolution ◽  
Control Parameter ◽  
Pulse Width Modulation ◽  
Differential Evolution Algorithm ◽  
De Algorithm ◽  
Harmonic Elimination ◽  
Switching Patterns ◽  
Convergence Process ◽  
Speed Up

Differential evolution (DE) algorithm has been applied as a powerful tool to find optimum switching angles for selective harmonic elimination pulse width modulation (SHEPWM) inverters. However, the DE’s performace is very dependent on its control parameters. Conventional DE generally uses either trial and error mechanism or tuning technique to determine appropriate values of the control paramaters. The disadvantage of this process is that it is very time comsuming. In this paper, an adaptive control parameter is proposed in order to speed up the DE algorithm in optimizing SHEPWM switching angles precisely. The proposed adaptive control parameter is proven to enhance the convergence process of the DE algorithm without requiring initial guesses. The results for both negative and positive modulation index (<em>M</em>) also indicate that the proposed adaptive DE is superior to the conventional DE in generating SHEPWM switching patterns

Intrusion Detection Based on Dynamic Gemini Population DE-K-mediods Clustering on Hadoop Platform

2020 ◽  
pp. 2150001
Author(s):  
Wentie Wu ◽  
Shengchao Xu
Keyword(s):  
Cloud Computing ◽  
Intrusion Detection ◽  
Differential Evolution ◽  
Large Scale ◽  
Clustering Algorithm ◽  
Detection System ◽  
Differential Evolution Algorithm ◽  
Detection Algorithm ◽  
Local Optimum ◽  
Evolution Algorithm

In view of the fact that the existing intrusion detection system (IDS) based on clustering algorithm cannot adapt to the large-scale growth of system logs, a K-mediods clustering intrusion detection algorithm based on differential evolution suitable for cloud computing environment is proposed. First, the differential evolution algorithm is combined with the K-mediods clustering algorithm in order to use the powerful global search capability of the differential evolution algorithm to improve the convergence efficiency of large-scale data sample clustering. Second, in order to further improve the optimization ability of clustering, a dynamic Gemini population scheme was adopted to improve the differential evolution algorithm, thereby maintaining the diversity of the population while improving the problem of being easily trapped into a local optimum. Finally, in the intrusion detection processing of big data, the optimized clustering algorithm is designed in parallel under the Hadoop Map Reduce framework. Simulation experiments were performed in the open source cloud computing framework Hadoop cluster environment. Experimental results show that the overall detection effect of the proposed algorithm is significantly better than the existing intrusion detection algorithms.

Enhanced Directed Differential Evolution Algorithm for Solving Constrained Engineering Optimization Problems

2019 ◽  
Vol 10 (1) ◽  
pp. 1-28 ◽  
Author(s):  
Ali Wagdy Mohamed ◽  
Ali Khater Mohamed ◽  
Ehab Z. Elfeky ◽  
Mohamed Saleh
Keyword(s):  
Differential Evolution ◽  
Optimization Problems ◽  
Differential Evolution Algorithm ◽  
Test Problems ◽  
New Mutation ◽  
De Algorithm ◽  
Engineering Problems ◽  
Evolution Algorithm ◽  
Mutation Scheme ◽  
Exploration Exploitation

The performance of Differential Evolution is significantly affected by the mutation scheme, which attracts many researchers to develop and enhance the mutation scheme in DE. In this article, the authors introduce an enhanced DE algorithm (EDDE) that utilizes the information given by good individuals and bad individuals in the population. The new mutation scheme maintains effectively the exploration/exploitation balance. Numerical experiments are conducted on 24 test problems presented in CEC'2006, and five constrained engineering problems from the literature for verifying and analyzing the performance of EDDE. The presented algorithm showed competitiveness in some cases and superiority in other cases in terms of robustness, efficiency and quality the of the results.

Sign in / Sign up

Export Citation Format

Share Document