scholarly journals Performance of a Novel Chaotic Firefly Algorithm with Enhanced Exploration for Tackling Global Optimization Problems: Application for Dropout Regularization

Mathematics ◽  
2021 ◽  
Vol 9 (21) ◽  
pp. 2705
Author(s):  
Nebojsa Bacanin ◽  
Ruxandra Stoean ◽  
Miodrag Zivkovic ◽  
Aleksandar Petrovic ◽  
Tarik A. Rashid ◽  
...  
Keyword(s):  
Neural Networks ◽  
Global Optimization ◽  
Deep Learning ◽  
Firefly Algorithm ◽  
Optimization Problems ◽  
Wide Spectrum ◽  
Statistical Tests ◽  
Dropout Rate ◽  
Original Method ◽  
Benchmark Datasets

Swarm intelligence techniques have been created to respond to theoretical and practical global optimization problems. This paper puts forward an enhanced version of the firefly algorithm that corrects the acknowledged drawbacks of the original method, by an explicit exploration mechanism and a chaotic local search strategy. The resulting augmented approach was theoretically tested on two sets of bound-constrained benchmark functions from the CEC suites and practically validated for automatically selecting the optimal dropout rate for the regularization of deep neural networks. Despite their successful applications in a wide spectrum of different fields, one important problem that deep learning algorithms face is overfitting. The traditional way of preventing overfitting is to apply regularization; the first option in this sense is the choice of an adequate value for the dropout parameter. In order to demonstrate its ability in finding an optimal dropout rate, the boosted version of the firefly algorithm has been validated for the deep learning subfield of convolutional neural networks, with respect to five standard benchmark datasets for image processing: MNIST, Fashion-MNIST, Semeion, USPS and CIFAR-10. The performance of the proposed approach in both types of experiments was compared with other recent state-of-the-art methods. To prove that there are significant improvements in results, statistical tests were conducted. Based on the experimental data, it can be concluded that the proposed algorithm clearly outperforms other approaches.

scholarly journals Gender-Based Deep Learning Firefly Optimization Method for Test Data Generation

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Wenning Zhang ◽  
Chongyang Jiao ◽  
Qinglei Zhou ◽  
Yang Liu ◽  
Ting Xu
Keyword(s):  
Deep Learning ◽  
Test Data ◽  
Firefly Algorithm ◽  
Optimization Problems ◽  
Optimization Method ◽  
Test Data Generation ◽  
Data Generation ◽  
Whole Space ◽  
Automatic Test Data Generation ◽  
Gender Based

Software testing is a widespread validation means of software quality assurance in industry. Intelligent optimization algorithms have been proved to be an effective way of automatic test data generation. Firefly algorithm has received extensive attention and been widely used to solve optimization problems because of less parameters and simple implement. To overcome slow convergence rate and low accuracy of the firefly algorithm, a novel firefly algorithm with deep learning is proposed to generate structural test data. Initially, the population is divided into male subgroup and female subgroup. Following the randomly attracted model, each male firefly will be attracted by another randomly selected female firefly to focus on global search in whole space. Each female firefly implements local search under the leadership of the general center firefly, constructed based on historical experience with deep learning. At the final period of searching, chaos search is conducted near the best firefly to improve search accuracy. Simulation results show that the proposed algorithm can achieve better performance in terms of success coverage rate, coverage time, and diversity of solutions.

scholarly journals Unsupervised phenotypic analysis of cellular images with multi-scale convolutional neural networks

2018 ◽  
Author(s):  
William J. Godinez ◽  
Imtiaz Hossain ◽  
Xian Zhang
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Large Scale ◽  
Cellular Imaging ◽  
Phenotypic Analysis ◽  
Learning Methods ◽  
Multi Scale ◽  
Full Resolution ◽  
Benchmark Datasets ◽  
Microscopy Images

AbstractLarge-scale cellular imaging and phenotyping is a widely adopted strategy for understanding biological systems and chemical perturbations. Quantitative analysis of cellular images for identifying phenotypic changes is a key challenge within this strategy, and has recently seen promising progress with approaches based on deep neural networks. However, studies so far require either pre-segmented images as input or manual phenotype annotations for training, or both. To address these limitations, we have developed an unsupervised approach that exploits the inherent groupings within cellular imaging datasets to define surrogate classes that are used to train a multi-scale convolutional neural network. The trained network takes as input full-resolution microscopy images, and, without the need for segmentation, yields as output feature vectors that support phenotypic profiling. Benchmarked on two diverse benchmark datasets, the proposed approach yields accurate phenotypic predictions as well as compound potency estimates comparable to the state-of-the-art. More importantly, we show that the approach identifies novel cellular phenotypes not included in the manual annotation nor detected by previous studies.Author summaryCellular microscopy images provide detailed information about how cells respond to genetic or chemical treatments, and have been widely and successfully used in basic research and drug discovery. The recent breakthrough of deep learning methods for natural imaging recognition tasks has triggered the development and application of deep learning methods to cellular images to understand how cells change upon perturbation. Although successful, deep learning studies so far either can only take images of individual cells as input or require human experts to label a large amount of images. In this paper, we present an unsupervised deep learning approach that, without any human annotation, analyzes directly full-resolution microscopy images displaying typically hundreds of cells. We apply the approach to two benchmark datasets, and show that the approach identifies novel visual phenotypes not detected by previous studies.

scholarly journals Evolutionary Spiking Neural Networks for Solving Supervised Classification Problems

2019 ◽  
Vol 2019 ◽  
pp. 1-13 ◽  
Author(s):  
G. López-Vázquez ◽  
M. Ornelas-Rodriguez ◽  
A. Espinal ◽  
J. A. Soria-Alcaraz ◽  
A. Rojas-Domínguez ◽  
...  
Keyword(s):  
Neural Networks ◽  
Supervised Classification ◽  
Fitness Function ◽  
Statistical Tests ◽  
Search Space ◽  
Spiking Neural Networks ◽  
Classification Problems ◽  
Benchmark Datasets ◽  
Explicit Training ◽  
Better Than

This paper presents a grammatical evolution (GE)-based methodology to automatically design third generation artificial neural networks (ANNs), also known as spiking neural networks (SNNs), for solving supervised classification problems. The proposal performs the SNN design by exploring the search space of three-layered feedforward topologies with configured synaptic connections (weights and delays) so that no explicit training is carried out. Besides, the designed SNNs have partial connections between input and hidden layers which may contribute to avoid redundancies and reduce the dimensionality of input feature vectors. The proposal was tested on several well-known benchmark datasets from the UCI repository and statistically compared against a similar design methodology for second generation ANNs and an adapted version of that methodology for SNNs; also, the results of the two methodologies and the proposed one were improved by changing the fitness function in the design process. The proposed methodology shows competitive and consistent results, and the statistical tests support the conclusion that the designs produced by the proposal perform better than those produced by other methodologies.

Dead Sea Water Levels Analysis Using Artificial Neural Networks and Firefly Algorithm

2020 ◽  
Vol 11 (3) ◽  
pp. 19-29
Author(s):  
Nawaf N. Hamadneh
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Water Level ◽  
Firefly Algorithm ◽  
Sea Water ◽  
Statistical Tests ◽  
Dead Sea ◽  
Water Levels ◽  
Artificial Neural ◽  
The Dead

In this study, the performance of adaptive multilayer perceptron neural network (MLPNN) for predicting the Dead Sea water level is discussed. Firefly Algorithm (FFA), as an optimization algorithm is used for training the neural networks. To propose the MLPNN-FFA model, Dead Sea water levels over the period 1810–2005 are applied to train MLPNN. Statistical tests evaluate the accuracy of the hybrid MLPNN-FFA model. The predicted values of the proposed model were compared with the results obtained by another method. The results reveal that the artificial neural network (ANN) models exhibit high accuracy and reliability for the prediction of the Dead Sea water levels. The results also reveal that the Dead Sea water level would be around -450 until 2050.

Hybrid Genetic Firefly Algorithm for Global Optimization Problems

2017 ◽  
Vol 49 (004) ◽  
pp. 899--906 ◽  
Author(s):  
M. ASIM ◽  
W. K. MASHWANI ◽  
M. A. JAN
Keyword(s):  
Global Optimization ◽  
Firefly Algorithm ◽  
Optimization Problems

scholarly journals Image Denoising Using Nonlocal Regularized Deep Image Prior

Symmetry ◽  
2021 ◽  
Vol 13 (11) ◽  
pp. 2114
Author(s):  
Zhonghua Xie ◽  
Lingjun Liu ◽  
Zhongliang Luo ◽  
Jianfeng Huang
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Image Denoising ◽  
Deep Neural Networks ◽  
Optimization Problems ◽  
Training Data ◽  
Superior Performance ◽  
Convolutional Network ◽  
Image Prior ◽  
Deep Image

Deep neural networks have shown great potential in various low-level vision tasks, leading to several state-of-the-art image denoising techniques. Training a deep neural network in a supervised fashion usually requires the collection of a great number of examples and the consumption of a significant amount of time. However, the collection of training samples is very difficult for some application scenarios, such as the full-sampled data of magnetic resonance imaging and the data of satellite remote sensing imaging. In this paper, we overcome the problem of a lack of training data by using an unsupervised deep-learning-based method. Specifically, we propose a deep-learning-based method based on the deep image prior (DIP) method, which only requires a noisy image as training data, without any clean data. It infers the natural images with random inputs and the corrupted observation with the help of performing correction via a convolutional network. We improve the original DIP method as follows: Firstly, the original optimization objective function is modified by adding nonlocal regularizers, consisting of a spatial filter and a frequency domain filter, to promote the gradient sparsity of the solution. Secondly, we solve the optimization problem with the alternating direction method of multipliers (ADMM) framework, resulting in two separate optimization problems, including a symmetric U-Net training step and a plug-and-play proximal denoising step. As such, the proposed method exploits the powerful denoising ability of both deep neural networks and nonlocal regularizations. Experiments validate the effectiveness of leveraging a combination of DIP and nonlocal regularizers, and demonstrate the superior performance of the proposed method both quantitatively and visually compared with the original DIP method.

scholarly journals Adaptive Region Embedding for Text Classification

2019 ◽  
Vol 33 ◽  
pp. 7314-7321
Author(s):  
Liuyu Xiang ◽  
Xiaoming Jin ◽  
Lan Yi ◽  
Guiguang Ding
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
State Of The Art ◽  
Context Information ◽  
Great Success ◽  
Recurrent Networks ◽  
Learning Models ◽  
Benchmark Datasets

Deep learning models such as convolutional neural networks and recurrent networks are widely applied in text classification. In spite of their great success, most deep learning models neglect the importance of modeling context information, which is crucial to understanding texts. In this work, we propose the Adaptive Region Embedding to learn context representation to improve text classification. Specifically, a metanetwork is learned to generate a context matrix for each region, and each word interacts with its corresponding context matrix to produce the regional representation for further classification. Compared to previous models that are designed to capture context information, our model contains less parameters and is more flexible. We extensively evaluate our method on 8 benchmark datasets for text classification. The experimental results prove that our method achieves state-of-the-art performances and effectively avoids word ambiguity.

scholarly journals A New Hybrid BA_ABC Algorithm for Global Optimization Problems

Mathematics ◽  
2020 ◽  
Vol 8 (10) ◽  
pp. 1749 ◽  
Author(s):  
Gülnur Yildizdan ◽  
Ömer Kaan Baykan
Keyword(s):  
Global Optimization ◽  
Large Scale ◽  
Optimization Problems ◽  
Statistical Tests ◽  
Global Search ◽  
Small Scale ◽  
Test Functions ◽  
Engineering Design Problems ◽  
Scale Test ◽  
Search Capability

Bat Algorithm (BA) and Artificial Bee Colony Algorithm (ABC) are frequently used in solving global optimization problems. Many new algorithms in the literature are obtained by modifying these algorithms for both constrained and unconstrained optimization problems or using them in a hybrid manner with different algorithms. Although successful algorithms have been proposed, BA’s performance declines in complex and large-scale problems are still an ongoing problem. The inadequate global search capability of the BA resulting from its algorithm structure is the major cause of this problem. In this study, firstly, inertia weight was added to the speed formula to improve the search capability of the BA. Then, a new algorithm that operates in a hybrid manner with the ABC algorithm, whose diversity and global search capability is stronger than the BA, was proposed. The performance of the proposed algorithm (BA_ABC) was examined in four different test groups, including classic benchmark functions, CEC2005 small-scale test functions, CEC2010 large-scale test functions, and classical engineering design problems. The BA_ABC results were compared with different algorithms in the literature and current versions of the BA for each test group. The results were interpreted with the help of statistical tests. Furthermore, the contribution of BA and ABC algorithms, which constitute the hybrid algorithm, to the solutions is examined. The proposed algorithm has been found to produce successful and acceptable results.

Sign in / Sign up

Export Citation Format

Share Document