A gradient-based automatic optimization CNN framework for EEG state recognition

2021 ◽  
Author(s):  
He Wang ◽  
Xinshan Zhu ◽  
Pinyin Chen ◽  
Yuxuan Yang ◽  
Chao Ma ◽  
...  
Keyword(s):  
Deep Learning ◽  
High Performance ◽  
Search Algorithm ◽  
Search Space ◽  
Structure Design ◽  
Eeg Signal ◽  
Eeg Signals ◽  
Automatic Feature Extraction ◽  
Model Search ◽  
Gradient Based

Abstract The Electroencephalogram (EEG) signal, as a data carrier that can contain a large amount of information about the human brain in different states, is one of the most widely used metrics for assessing human psychophysiological states. Among a variety of analysis methods, deep learning, especially convolutional neural network (CNN), has achieved remarkable results in recent years as a method to effectively extract features from EEG signals. Although deep learning has the advantages of automatic feature extraction and effective classification, it also faces difficulties in network structure design and requires an army of prior knowledge. Automating the design of these hyperparameters can therefore save experts' time and manpower. Neural architecture search techniques have thus emerged. In this paper, based on an existing gradient-based NAS algorithm, PC-DARTS, with targeted improvements and optimizations for the characteristics of EEG signals. Specifically, we establish the model architecture step by step based on the manually designed deep learning models for EEG discrimination by retaining the framework of the search algorithm and performing targeted optimization of the model search space. Corresponding features are extracted separately according to the frequency domain, time domain characteristics of the EEG signal and the spatial position of the EEG electrode. The architecture was applied to EEG-based emotion recognition and driver drowsiness assessment tasks. The results illustrate that compared with the existing methods, the model architecture obtained in this paper can achieve competitive overall accuracy and better standard deviation in both tasks. Therefore, this approach is an effective migration of NAS technology into the field of EEG analysis and has great potential to provide high-performance results for other types of classification and prediction tasks. This can effectively reduce the time cost for researchers and facilitate the application of CNN in more areas.

scholarly journals Binary Spring Search Algorithm for Solving Various Optimization Problems

2021 ◽  
Vol 11 (3) ◽  
pp. 1286 ◽  
Author(s):  
Mohammad Dehghani ◽  
Zeinab Montazeri ◽  
Ali Dehghani ◽  
Om P. Malik ◽  
Ruben Morales-Menendez ◽  
...  
Keyword(s):  
Optimization Algorithm ◽  
High Performance ◽  
Optimization Problems ◽  
Search Algorithm ◽  
Gravitational Search Algorithm ◽  
Bat Algorithm ◽  
Search Space ◽  
Binary Particle Swarm Optimization ◽  
Dragonfly Algorithm ◽  
Grasshopper Optimization Algorithm

One of the most powerful tools for solving optimization problems is optimization algorithms (inspired by nature) based on populations. These algorithms provide a solution to a problem by randomly searching in the search space. The design’s central idea is derived from various natural phenomena, the behavior and living conditions of living organisms, laws of physics, etc. A new population-based optimization algorithm called the Binary Spring Search Algorithm (BSSA) is introduced to solve optimization problems. BSSA is an algorithm based on a simulation of the famous Hooke’s law (physics) for the traditional weights and springs system. In this proposal, the population comprises weights that are connected by unique springs. The mathematical modeling of the proposed algorithm is presented to be used to achieve solutions to optimization problems. The results were thoroughly validated in different unimodal and multimodal functions; additionally, the BSSA was compared with high-performance algorithms: binary grasshopper optimization algorithm, binary dragonfly algorithm, binary bat algorithm, binary gravitational search algorithm, binary particle swarm optimization, and binary genetic algorithm. The results show the superiority of the BSSA. The results of the Friedman test corroborate that the BSSA is more competitive.

scholarly journals EEG signal classification based on SVM with improved squirrel search algorithm

2020 ◽  
Vol 0 (0) ◽  
Author(s):  
Miao Shi ◽  
Chao Wang ◽  
Xian-Zhe Li ◽  
Ming-Qiang Li ◽  
Lu Wang ◽  
...  
Keyword(s):  
Performance Test ◽  
Search Algorithm ◽  
Comparison Method ◽  
Support Vector ◽  
Data Sets ◽  
Eeg Signal ◽  
Eeg Signals ◽  
Svm Model ◽  
Bioelectrical Signal ◽  
Point Set

AbstractElectroencephalography (EEG) is a complex bioelectrical signal. Analysis of which can provide researchers with useful physiological information. In order to recognize and classify EEG signals, a pattern recognition method for optimizing the support vector machine (SVM) by using improved squirrel search algorithm (ISSA) is proposed. The EEG signal is preprocessed, with its time domain features being extracted and directed to the SVM as feature vectors for classification and identification. In this paper, the method of good point set is used to initialize the population position, chaos and reverse learning mechanism are introduced into the algorithm. The performance test of the improved squirrel algorithm (ISSA) is carried out by using the benchmark function. As can be seen from the statistical analysis of the results, the exploration ability and convergence speed of the algorithm are improved. This is then used to optimize SVM parameters. ISSA-SVM model is established and built for classification of EEG signals, compared with other common SVM parameter optimization models. For data sets, the average classification accuracy of this method is 85.9%. This result is an improvement of 2–5% over the comparison method.

Weakly Supervised Deep Neural Network for Bearing Fault Diagnosis

2020 ◽  
Author(s):  
Daisuke Miki ◽  
Kazuyuki Demachi
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Fault Detection ◽  
High Performance ◽  
Time Series Data ◽  
Vibration Signal ◽  
Detection Methods ◽  
Series Data ◽  
Vibration Signals ◽  
Automatic Feature Extraction

Abstract Bearings are one of the main components of rotating machinery, and their failure is one of the most common cause of mechanical failure. Therefore, many fault detection methods based on artificial intelligence, such as machine learning and deep learning, have been proposed. Particularly, with recent advances in deep learning, many anomaly detection methods based on deep neural networks (DNN) have been proposed. DNNs provide high-performance recognition and are easy to implement; however, optimizing DNNs require large annotated datasets. Additionally, the annotation of time-series data, such as abnormal vibration signals, is time consuming. To solve these problems, we proposed a method to automatically extract features from abnormal vibration signals from the time-series data. In this research, we propose a new DNN training method and fault detection method inspired by multi-instance learning. Additionally, we propose a new loss function for optimizing the DNN model that identifies anomalies from a time-series data. Furthermore, to evaluate the feasibility of automatic feature extraction from vibration signal data using the proposed method, we conducted experiments to determine whether anomalies could be detected, identified, and localized in published datasets.

scholarly journals A Streamlined Search Algorithm for Path Modifications of a Safe Robot Manipulator*

2010 ◽  
Vol 1 (2) ◽  
Author(s):  
Nima Najmaei ◽  
Mehrdad R. Kermani
Keyword(s):  
Robot Manipulator ◽  
Search Algorithm ◽  
Search Space ◽  
Planning Problem ◽  
Minimum Number ◽  
Gradient Based ◽  
On Line ◽  
Line Path ◽  
Path Modification ◽  
Path Planning Problem

AbstractIn recent years, the interest in human-robot interactions has added a new dimension to the on-line path planning problem by requiring a method that guarantees a risk-free path. This paper presents a streamlined search algorithm for fast path modification. The algorithm is formulated as an optimization problem that evaluates alternative paths nearby each obstacle. Each path is evaluated based on the value of the danger assigned to that path. To reduce the size of the search space, the minimum number of via points necessary to alter the path is initially obtained using a geometrical method. Given the number of via points, the algorithm proceeds to locate the via points around the obstacle such that the resulting path through these via points satisfies all problem constraints. Obtaining a solution in this way renders a fast algorithm for path modification, while it better avoids problems often encountered in other gradient-based search algorithms. Case studies for two planar robots are provided to highlight some of the advantages of the proposed algorithm. Experimental results using a CRS-F3 robot manipulator validate the effectiveness of the algorithm for applications involving human-robot interactions.

Kinetic Gas Molecule Optimization (KGMO)

2018 ◽  
pp. 114-149
Keyword(s):  
High Performance ◽  
Search Algorithm ◽  
Gravitational Search Algorithm ◽  
Search Space ◽  
Kinetic Theory Of Gases ◽  
Swarm Optimization ◽  
Gas Molecule ◽  
Gas Molecules ◽  
Gravitational Search ◽  
Molecule Interactions

In this chapter, an optimization algorithm that is based on the kinetic energy of gas molecules, namely kinetic gas molecule optimization (KGMO), is introduced. This algorithm has some agents that are gas molecules, which move in the search space; these agents are subject to the kinetic theory of gases, which defines the rules for gas molecule interactions in the model. This algorithm has a good performance in terms of finding the global minima in 23 nonlinear benchmark functions, and the performance is compared with two other benchmark algorithms, namely particle swarm optimization (PSO) and the recently developed high-performance gravitational search algorithm (GSA).

scholarly journals Genetic Algorithm Based Deep Learning Neural Network Structure and Hyperparameter Optimization

2021 ◽  
Vol 11 (2) ◽  
pp. 744
Author(s):  
Sanghyeop Lee ◽  
Junyeob Kim ◽  
Hyeon Kang ◽  
Do-Young Kang ◽  
Jangsik Park
Keyword(s):  
Alzheimer’S Disease ◽  
Genetic Algorithm ◽  
Alzheimer's Disease ◽  
Deep Learning ◽  
Network Structure ◽  
Network Architecture ◽  
High Performance ◽  
Search Space ◽  
Disease Diagnosis ◽  
Research Area

Alzheimer’s disease is one of the major challenges of population ageing, and diagnosis and prediction of the disease through various biomarkers is the key. While the application of deep learning as imaging technologies has recently expanded across the medical industry, empirical design of these technologies is very difficult. The main reason for this problem is that the performance of the Convolutional Neural Networks (CNN) differ greatly depending on the statistical distribution of the input dataset. Different hyperparameters also greatly affect the convergence of the CNN models. With this amount of information, selecting appropriate parameters for the network structure has became a large research area. Genetic Algorithm (GA), is a very popular technique to automatically select a high-performance network architecture. In this paper, we show the possibility of optimising the network architecture using GA, where its search space includes both network structure configuration and hyperparameters. To verify the performance of our Algorithm, we used an amyloid brain image dataset that is used for Alzheimer’s disease diagnosis. As a result, our algorithm outperforms Genetic CNN by 11.73% on a given classification task.

scholarly journals Designing convolutional neural networks with constrained evolutionary piecemeal training

2021 ◽  
Author(s):  
Dolly Sapra ◽  
Andy D. Pimentel
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Classification ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
Network Architecture ◽  
High Performance ◽  
Search Algorithm ◽  
Search Space ◽  
Pareto Optimal Set

AbstractThe automated architecture search methodology for neural networks is known as Neural Architecture Search (NAS). In recent times, Convolutional Neural Networks (CNNs) designed through NAS methodologies have achieved very high performance in several fields, for instance image classification and natural language processing. Our work is in the same domain of NAS, where we traverse the search space of neural network architectures with the help of an evolutionary algorithm which has been augmented with a novel approach of piecemeal-training. In contrast to the previously published NAS techniques, wherein the training with given data is considered an isolated task to estimate the performance of neural networks, our work demonstrates that a neural network architecture and the related weights can be jointly learned by combining concepts of the traditional training process and evolutionary architecture search in a single algorithm. The consolidation has been realised by breaking down the conventional training technique into smaller slices and collating them together with an integrated evolutionary architecture search algorithm. The constraints on architecture search space are placed by limiting its various parameters within a specified range of values, consequently regulating the neural network’s size and memory requirements. We validate this concept on two vastly different datasets, namely, the CIFAR-10 dataset in the domain of image classification, and PAMAP2 dataset in the Human Activity Recognition (HAR) domain. Starting from randomly initialized and untrained CNNs, the algorithm discovers models with competent architectures, which after complete training, reach an accuracy of of 92.5% for CIFAR-10 and 94.36% PAMAP2. We further extend the algorithm to include an additional conflicting search objective: the number of parameters of the neural network. Our multi-objective algorithm produces a Pareto optimal set of neural networks, by optimizing the search for both the accuracy and the parameter count, thus emphasizing the versatility of our approach.

scholarly journals Medical Augmentation (Med-Aug) for Optimal Data Augmentation in Medical Deep Learning Networks

Sensors ◽  
2021 ◽  
Vol 21 (21) ◽  
pp. 7018
Author(s):  
Justin Lo ◽  
Jillian Cardinell ◽  
Alejo Costanzo ◽  
Dafna Sussman
Keyword(s):  
Deep Learning ◽  
High Performance ◽  
Data Augmentation ◽  
Search Algorithm ◽  
High Dimensional ◽  
Learning Networks ◽  
Imaging Data ◽  
Covariance Matrix Adaptation ◽  
Using Data ◽  
Adaptation Evolution

Deep learning (DL) algorithms have become an increasingly popular choice for image classification and segmentation tasks; however, their range of applications can be limited. Their limitation stems from them requiring ample data to achieve high performance and adequate generalizability. In the case of clinical imaging data, images are not always available in large quantities. This issue can be alleviated by using data augmentation (DA) techniques. The choice of DA is important because poor selection can possibly hinder the performance of a DL algorithm. We propose a DA policy search algorithm that offers an extended set of transformations that accommodate the variations in biomedical imaging datasets. The algorithm makes use of the efficient and high-dimensional optimizer Bi-Population Covariance Matrix Adaptation Evolution Strategy (BIPOP-CMA-ES) and returns an optimal DA policy based on any input imaging dataset and a DL algorithm. Our proposed algorithm, Medical Augmentation (Med-Aug), can be implemented by other researchers in related medical DL applications to improve their model’s performance. Furthermore, we present our found optimal DA policies for a variety of medical datasets and popular segmentation networks for other researchers to use in related tasks.

scholarly journals An Efficient and Accurate Iris Recognition Algorithm Based on a Novel Condensed 2-ch Deep Convolutional Neural Network

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3721
Author(s):  
Guoyang Liu ◽  
Weidong Zhou ◽  
Lan Tian ◽  
Wei Liu ◽  
Yingjian Liu ◽  
...  
Keyword(s):  
High Performance ◽  
Iris Recognition ◽  
Recognition Algorithm ◽  
Learning Approaches ◽  
Large Database ◽  
Computational Burden ◽  
Automatic Feature Extraction ◽  
Training Samples ◽  
Gradient Based ◽  
Iris Identification

Recently, deep learning approaches, especially convolutional neural networks (CNNs), have attracted extensive attention in iris recognition. Though CNN-based approaches realize automatic feature extraction and achieve outstanding performance, they usually require more training samples and higher computational complexity than the classic methods. This work focuses on training a novel condensed 2-channel (2-ch) CNN with few training samples for efficient and accurate iris identification and verification. A multi-branch CNN with three well-designed online augmentation schemes and radial attention layers is first proposed as a high-performance basic iris classifier. Then, both branch pruning and channel pruning are achieved by analyzing the weight distribution of the model. Finally, fast finetuning is optionally applied, which can significantly improve the performance of the pruned CNN while alleviating the computational burden. In addition, we further investigate the encoding ability of 2-ch CNN and propose an efficient iris recognition scheme suitable for large database application scenarios. Moreover, the gradient-based analysis results indicate that the proposed algorithm is robust to various image contaminations. We comprehensively evaluated our algorithm on three publicly available iris databases for which the results proved satisfactory for real-time iris recognition.

scholarly journals Multilevel Thresholding Segmentation Based on Harmony Search Optimization

2013 ◽  
Vol 2013 ◽  
pp. 1-24 ◽  
Author(s):  
Diego Oliva ◽  
Erik Cuevas ◽  
Gonzalo Pajares ◽  
Daniel Zaldivar ◽  
Marco Perez-Cisneros
Keyword(s):  
High Performance ◽  
Search Algorithm ◽  
Harmony Search ◽  
Optimal Solution ◽  
Search Space ◽  
Harmony Search Algorithm ◽  
Multilevel Thresholding ◽  
Image Histogram ◽  
Computational Overhead ◽  
Search Optimization

In this paper, a multilevel thresholding (MT) algorithm based on the harmony search algorithm (HSA) is introduced. HSA is an evolutionary method which is inspired in musicians improvising new harmonies while playing. Different to other evolutionary algorithms, HSA exhibits interesting search capabilities still keeping a low computational overhead. The proposed algorithm encodes random samples from a feasible search space inside the image histogram as candidate solutions, whereas their quality is evaluated considering the objective functions that are employed by the Otsu’s or Kapur’s methods. Guided by these objective values, the set of candidate solutions are evolved through the HSA operators until an optimal solution is found. Experimental results demonstrate the high performance of the proposed method for the segmentation of digital images.

Sign in / Sign up

Export Citation Format

Share Document