scholarly journals Toward a General-Purpose Heterogeneous Ensemble for Pattern Classification

2015 ◽  
Vol 2015 ◽  
pp. 1-10 ◽  
Author(s):  
Loris Nanni ◽  
Sheryl Brahnam ◽  
Stefano Ghidoni ◽  
Alessandra Lumini
Keyword(s):  
State Of The Art ◽  
Parameter Tuning ◽  
General Purpose ◽  
Extensive Study ◽  
Support Vector ◽  
Svm Classifier ◽  
Random Subspace ◽  
Sum Rule ◽  
Multiple Datasets ◽  
Heterogeneous Ensemble

We perform an extensive study of the performance of different classification approaches on twenty-five datasets (fourteen image datasets and eleven UCI data mining datasets). The aim is to find General-Purpose (GP) heterogeneous ensembles (requiring little to no parameter tuning) that perform competitively across multiple datasets. The state-of-the-art classifiers examined in this study include the support vector machine, Gaussian process classifiers, random subspace of adaboost, random subspace of rotation boosting, and deep learning classifiers. We demonstrate that a heterogeneous ensemble based on the simple fusion by sum rule of different classifiers performs consistently well across all twenty-five datasets. The most important result of our investigation is demonstrating that some very recent approaches, including the heterogeneous ensemble we propose in this paper, are capable of outperforming an SVM classifier (implemented with LibSVM), even when both kernel selection and SVM parameters are carefully tuned for each dataset.

scholarly journals HyP-ABC: A Novel Automated Hyper-Parameter Tuning Algorithm Using Evolutionary Optimization

2021 ◽  
Author(s):  
Leila Zahedi ◽  
Farid Ghareh Mohammadi ◽  
M. Hadi Amini
Keyword(s):  
Parameter Optimization ◽  
Real World ◽  
Optimization Problems ◽  
State Of The Art ◽  
Parameter Tuning ◽  
Gradient Boosting ◽  
Support Vector ◽  
Wide Range ◽  
Extreme Gradient Boosting ◽  
Art Techniques

Machine learning techniques lend themselves as promising decision-making and analytic tools in a wide range of applications. Different ML algorithms have various hyper-parameters. In order to tailor an ML model towards a specific application, a large number of hyper-parameters should be tuned. Tuning the hyper-parameters directly affects the performance (accuracy and run-time). However, for large-scale search spaces, efficiently exploring the ample number of combinations of hyper-parameters is computationally challenging. Existing automated hyper-parameter tuning techniques suffer from high time complexity. In this paper, we propose HyP-ABC, an automatic innovative hybrid hyper-parameter optimization algorithm using the modified artificial bee colony approach, to measure the classification accuracy of three ML algorithms, namely random forest, extreme gradient boosting, and support vector machine. Compared to the state-of-the-art techniques, HyP-ABC is more efficient and has a limited number of parameters to be tuned, making it worthwhile for real-world hyper-parameter optimization problems. We further compare our proposed HyP-ABC algorithm with state-of-the-art techniques. In order to ensure the robustness of the proposed method, the algorithm takes a wide range of feasible hyper-parameter values, and is tested using a real-world educational dataset.

scholarly journals Indoor versus outdoor scene recognition for navigation of a micro aerial vehicle using spatial color gist wavelet descriptors

2019 ◽  
Vol 2 (1) ◽  
Author(s):  
Anitha Ganesan ◽  
Anbarasu Balasubramanian
Keyword(s):  
State Of The Art ◽  
Characteristic Curve ◽  
Scene Recognition ◽  
Color Histogram ◽  
Support Vector ◽  
Svm Classifier ◽  
Scene Classification ◽  
Visual Descriptors ◽  
Outdoor Scenes ◽  
Institute Of Technology

AbstractIn the context of improved navigation for micro aerial vehicles, a new scene recognition visual descriptor, called spatial color gist wavelet descriptor (SCGWD), is proposed. SCGWD was developed by combining proposed Ohta color-GIST wavelet descriptors with census transform histogram (CENTRIST) spatial pyramid representation descriptors for categorizing indoor versus outdoor scenes. A binary and multiclass support vector machine (SVM) classifier with linear and non-linear kernels was used to classify indoor versus outdoor scenes and indoor scenes, respectively. In this paper, we have also discussed the feature extraction methodology of several, state-of-the-art visual descriptors, and four proposed visual descriptors (Ohta color-GIST descriptors, Ohta color-GIST wavelet descriptors, enhanced Ohta color histogram descriptors, and SCGWDs), in terms of experimental perspectives. The proposed enhanced Ohta color histogram descriptors, Ohta color-GIST descriptors, Ohta color-GIST wavelet descriptors, SCGWD, and state-of-the-art visual descriptors were evaluated, using the Indian Institute of Technology Madras Scene Classification Image Database two, an Indoor-Outdoor Dataset, and the Massachusetts Institute of Technology indoor scene classification dataset [(MIT)-67]. Experimental results showed that the indoor versus outdoor scene recognition algorithm, employing SVM with SCGWDs, produced the highest classification rates (CRs)—95.48% and 99.82% using radial basis function kernel (RBF) kernel and 95.29% and 99.45% using linear kernel for the IITM SCID2 and Indoor-Outdoor datasets, respectively. The lowest CRs—2.08% and 4.92%, respectively—were obtained when RBF and linear kernels were used with the MIT-67 dataset. In addition, higher CRs, precision, recall, and area under the receiver operating characteristic curve values were obtained for the proposed SCGWDs, in comparison with state-of-the-art visual descriptors.

Spectral-Spatial Classification of Hyperspectral Image Based on Support Vector Machine

2021 ◽  
Vol 16 (1) ◽  
pp. 56-74
Author(s):  
Weiwei Yang ◽  
Haifeng Song
Keyword(s):  
Support Vector Machine ◽  
Classification Accuracy ◽  
Spatial Information ◽  
Hyperspectral Image ◽  
State Of The Art ◽  
Support Vector ◽  
Svm Classifier ◽  
Spatial Classification ◽  
Homogeneous Regions

Recent research has shown that integration of spatial information has emerged as a powerful tool in improving the classification accuracy of hyperspectral image (HSI). However, partitioning homogeneous regions of the HSI remains a challenging task. This paper proposes a novel spectral-spatial classification method inspired by the support vector machine (SVM). The model consists of spectral-spatial feature extraction channel (SSC) and SVM classifier. SSC is mainly used to extract spatial-spectral features of HSI. SVM is mainly used to classify the extracted features. The model can automatically extract the features of HSI and classify them. Experiments are conducted on benchmark HSI dataset (Indian Pines). It is found that the proposed method yields more accurate classification results compared to the state-of-the-art techniques.

Driver cognitive distraction detection: Feature estimation and implementation

2007 ◽  
Vol 221 (9) ◽  
pp. 1027-1040 ◽  
Author(s):  
M H Kutila ◽  
M Jokela ◽  
T Mäkinen ◽  
J Viitanen ◽  
G Markkula ◽  
...  
Keyword(s):  
Time Window ◽  
Cognitive Task ◽  
Parameter Tuning ◽  
Cognitive Tasks ◽  
Test Material ◽  
Support Vector ◽  
Svm Classifier ◽  
Practical Implementation ◽  
Cognitive Distraction ◽  
Feature Estimation

This article focuses on monitoring a driver's cognitive impairment due to talking to passengers or on a mobile phone, daydreaming, or just thinking about other than driving-related matters. This paper describes an investigation of cognitive distraction, firstly, giving an overall idea of its effects on the driver and, secondly, discussing the practical implementation of an algorithm for detection of cognitive distraction using a support vector machine (SVM) classifier. The evaluation data have been gathered by recruiting 12 professional drivers to drive for approximately 45 min in various environments and inducing cognitive tasks, i.e. arithmetic calculations. According to the prior knowledge and the experimental analysis, gaze, head and lane-keeping variances over a 15 s time window were selected indicative features. The SVM classifier's performance was optimized through exhaustive parameter tuning. The executed tests show that the cognitive workload can be detected with approximately 65-80 per cent confidence despite the fact that the test material represented medium-difficulty cognitive tasks (i.e. the induced workload was not very high). Thus, it could be assumed that a more challenging cognitive task would yield better detection results.

scholarly journals Algorithmic-Level Approximate Tensorial SVM Using High-Level Synthesis on FPGA

Electronics ◽  
2021 ◽  
Vol 10 (2) ◽  
pp. 205
Author(s):  
Hamoud Younes ◽  
Ali Ibrahim ◽  
Mostafa Rizk ◽  
Maurizio Valle
Keyword(s):  
Machine Learning ◽  
State Of The Art ◽  
Machine Learning Algorithms ◽  
High Level Synthesis ◽  
Support Vector ◽  
Svm Classifier ◽  
Approximate Computing ◽  
Classification Framework ◽  
Time Latency ◽  
High Level

Approximate Computing Techniques (ACT) are promising solutions towards the achievement of reduced energy, time latency and hardware size for embedded implementations of machine learning algorithms. In this paper, we present the first FPGA implementation of an approximate tensorial Support Vector Machine (SVM) classifier with algorithmic level ACTs using High-Level Synthesis (HLS). A touch modality classification framework was adopted to validate the effectiveness of the proposed implementation. When compared to exact implementation presented in the state-of-the-art, the proposed implementation achieves a reduction in power consumption by up to 49% with a speedup of 3.2×. Moreover, the hardware resources are reduced by 40% while consuming 82% less energy in classifying an input touch with an accuracy loss less than 5%.

scholarly journals An Empirical Study of Different Approaches for Protein Classification

2014 ◽  
Vol 2014 ◽  
pp. 1-17 ◽  
Author(s):  
Loris Nanni ◽  
Alessandra Lumini ◽  
Sheryl Brahnam
Keyword(s):  
Tertiary Structure ◽  
State Of The Art ◽  
Support Vector ◽  
Protein Classification ◽  
Matrix Representations ◽  
New Methods ◽  
Multiple Datasets ◽  
Different Types ◽  
Scoring Matrix ◽  
Better Than

Many domains would benefit from reliable and efficient systems for automatic protein classification. An area of particular interest in recent studies on automatic protein classification is the exploration of new methods for extracting features from a protein that work well for specific problems. These methods, however, are not generalizable and have proven useful in only a few domains. Our goal is to evaluate several feature extraction approaches for representing proteins by testing them across multiple datasets. Different types of protein representations are evaluated: those starting from the position specific scoring matrix of the proteins (PSSM), those derived from the amino-acid sequence, two matrix representations, and features taken from the 3D tertiary structure of the protein. We also test new variants of proteins descriptors. We develop our system experimentally by comparing and combining different descriptors taken from the protein representations. Each descriptor is used to train a separate support vector machine (SVM), and the results are combined by sum rule. Some stand-alone descriptors work well on some datasets but not on others. Through fusion, the different descriptors provide a performance that works well across all tested datasets, in some cases performing better than the state-of-the-art.

scholarly journals Feature Descriptor for Crowd Density Estimation

2019 ◽  
Author(s):  
Muhammad Bilal ◽  
Adwan Alanazi
Keyword(s):  
Density Estimation ◽  
State Of The Art ◽  
Texture Feature ◽  
Support Vector ◽  
Svm Classifier ◽  
Feature Descriptor ◽  
Crowd Density Estimation ◽  
Crowd Density ◽  
Feature Based ◽  
Crowd Monitoring

Crowd density estimation is an important task for crowd monitoring. Many efforts have been done to automate the process of estimating crowd density from images and videos. Despite series of efforts, it remains a challenging task. In this paper, we proposes a new texture feature-based approach for the estimation of crowd density based on Completed Local Binary Pattern (CLBP). We first divide the image into blocks and then re-divide the blocks into cells. For each cell, we compute CLBP and then concatenate them to describe the texture of the corresponding block. We then train a multi-class Support Vector Machine (SVM) classifier, which classifies each block of image into one of four categories, i.e. Very Low, Low, Medium, and High. We evaluate our technique on the PETS 2009 dataset, and from the experiments, we show to achieve 95% accuracy for the proposed descriptor.  We also compare other state-of-the-art texture descriptors and from the experimental results, we show that our proposed method outperforms other state-of-the-art methods.

scholarly journals Ensemble of convolutional neural networks for bioimage classification

2020 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Loris Nannia ◽  
Stefano Ghidoni ◽  
Sheryl Brahnam
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Color Images ◽  
Discriminative Power ◽  
Sum Rule ◽  
Matlab Code ◽  
Link Type ◽  
Multiple Datasets ◽  
Learning Parameter

This work presents a system based on an ensemble of Convolutional Neural Networks (CNNs) and descriptors for bioimage classification that has been validated on different datasets of color images. The proposed system represents a very simple yet effective way of boosting the performance of trained CNNs by composing multiple CNNs into an ensemble and combining scores by sum rule. Several types of ensembles are considered, with different CNN topologies along with different learning parameter sets. The proposed system not only exhibits strong discriminative power but also generalizes well over multiple datasets thanks to the combination of multiple descriptors based on different feature types, both learned and handcrafted. Separate classifiers are trained for each descriptor, and the entire set of classifiers is combined by sum rule. Results show that the proposed system obtains state-of-the-art performance across four different bioimage and medical datasets. The MATLAB code of the descriptors will be available at https://github.com/LorisNanni.

Synthesizing Conjunctive & Disjunctive Linear Invariants by K-means++ and SVM

2020 ◽  
Vol 17 (6) ◽  
pp. 847-856
Author(s):  
Shengbing Ren ◽  
Xiang Zhang
Keyword(s):  
Software Verification ◽  
State Of The Art ◽  
Positive Sample ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Hoare Logic ◽  
Excellent Performance ◽  
Automated Software ◽  
Inductive Invariants ◽  
Linear Invariants

The problem of synthesizing adequate inductive invariants lies at the heart of automated software verification. The state-of-the-art machine learning algorithms for synthesizing invariants have gradually shown its excellent performance. However, synthesizing disjunctive invariants is a difficult task. In this paper, we propose a method k++ Support Vector Machine (SVM) integrating k-means++ and SVM to synthesize conjunctive and disjunctive invariants. At first, given a program, we start with executing the program to collect program states. Next, k++SVM adopts k-means++ to cluster the positive samples and then applies SVM to distinguish each positive sample cluster from all negative samples to synthesize the candidate invariants. Finally, a set of theories founded on Hoare logic are adopted to check whether the candidate invariants are true invariants. If the candidate invariants fail the check, we should sample more states and repeat our algorithm. The experimental results show that k++SVM is compatible with the algorithms for Intersection Of Half-space (IOH) and more efficient than the tool of Interproc. Furthermore, it is shown that our method can synthesize conjunctive and disjunctive invariants automatically

Binary Spectrum Feature for Improved Classifier Performance

2020 ◽  
Author(s):  
Nalika Ulapane ◽  
Karthick Thiyagarajan ◽  
sarath kodagoda
Keyword(s):  
Machine Learning ◽  
Classification Performance ◽  
Feature Reduction ◽  
Sensor Data ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Svm Classifier ◽  
Monitoring Task ◽  
Classifier Performance ◽  
Spectrum Feature

<div>Classification has become a vital task in modern machine learning and Artificial Intelligence applications, including smart sensing. Numerous machine learning techniques are available to perform classification. Similarly, numerous practices, such as feature selection (i.e., selection of a subset of descriptor variables that optimally describe the output), are available to improve classifier performance. In this paper, we consider the case of a given supervised learning classification task that has to be performed making use of continuous-valued features. It is assumed that an optimal subset of features has already been selected. Therefore, no further feature reduction, or feature addition, is to be carried out. Then, we attempt to improve the classification performance by passing the given feature set through a transformation that produces a new feature set which we have named the “Binary Spectrum”. Via a case study example done on some Pulsed Eddy Current sensor data captured from an infrastructure monitoring task, we demonstrate how the classification accuracy of a Support Vector Machine (SVM) classifier increases through the use of this Binary Spectrum feature, indicating the feature transformation’s potential for broader usage.</div><div><br></div>

Sign in / Sign up

Export Citation Format

Share Document