A novel feature selection method for large-scale data sets1

2005 ◽  
Vol 9 (3) ◽  
pp. 237-251 ◽  
Author(s):  
Wei-Chou Chen ◽  
Ming-Chun Yang ◽  
Shian-Shyong Tseng
Keyword(s):  
Feature Selection ◽  
Large Scale ◽  
Feature Selection Method ◽  
Selection Method ◽  
Large Scale Data ◽  
Scale Data

scholarly journals IoT Botnet Attack Detection Based on Optimized Extreme Gradient Boosting and Feature Selection

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6336 ◽  
Author(s):  
Mnahi Alqahtani ◽  
Hassan Mathkour ◽  
Mohamed Maher Ben Ismail
Keyword(s):  
Feature Selection ◽  
Large Scale ◽  
Feature Selection Method ◽  
Selection Method ◽  
Attack Detection ◽  
Gradient Boosting ◽  
Fisher Score ◽  
Detection Approach ◽  
Extreme Gradient Boosting ◽  
Iot Devices

Nowadays, Internet of Things (IoT) technology has various network applications and has attracted the interest of many research and industrial communities. Particularly, the number of vulnerable or unprotected IoT devices has drastically increased, along with the amount of suspicious activity, such as IoT botnet and large-scale cyber-attacks. In order to address this security issue, researchers have deployed machine and deep learning methods to detect attacks targeting compromised IoT devices. Despite these efforts, developing an efficient and effective attack detection approach for resource-constrained IoT devices remains a challenging task for the security research community. In this paper, we propose an efficient and effective IoT botnet attack detection approach. The proposed approach relies on a Fisher-score-based feature selection method along with a genetic-based extreme gradient boosting (GXGBoost) model in order to determine the most relevant features and to detect IoT botnet attacks. The Fisher score is a representative filter-based feature selection method used to determine significant features and discard irrelevant features through the minimization of intra-class distance and the maximization of inter-class distance. On the other hand, GXGBoost is an optimal and effective model, used to classify the IoT botnet attacks. Several experiments were conducted on a public botnet dataset of IoT devices. The evaluation results obtained using holdout and 10-fold cross-validation techniques showed that the proposed approach had a high detection rate using only three out of the 115 data traffic features and improved the overall performance of the IoT botnet attack detection process.

E-Commerce data classification in the cloud environment based on bayesian algorithm

2020 ◽  
pp. 1-8
Author(s):  
Bing Xu
Keyword(s):  
Large Scale ◽  
High Efficiency ◽  
Feature Selection Method ◽  
Data Classification ◽  
Classification Algorithm ◽  
Testing Time ◽  
Bayesian Algorithm ◽  
Large Scale Data ◽  
Distributed Platform ◽  
Scale Data

In the process of e-commerce transactions, a large amount of data will be generated, whose effective classification is one of current research hotspots. An improved feature selection method was proposed based on the characteristics of Bayesian classification algorithm. Due to the long training and testing time of modern large-scale data classification on a single computer, a data classification algorithm based on Naive Bayes was designed and implemented on the Hadoop distributed platform. The experimental results showed that the improved algorithm could effectively improve the accuracy of classification, and the designed parallel Bayesian data classification algorithm had high efficiency, which was suitable for the processing and analysis of massive data.

scholarly journals An ensemble-based feature selection framework to select risk factors of childhood obesity for policy decision making

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Xi Shi ◽  
Gorana Nikolic ◽  
Gorka Epelde ◽  
Mónica Arrúe ◽  
Joseba Bidaurrazaga Van-Dierdonck ◽  
...  
Keyword(s):  
Risk Factors ◽  
Feature Selection ◽  
Childhood Obesity ◽  
Random Forest ◽  
Clinical Relevance ◽  
Large Scale ◽  
Large Scale Data ◽  
Selection Framework ◽  
Voting Strategy ◽  
Scale Data

Abstract Background The increasing prevalence of childhood obesity makes it essential to study the risk factors with a sample representative of the population covering more health topics for better preventive policies and interventions. It is aimed to develop an ensemble feature selection framework for large-scale data to identify risk factors of childhood obesity with good interpretability and clinical relevance. Methods We analyzed the data collected from 426,813 children under 18 during 2000–2019. A BMI above the 90th percentile for the children of the same age and gender was defined as overweight. An ensemble feature selection framework, Bagging-based Feature Selection framework integrating MapReduce (BFSMR), was proposed to identify risk factors. The framework comprises 5 models (filter with mutual information/SVM-RFE/Lasso/Ridge/Random Forest) from filter, wrapper, and embedded feature selection methods. Each feature selection model identified 10 variables based on variable importance. Considering accuracy, F-score, and model characteristics, the models were classified into 3 levels with different weights: Lasso/Ridge, Filter/SVM-RFE, and Random Forest. The voting strategy was applied to aggregate the selected features, with both feature weights and model weights taken into consideration. We compared our voting strategy with another two for selecting top-ranked features in terms of 6 dimensions of interpretability. Results Our method performed the best to select the features with good interpretability and clinical relevance. The top 10 features selected by BFSMR are age, sex, birth year, breastfeeding type, smoking habit and diet-related knowledge of both children and mothers, exercise, and Mother’s systolic blood pressure. Conclusion Our framework provides a solution for identifying a diverse and interpretable feature set without model bias from large-scale data, which can help identify risk factors of childhood obesity and potentially some other diseases for future interventions or policies.

scholarly journals Identifying the Signatures and Rules of Circulating Extracellular MicroRNA for Distinguishing Cancer Subtypes

2021 ◽  
Vol 12 ◽  
Author(s):  
Fei Yuan ◽  
Zhandong Li ◽  
Lei Chen ◽  
Tao Zeng ◽  
Yu-Hang Zhang ◽  
...  
Keyword(s):  
Feature Selection ◽  
Cancer Diagnosis ◽  
Large Scale ◽  
Feature Selection Method ◽  
Selection Method ◽  
Quantitative Classification ◽  
Cancer Subtypes ◽  
Clinical Cancer ◽  
Cancer Types ◽  
Functional Analyses

Cancer is one of the most threatening diseases to humans. It can invade multiple significant organs, including lung, liver, stomach, pancreas, and even brain. The identification of cancer biomarkers is one of the most significant components of cancer studies as the foundation of clinical cancer diagnosis and related drug development. During the large-scale screening for cancer prevention and early diagnosis, obtaining cancer-related tissues is impossible. Thus, the identification of cancer-associated circulating biomarkers from liquid biopsy targeting has been proposed and has become the most important direction for research on clinical cancer diagnosis. Here, we analyzed pan-cancer extracellular microRNA profiles by using multiple machine-learning models. The extracellular microRNA profiles on 11 cancer types and non-cancer were first analyzed by Boruta to extract important microRNAs. Selected microRNAs were then evaluated by the Max-Relevance and Min-Redundancy feature selection method, resulting in a feature list, which were fed into the incremental feature selection method to identify candidate circulating extracellular microRNA for cancer recognition and classification. A series of quantitative classification rules was also established for such cancer classification, thereby providing a solid research foundation for further biomarker exploration and functional analyses of tumorigenesis at the level of circulating extracellular microRNA.

Sign in / Sign up

Export Citation Format

Share Document