scholarly journals Machine Learning in Apache Spark Environment for Diagnosis of Diabetes

2021 ◽  
Author(s):  
Farshid Bagheri Saravi ◽  
Shadi Moghanian ◽  
Giti Javidi ◽  
Ehsan O Sheybani
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Support Vector Machine ◽  
Big Data ◽  
Random Forest ◽  
Apache Spark ◽  
Support Vector ◽  
Computing Environment ◽  
Large Dataset ◽  
Related Data

Disease-related data and information collected by physicians, patients, and researchers seem insignificant at first glance. Still, the same unorganized data contain valuable information that is often hidden. The task of data mining techniques is to extract patterns to classify the data accurately. One of the various Data mining and its methods have been used often to diagnose various diseases. In this study, a machine learning (ML) technique based on distributed computing in the Apache Spark computing space is used to diagnose diabetics or hidden pattern of the illness to detect the disease using a large dataset in real-time. Implementation results of three ML techniques of Decision Tree (DT) technique or Random Forest (RF) or Support Vector Machine (SVM) in the Apache Spark computing environment using the Scala programming language and WEKA show that RF is more efficient and faster to diagnose diabetes in big data.

scholarly journals Klasifikasi Pengembalian Radar dari Ionosfer Menggunakan SVM, Naïve Bayes dan Random Forest

2021 ◽  
Vol 10 (2) ◽  
pp. 111-117
Author(s):  
Yulia Aryani ◽  
Arie Wahyu Wijayanto
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Support Vector Machine ◽  
Random Forest ◽  
Support Vector ◽  
Ve Bayes

ABSTRAK – Klasifikasi merupakan salah satu topik utama dalam data mining atau machine learning. Klasifikasi adalah suatu pengelompokan data dimana data yang digunakan tersebut mempunyai kelas label atau target. Klasifikasi digunakan untuk mengambil data dan ditempatkan kedalam kelompok tertentu.  Studi tentang ionosfer penting untuk penelitian di berbagai domain, khususnya dalam sistem komunikasi.  Dalam penelitian ionosfer, perlu dilakukan klasifikasi radar yang berguna dan tidak berguna dari ionosfer. Pada makalah ini, akan dilakukan klasifikasi  terhadap data inosphere yang diambil dari UCI machine learning repository.  Klasifikasi dilakukan dengan menggunakan tiga metode klasifikasi, yakni  SVM ( Support Vector Machine ) , Naïve Bayes, dan Random Forest. Hasil dari percobaan ini bisa menunjukkan prediksi dari setiap percobaan dengan tingkat akurasi dan prediksi yang berbeda-beda di setiap metode yang digunakan. Hasil akurasi, presisi, dan recall terbaik didapatkan pada metode Random Forest dengan rasio data latih dan data uji sebesar 85% didapat akurasi dari data uji sebesar 90,57% dengan presisi sebesar 94,12%. Kata Kunci – Ionosfer; Klasifikasi; SVM; Naïve Bayes; Random Forest.

scholarly journals Big Data Mining and Classification of Intelligent Material Science Data Using Machine Learning

2021 ◽  
Vol 11 (18) ◽  
pp. 8596
Author(s):  
Swetha Chittam ◽  
Balakrishna Gokaraju ◽  
Zhigang Xu ◽  
Jagannathan Sankar ◽  
Kaushik Roy
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Logistic Regression ◽  
Big Data ◽  
Random Forest ◽  
Material Science ◽  
Classification Model ◽  
Support Vector ◽  
Material Strength ◽  
Science Data

There is a high need for a big data repository for material compositions and their derived analytics of metal strength, in the material science community. Currently, many researchers maintain their own excel sheets, prepared manually by their team by tabulating the experimental data collected from scientific journals, and analyzing the data by performing manual calculations using formulas to determine the strength of the material. In this study, we propose a big data storage for material science data and its processing parameters information to address the laborious process of data tabulation from scientific articles, data mining techniques to retrieve the information from databases to perform big data analytics, and a machine learning prediction model to determine material strength insights. Three models are proposed based on Logistic regression, Support vector Machine SVM and Random Forest Algorithms. These models are trained and tested using a 10-fold cross validation approach. The Random Forest classification model performed better on the independent dataset, with 87% accuracy in comparison to Logistic regression and SVM with 72% and 78%, respectively.

scholarly journals Research on Parallel Support Vector Machine Based on Spark Big Data Platform

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Yao Huimin
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Big Data ◽  
Support Vector Machines ◽  
Cross Validation ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Lambda Architecture ◽  
Vector Machines ◽  
Data Platform

With the development of cloud computing and distributed cluster technology, the concept of big data has been expanded and extended in terms of capacity and value, and machine learning technology has also received unprecedented attention in recent years. Traditional machine learning algorithms cannot solve the problem of effective parallelization, so a parallelization support vector machine based on Spark big data platform is proposed. Firstly, the big data platform is designed with Lambda architecture, which is divided into three layers: Batch Layer, Serving Layer, and Speed Layer. Secondly, in order to improve the training efficiency of support vector machines on large-scale data, when merging two support vector machines, the “special points” other than support vectors are considered, that is, the points where the nonsupport vectors in one subset violate the training results of the other subset, and a cross-validation merging algorithm is proposed. Then, a parallelized support vector machine based on cross-validation is proposed, and the parallelization process of the support vector machine is realized on the Spark platform. Finally, experiments on different datasets verify the effectiveness and stability of the proposed method. Experimental results show that the proposed parallelized support vector machine has outstanding performance in speed-up ratio, training time, and prediction accuracy.

scholarly journals Integration of synthetic minority oversampling technique for imbalanced class

2019 ◽  
Vol 13 (1) ◽  
pp. 102
Author(s):  
Noviyanti Santoso ◽  
Wahyu Wibowo ◽  
Hilda Hikmawati
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Support Vector Machine ◽  
Class Imbalance ◽  
Original Data ◽  
Support Vector ◽  
Classification Methods ◽  
Problematic Issue ◽  
Imbalanced Class ◽  
F Measure

In the data mining, a class imbalance is a problematic issue to look for the solutions. It probably because machine learning is constructed by using algorithms with assuming the number of instances in each balanced class, so when using a class imbalance, it is possible that the prediction results are not appropriate. They are solutions offered to solve class imbalance issues, including oversampling, undersampling, and synthetic minority oversampling technique (SMOTE). Both oversampling and undersampling have its disadvantages, so SMOTE is an alternative to overcome it. By integrating SMOTE in the data mining classification method such as Naive Bayes, Support Vector Machine (SVM), and Random Forest (RF) is expected to improve the performance of accuracy. In this research, it was found that the data of SMOTE gave better accuracy than the original data. In addition to the three classification methods used, RF gives the highest average AUC, F-measure, and G-means score.

Classification study of solvation free energies of organic molecules using machine learning techniques

RSC Advances ◽  
2014 ◽  
Vol 4 (106) ◽  
pp. 61624-61630 ◽  
Author(s):  
N. S. Hari Narayana Moorthy ◽  
Silvia A. Martins ◽  
Sergio F. Sousa ◽  
Maria J. Ramos ◽  
Pedro A. Fernandes
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Random Forest ◽  
Organic Molecules ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Classification Models ◽  
Free Energies ◽  
Learning Techniques ◽  
Solvation Free Energies

Classification models to predict the solvation free energies of organic molecules were developed using decision tree, random forest and support vector machine approaches and with MACCS fingerprints, MOE and PaDEL descriptors.

scholarly journals Execution Assessment of Machine Learning Algorithms for Spam Profile Detection on Instagram

2021 ◽  
Vol 10 (3) ◽  
pp. 1889-1894
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Random Forest ◽  
Nearest Neighbor ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Learning Tools ◽  
Learning Models ◽  
K Nearest Neighbor

Witheverypassingsecondsocialnetworkcommunityisgrowingrapidly,becauseofthat,attackershaveshownkeeninterestinthesekindsofplatformsandwanttodistributemischievouscontentsontheseplatforms.Withthefocus on introducing new set of characteristics and features forcounteractivemeasures,agreatdealofstudieshasresearchedthe possibility of lessening the malicious activities on social medianetworks. This research was to highlight features for identifyingspammers on Instagram and additional features were presentedto improve the performance of different machine learning algorithms. Performance of different machine learning algorithmsnamely, Multilayer Perceptron (MLP), Random Forest (RF), K-Nearest Neighbor (KNN) and Support Vector Machine (SVM)were evaluated on machine learning tools named, RapidMinerand WEKA. The results from this research tells us that RandomForest (RF) outperformed all other selected machine learningalgorithmsonbothselectedmachinelearningtools.OverallRandom Forest (RF) provided best results on RapidMiner. Theseresultsareusefulfortheresearcherswhoarekeentobuildmachine learning models to find out the spamming activities onsocialnetworkcommunities.

scholarly journals Application of Natural Language Processing with Supervised Machine Learning Techniques to Predict the Overall Drugs Performance

2020 ◽  
Vol 11 (40) ◽  
pp. 8-23
Author(s):  
Pius MARTHIN ◽  
Duygu İÇEN
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Random Forest ◽  
Semantic Analysis ◽  
Classification Tree ◽  
Supervised Machine Learning ◽  
Training Dataset ◽  
Support Vector ◽  
Learning Models ◽  
Machine Learning Models

Online product reviews have become a valuable source of information which facilitate customer decision with respect to a particular product. With the wealthy information regarding user's satisfaction and experiences about a particular drug, pharmaceutical companies make the use of online drug reviews to improve the quality of their products. Machine learning has enabled scientists to train more efficient models which facilitate decision making in various fields. In this manuscript we applied a drug review dataset used by (Gräβer, Kallumadi, Malberg,& Zaunseder, 2018), available freely from machine learning repository website of the University of California Irvine (UCI) to identify best machine learning model which provide a better prediction of the overall drug performance with respect to users' reviews. Apart from several manipulations done to improve model accuracy, all necessary procedures required for text analysis were followed including text cleaning and transformation of texts to numeric format for easy training machine learning models. Prior to modeling, we obtained overall sentiment scores for the reviews. Customer's reviews were summarized and visualized using a bar plot and word cloud to explore the most frequent terms. Due to scalability issues, we were able to use only the sample of the dataset. We randomly sampled 15000 observations from the 161297 training dataset and 10000 observations were randomly sampled from the 53766 testing dataset. Several machine learning models were trained using 10 folds cross-validation performed under stratified random sampling. The trained models include Classification and Regression Trees (CART), classification tree by C5.0, logistic regression (GLM), Multivariate Adaptive Regression Spline (MARS), Support vector machine (SVM) with both radial and linear kernels and a classification tree using random forest (Random Forest). Model selection was done through a comparison of accuracies and computational efficiency. Support vector machine (SVM) with linear kernel was significantly best with an accuracy of 83% compared to the rest. Using only a small portion of the dataset, we managed to attain reasonable accuracy in our models by applying the TF-IDF transformation and Latent Semantic Analysis (LSA) technique to our TDM.

scholarly journals Teknik Resampling untuk Mengatasi Ketidakseimbangan Kelas pada Klasifikasi Penyakit Diabetes Menggunakan C4.5, Random Forest, dan SVM

Techno Com ◽  
2021 ◽  
Vol 20 (3) ◽  
pp. 352-361
Author(s):  
Wahyu Nugraha ◽  
Raja Sabaruddin
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Random Forest ◽  
Area Under Curve ◽  
Support Vector ◽  
Pima Indians ◽  
R Language ◽  
Level Data ◽  
Vector Machines ◽  
Under Sampling

Penderita diabetes di seluruh dunia terus mengalami peningkatan dengan angka kematian sebesar 4,6 juta pada tahun 2011 dan diperkirakan akan terus meningkat secara global menjadi 552 juta pada tahun 2030. Pencegahan Penyakit diabetes mungkin dapat dilakukan secara efektif dengan cara mendeteksinya sejak dini. Data mining dan machine learning terus dikembangkan agar menjadi alat yang handal dalam membangun model komputasi untuk mengidentifikasi penyakit diabetes pada tahap awal. Namun, masalah yang sering dihadapi dalam menganalisis penyakit diabetes ialah masalah ketidakseimbangan class. Kelas yang tidak seimbang membuat model pembelajaran akan sulit melakukan prediksi karena model pembelajaran didominasi oleh instance kelas mayoritas sehingga mengabaikan prediksi kelas minoritas. Pada penelitian ini kami mencoba menganalisa dan mencoba mengatasi masalah ketidakseimbangan kelas dengan menggunakan pendekatan level data yaitu teknik resampling data. Eksperimen ini menggunakan R language dengan library ROSE (version 0.0-4). Dataset Pima Indians dipilih pada penelitian ini karena merupakan salah satu dataset yang mengalami ketidakseimbangan kelas. Model pengklasifikasian pada penelitian ini menggunakan algoritma decision tree C4.5, RF (Random Forest), dan SVM (Support Vector Machines). Dari hasil eksperimen yang dilakukan model klasifikasi SVM dengan teknik resampling yang menggabungkan over dan under-sampling menjadi model yang memiliki performa terbaik dengan nilai AUC (Area Under Curve) sebesar 0.80

scholarly journals Improvement of Support Vector Machine Algorithm in Big Data Background

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Babacar Gaye ◽  
Dezheng Zhang ◽  
Aziguli Wulamu
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Big Data ◽  
Time Complexity ◽  
Dual Problem ◽  
Learning Algorithm ◽  
Rapid Development ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Original Space

With the rapid development of the Internet and the rapid development of big data analysis technology, data mining has played a positive role in promoting industry and academia. Classification is an important problem in data mining. This paper explores the background and theory of support vector machines (SVM) in data mining classification algorithms and analyzes and summarizes the research status of various improved methods of SVM. According to the scale and characteristics of the data, different solution spaces are selected, and the solution of the dual problem is transformed into the classification surface of the original space to improve the algorithm speed. Research Process. Incorporating fuzzy membership into multicore learning, it is found that the time complexity of the original problem is determined by the dimension, and the time complexity of the dual problem is determined by the quantity, and the dimension and quantity constitute the scale of the data, so it can be based on the scale of the data Features Choose different solution spaces. The algorithm speed can be improved by transforming the solution of the dual problem into the classification surface of the original space. Conclusion. By improving the calculation rate of traditional machine learning algorithms, it is concluded that the accuracy of the fitting prediction between the predicted data and the actual value is as high as 98%, which can make the traditional machine learning algorithm meet the requirements of the big data era. It can be widely used in the context of big data.

Sign in / Sign up

Export Citation Format

Share Document