Machine Learning Models and Algorithms for Big Data Classification

2016 ◽  
Author(s):  
Shan Suthaharan
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Classification ◽  
Learning Models ◽  
Big Data Classification ◽  
Machine Learning Models

scholarly journals Bi-LSTM Sentiment Classifier for Climate Change Issues in South Korea

2019 ◽  
Vol 8 (2S6) ◽  
pp. 295-299
Keyword(s):  
Climate Change ◽  
Machine Learning ◽  
Big Data ◽  
South Korea ◽  
Sentiment Analysis ◽  
Training Data ◽  
Learning Models ◽  
Wide Range ◽  
Machine Learning Models ◽  
Big Data Technology

A sentiment analysis using SNS data can confirm various people’s thoughts. Thus an analysis using SNS can predict social problems and more accurately identify the complex causes of the problem. In addition, big data technology can identify SNS information that is generated in real time, allowing a wide range of people’s opinions to be understood without losing time. It can supplement traditional opinion surveys. The incumbent government mainly uses SNS to promote its policies. However, measures are needed to actively reflect SNS in the process of carrying out the policy. Therefore this paper developed a sentiment classifier that can identify public feelings on SNS about climate change. To that end, based on a dictionary formulated on the theme of climate change, we collected climate change SNS data for learning and tagged seven sentiments. Using training data, the sentiment classifier models were developed using machine learning models. The analysis showed that the Bi-LSTM model had the best performance than shallow models. It showed the highest accuracy (85.10%) in the seven sentiments classified, outperforming traditional machine learning (Naive Bayes and SVM) by approximately 34.53%p, and 7.14%p respectively. These findings substantiate the applicability of the proposed Bi-LSTM-based sentiment classifier to the analysis of sentiments relevant to diverse climate change issues.

scholarly journals Using Big Data-machine learning models for diabetes prediction and flight delays analytics

2020 ◽  
Vol 7 (1) ◽  
Author(s):  
Thérence Nibareke ◽  
Jalal Laassiri
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Linear Regression ◽  
Decision Tree ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Smart Devices ◽  
Learning Models ◽  
Flight Delays ◽  
Machine Learning Models

Abstract Introduction Nowadays large data volumes are daily generated at a high rate. Data from health system, social network, financial, government, marketing, bank transactions as well as the censors and smart devices are increasing. The tools and models have to be optimized. In this paper we applied and compared Machine Learning algorithms (Linear Regression, Naïve bayes, Decision Tree) to predict diabetes. Further more, we performed analytics on flight delays. The main contribution of this paper is to give an overview of Big Data tools and machine learning models. We highlight some metrics that allow us to choose a more accurate model. We predict diabetes disease using three machine learning models and then compared their performance. Further more we analyzed flight delay and produced a dashboard which can help managers of flight companies to have a 360° view of their flights and take strategic decisions. Case description We applied three Machine Learning algorithms for predicting diabetes and we compared the performance to see what model give the best results. We performed analytics on flights datasets to help decision making and predict flight delays. Discussion and evaluation The experiment shows that the Linear Regression, Naive Bayesian and Decision Tree give the same accuracy (0.766) but Decision Tree outperforms the two other models with the greatest score (1) and the smallest error (0). For the flight delays analytics, the model could show for example the airport that recorded the most flight delays. Conclusions Several tools and machine learning models to deal with big data analytics have been discussed in this paper. We concluded that for the same datasets, we have to carefully choose the model to use in prediction. In our future works, we will test different models in other fields (climate, banking, insurance.).

DATA QUALITY CONSIDERATIONS FOR PETROPHYSICAL MACHINE LEARNING MODELS

2021 ◽  
Author(s):  
Andrew McDonald ◽  
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Quality ◽  
Input Data ◽  
Well Log ◽  
Learning Models ◽  
Log Data ◽  
Quality Issues ◽  
Machine Learning Models

Decades of subsurface exploration and characterisation have led to the collation and storage of large volumes of well related data. The amount of data gathered daily continues to grow rapidly as technology and recording methods improve. With the increasing adoption of machine learning techniques in the subsurface domain, it is essential that the quality of the input data is carefully considered when working with these tools. If the input data is of poor quality, the impact on precision and accuracy of the prediction can be significant. Consequently, this can impact key decisions about the future of a well or a field. This study focuses on well log data, which can be highly multi-dimensional, diverse and stored in a variety of file formats. Well log data exhibits key characteristics of Big Data: Volume, Variety, Velocity, Veracity and Value. Well data can include numeric values, text values, waveform data, image arrays, maps, volumes, etc. All of which can be indexed by time or depth in a regular or irregular way. A significant portion of time can be spent gathering data and quality checking it prior to carrying out petrophysical interpretations and applying machine learning models. Well log data can be affected by numerous issues causing a degradation in data quality. These include missing data - ranging from single data points to entire curves; noisy data from tool related issues; borehole washout; processing issues; incorrect environmental corrections; and mislabelled data. Having vast quantities of data does not mean it can all be passed into a machine learning algorithm with the expectation that the resultant prediction is fit for purpose. It is essential that the most important and relevant data is passed into the model through appropriate feature selection techniques. Not only does this improve the quality of the prediction, it also reduces computational time and can provide a better understanding of how the models reach their conclusion. This paper reviews data quality issues typically faced by petrophysicists when working with well log data and deploying machine learning models. First, an overview of machine learning and Big Data is covered in relation to petrophysical applications. Secondly, data quality issues commonly faced with well log data are discussed. Thirdly, methods are suggested on how to deal with data issues prior to modelling. Finally, multiple case studies are discussed covering the impacts of data quality on predictive capability.

scholarly journals Machine Learning Models and Big Data Tools for Evaluating Kidney Acceptance

2021 ◽  
Vol 185 ◽  
pp. 177-184
Author(s):  
Lirim Ashiku ◽  
Md. Al-Amin ◽  
Sanjay Madria ◽  
Cihan Dagli
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Learning Models ◽  
Machine Learning Models

scholarly journals Norm-Based Binary Search Trees for Speeding Up KNN Big Data Classification

Computers ◽  
2018 ◽  
Vol 7 (4) ◽  
pp. 54 ◽  
Author(s):  
Ahmad Hassanat
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Classification ◽  
Search Tree ◽  
Binary Search ◽  
Binary Search Tree ◽  
K Nearest Neighbors ◽  
Knn Classifier ◽  
Big Data Classification ◽  
Training Examples

Due to their large sizes and/or dimensions, the classification of Big Data is a challenging task using traditional machine learning, particularly if it is carried out using the well-known K-nearest neighbors classifier (KNN) classifier, which is a slow and lazy classifier by its nature. In this paper, we propose a new approach to Big Data classification using the KNN classifier, which is based on inserting the training examples into a binary search tree to be used later for speeding up the searching process for test examples. For this purpose, we used two methods to sort the training examples. The first calculates the minimum/maximum scaled norm and rounds it to 0 or 1 for each example. Examples with 0-norms are sorted in the left-child of a node, and those with 1-norms are sorted in the right child of the same node; this process continues recursively until we obtain one example or a small number of examples with the same norm in a leaf node. The second proposed method inserts each example into the binary search tree based on its similarity to the examples of the minimum and maximum Euclidean norms. The experimental results of classifying several machine learning big datasets show that both methods are much faster than most of the state-of-the-art methods compared, with competing accuracy rates obtained by the second method, which shows great potential for further enhancements of both methods to be used in practice.

Edge Learning for Distributed Big Data Analytics

2022 ◽  
Author(s):  
Song Guo ◽  
Zhihao Qu
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
System Design ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Security And Privacy ◽  
Learning Performance ◽  
Learning Models ◽  
Key Technologies ◽  
Machine Learning Models

Discover this multi-disciplinary and insightful work, which integrates machine learning, edge computing, and big data. Presents the basics of training machine learning models, key challenges and issues, as well as comprehensive techniques including edge learning algorithms, and system design issues. Describes architectures, frameworks, and key technologies for learning performance, security, and privacy, as well as incentive issues in training/inference at the network edge. Intended to stimulate fruitful discussions, inspire further research ideas, and inform readers from both academia and industry backgrounds. Essential reading for experienced researchers and developers, or for those who are just entering the field.

Sign in / Sign up

Export Citation Format

Share Document