scholarly journals DATA PREPARATION ON LARGE DATASETS FOR DATA SCIENCE

2017 ◽  
Vol 10 (13) ◽  
pp. 485
Author(s):  
Darshan Barapatre ◽  
Vijayalakshmi A
Keyword(s):  
Machine Learning ◽  
Data Science ◽  
Data Cleaning ◽  
Large Datasets ◽  
Machine Learning Algorithms ◽  
Unstructured Data ◽  
Data Preparation ◽  
Data Profiling ◽  
Data Scientist ◽  
Preparation Techniques

 According to interviews and experts, data scientists spend 50-80% of the valuable time in the mundane task of collecting and preparing structured or unstructured data, before it can be explored for useful analysis. It is very valuable for a data scientist to restructure and refine the data into more meaningful datasets, which can be used further for analytics. Hence, the idea is to build a tool which will contain all the required data preparation techniques to make data well-structured by providing greater flexibility and easy to use UI. Tool will contain different data preparation techniques which will include the process of data cleaning, data structuring, transforming data, data compression, and data profiling and implementation of related machine learning algorithms.

Heat Disease Prediction using Machine Learning

2021 ◽  
Vol 9 (VI) ◽  
pp. 846-852
Author(s):  
Prof. Dr. R. Sandhiya
Keyword(s):  
Machine Learning ◽  
Heart Disease ◽  
Data Science ◽  
Machine Learning Algorithms ◽  
Assessment Process ◽  
Disease Prediction ◽  
Test Results ◽  
Medical Field ◽  
Modern Age ◽  
Linear Svm

In recent times, the diagnosis of heart disease has become a very critical task in the medical field. In the modern age, one person dies every minute due to heart disease. Data science has an important role in processing big amounts of data in the field of health sciences. Since the diagnosis of heart disease is a complex task, the assessment process should be automated to avoid the risks associated with it and alert the patient in advance. This paper uses the heart disease dataset available in the UCI Machine Learning Repository. The proposed work assesses the risk of heart disease in a patient by applying various data mining methods such as Naive Bayes, Decision Tree, KNN, Linear SVM, RBF SVM, Gaussian Process, Neural Network, Adabost, QDA and Random Forest. This paper provides a comparative study by analyzing the performance of various machine learning algorithms. Test results confirm that the KNN algorithm achieved the highest 97% accuracy compared to other implemented ML algorithms.

Machine Learning Techniques for Internet of Things

2019 ◽  
pp. 160-180
Author(s):  
P. Priakanth ◽  
S. Gopikrishnan
Keyword(s):  
Machine Learning ◽  
Data Science ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Independent Learning ◽  
Machine Learning Techniques ◽  
Analytical Models ◽  
Guided Learning ◽  
Learning Techniques ◽  
Learning Machine

The idea of an intelligent, independent learning machine has fascinated humans for decades. The philosophy behind machine learning is to automate the creation of analytical models in order to enable algorithms to learn continuously with the help of available data. Since IoT will be among the major sources of new data, data science will make a great contribution to make IoT applications more intelligent. Machine learning can be applied in cases where the desired outcome is known (guided learning) or the data is not known beforehand (unguided learning) or the learning is the result of interaction between a model and the environment (reinforcement learning). This chapter answers the questions: How could machine learning algorithms be applied to IoT smart data? What is the taxonomy of machine learning algorithms that can be adopted in IoT? And what are IoT data characteristics in real-world which requires data analytics?

A Literature Review on Thyroid Hormonal Problems in Women Using Data Science and Analytics

2021 ◽  
pp. 416-428
Author(s):  
R. Suganya ◽  
Rajaram S. ◽  
Kameswari M.
Keyword(s):  
Machine Learning ◽  
Literature Review ◽  
Data Science ◽  
Learning Algorithms ◽  
Research Literature ◽  
Machine Learning Algorithms ◽  
Thyroid Disorder ◽  
Classification Models ◽  
Indian Women ◽  
Using Data

Currently, thyroid disorders are more common and widespread among women worldwide. In India, seven out of ten women are suffering from thyroid problems. Various research literature studies predict that about 35% of Indian women are examined with prevalent goiter. It is very necessary to take preventive measures at its early stages, otherwise it causes infertility problem among women. The recent review discusses various analytics models that are used to handle different types of thyroid problems in women. This chapter is planned to analyze and compare different classification models, both machine learning algorithms and deep leaning algorithms, to classify different thyroid problems. Literature from both machine learning and deep learning algorithms is considered. This literature review on thyroid problems will help to analyze the reason and characteristics of thyroid disorder. The dataset used to build and to validate the algorithms was provided by UCI machine learning repository.

Machine Learning Techniques for Internet of Things

2021 ◽  
pp. 1490-1506
Author(s):  
P. Priakanth ◽  
S. Gopikrishnan
Keyword(s):  
Machine Learning ◽  
Data Science ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Independent Learning ◽  
Machine Learning Techniques ◽  
Analytical Models ◽  
Guided Learning ◽  
Learning Techniques ◽  
Learning Machine

The idea of an intelligent, independent learning machine has fascinated humans for decades. The philosophy behind machine learning is to automate the creation of analytical models in order to enable algorithms to learn continuously with the help of available data. Since IoT will be among the major sources of new data, data science will make a great contribution to make IoT applications more intelligent. Machine learning can be applied in cases where the desired outcome is known (guided learning) or the data is not known beforehand (unguided learning) or the learning is the result of interaction between a model and the environment (reinforcement learning). This chapter answers the questions: How could machine learning algorithms be applied to IoT smart data? What is the taxonomy of machine learning algorithms that can be adopted in IoT? And what are IoT data characteristics in real-world which requires data analytics?

Bio-informatics and psychiatric epidemiology

2020 ◽  
pp. 359-372
Author(s):  
Nicola Voyle ◽  
Maximilian Kerz ◽  
Steven Kiddle ◽  
Richard Dobson
Keyword(s):  
Machine Learning ◽  
Data Cleaning ◽  
Epidemiological Studies ◽  
Selection Method ◽  
Psychiatric Epidemiology ◽  
Large Datasets ◽  
Data Exploration ◽  
Feature Identification ◽  
Data Formats ◽  
Method Selection

This chapter highlights the methodologies which are increasingly being applied to large datasets or ‘big data’, with an emphasis on bio-informatics. The first stage of any analysis is to collect data from a well-designed study. The chapter begins by looking at the raw data that arises from epidemiological studies and highlighting the first stages in creating clean data that can be used to draw informative conclusions through analysis. The remainder of the chapter covers data formats, data exploration, data cleaning, missing data (i.e. the lack of data for a variable in an observation), reproducibility, classification versus regression, feature identification and selection, method selection (e.g. supervised versus unsupervised machine learning), training a classifier, and drawing conclusions from modelling.

Sign in / Sign up

Export Citation Format

Share Document