scholarly journals Dealing with Randomness and Concept Drift in Large Datasets

Data ◽  
2021 ◽  
Vol 6 (7) ◽  
pp. 77
Author(s):  
Kassim S. Mwitondi ◽  
Raed A. Said
Keyword(s):  
Machine Learning ◽  
Sustainable Development ◽  
Data Science ◽  
Concept Drift ◽  
Real Life ◽  
Quality Education ◽  
Data Driven ◽  
Machine Learning Techniques ◽  
Address Data ◽  
The Impact

Data-driven solutions to societal challenges continue to bring new dimensions to our daily lives. For example, while good-quality education is a well-acknowledged foundation of sustainable development, innovation and creativity, variations in student attainment and general performance remain commonplace. Developing data -driven solutions hinges on two fronts-technical and application. The former relates to the modelling perspective, where two of the major challenges are the impact of data randomness and general variations in definitions, typically referred to as concept drift in machine learning. The latter relates to devising data-driven solutions to address real-life challenges such as identifying potential triggers of pedagogical performance, which aligns with the Sustainable Development Goal (SDG) #4-Quality Education. A total of 3145 pedagogical data points were obtained from the central data collection platform for the United Arab Emirates (UAE) Ministry of Education (MoE). Using simple data visualisation and machine learning techniques via a generic algorithm for sampling, measuring and assessing, the paper highlights research pathways for educationists and data scientists to attain unified goals in an interdisciplinary context. Its novelty derives from embedded capacity to address data randomness and concept drift by minimising modelling variations and yielding consistent results across samples. Results show that intricate relationships among data attributes describe the invariant conditions that practitioners in the two overlapping fields of data science and education must identify.

scholarly journals COVID-19 Prognostic Models: A Pro-con Debate for Machine Learning vs. Traditional Statistics

2021 ◽  
Vol 3 ◽  
Author(s):  
Ahmed Al-Hindawi ◽  
Ahmed Abdulaal ◽  
Timothy M. Rawson ◽  
Saleh A. Alqahtani ◽  
Nabeela Mughal ◽  
...  
Keyword(s):  
Machine Learning ◽  
Programming Languages ◽  
Data Science ◽  
The Other ◽  
Machine Learning Techniques ◽  
Prognostic Models ◽  
Learning Techniques ◽  
Classical Statistics ◽  
Open Datasets ◽  
The Impact

The SARS-CoV-2 virus, which causes the COVID-19 pandemic, has had an unprecedented impact on healthcare requiring multidisciplinary innovation and novel thinking to minimize impact and improve outcomes. Wide-ranging disciplines have collaborated including diverse clinicians (radiology, microbiology, and critical care), who are working increasingly closely with data-science. This has been leveraged through the democratization of data-science with the increasing availability of easy to access open datasets, tutorials, programming languages, and hardware which makes it significantly easier to create mathematical models. To address the COVID-19 pandemic, such data-science has enabled modeling of the impact of the virus on the population and individuals for diagnostic, prognostic, and epidemiological ends. This has led to two large systematic reviews on this topic that have highlighted the two different ways in which this feat has been attempted: one using classical statistics and the other using more novel machine learning techniques. In this review, we debate the relative strengths and weaknesses of each method toward the specific task of predicting COVID-19 outcomes.

scholarly journals Cybersecurity Data Science: An Overview from Machine Learning Perspective

2020 ◽  
Author(s):  
Iqbal H. Sarker
Keyword(s):  
Machine Learning ◽  
Data Science ◽  
Cyber Attacks ◽  
Data Driven ◽  
Machine Learning Techniques ◽  
Research Issues ◽  
Security Incident ◽  
Intelligent Decision Making ◽  
Learning Techniques ◽  
Intelligent Decision

In a computing context, cybersecurity is undergoing massive shifts in technology and its operations in recent days, and data science is driving the change. Extracting security incident patterns or insights from cybersecurity data and building corresponding data-driven model, is the key to make a security system automated and intelligent. To understand and analyze the actual phenomena with data, various scientific methods, machine learning techniques, processes, and systems are used, which is commonly known as data science. In this paper, we focus and briefly discuss cybersecurity data science, where the data is being gathered from relevant cybersecurity sources, and the analytics complement the latest data-driven patterns for providing more effective security solutions. The concept of cybersecurity data science allows making the computing process more actionable and intelligent as compared to traditional ones in the domain of cybersecurity. We then discuss and summarize a number of associated research issues and future directions. Furthermore, we provide a machine learning-based multi-layered framework for the purpose of cybersecurity modeling. Overall, our goal is not only to discuss cybersecurity data science and relevant methods but also to focus the applicability towards data-driven intelligent decision making for protecting the systems from cyber-attacks.

scholarly journals On predicting academic performance with process mining in learning analytics

2017 ◽  
Vol 10 (2) ◽  
pp. 160-176 ◽  
Author(s):  
Rahila Umer ◽  
Teo Susnjak ◽  
Anuradha Mathrani ◽  
Suriadi Suriadi
Keyword(s):  
Machine Learning ◽  
Process Mining ◽  
Learning Experience ◽  
Dropout Rate ◽  
Data Driven ◽  
Machine Learning Techniques ◽  
Content Type ◽  
Machine Learning Classification ◽  
Learning Techniques ◽  
The Impact

Purpose The purpose of this paper is to propose a process mining approach to help in making early predictions to improve students’ learning experience in massive open online courses (MOOCs). It investigates the impact of various machine learning techniques in combination with process mining features to measure effectiveness of these techniques. Design/methodology/approach Student’s data (e.g. assessment grades, demographic information) and weekly interaction data based on event logs (e.g. video lecture interaction, solution submission time, time spent weekly) have guided this design. This study evaluates four machine learning classification techniques used in the literature (logistic regression (LR), Naïve Bayes (NB), random forest (RF) and K-nearest neighbor) to monitor weekly progression of students’ performance and to predict their overall performance outcome. Two data sets – one, with traditional features and second, with features obtained from process conformance testing – have been used. Findings The results show that techniques used in the study are able to make predictions on the performance of students. Overall accuracy (F1-score, area under curve) of machine learning techniques can be improved by integrating process mining features with standard features. Specifically, the use of LR and NB classifiers outperforms other techniques in a statistical significant way. Practical implications Although MOOCs provide a platform for learning in highly scalable and flexible manner, they are prone to early dropout and low completion rate. This study outlines a data-driven approach to improve students’ learning experience and decrease the dropout rate. Social implications Early predictions based on individual’s participation can help educators provide support to students who are struggling in the course. Originality/value This study outlines the innovative use of process mining techniques in education data mining to help educators gather data-driven insight on student performances in the enrolled courses.

Comparative Analysis of Machine Learning Techniques Using Predictive Modeling

2020 ◽  
Vol 13 ◽  
Author(s):  
Ritu Khandelwal ◽  
Hemlata Goyal ◽  
Rajveer Singh Shekhawat
Keyword(s):  
Machine Learning ◽  
Comparative Analysis ◽  
Data Science ◽  
Training Data ◽  
Machine Learning Techniques ◽  
Future Trends ◽  
Data Set ◽  
Learning Stage ◽  
Learning Techniques ◽  
Different Types

Introduction: Machine learning is an intelligent technology that works as a bridge between businesses and data science. With the involvement of data science, the business goal focuses on findings to get valuable insights on available data. The large part of Indian Cinema is Bollywood which is a multi-million dollar industry. This paper attempts to predict whether the upcoming Bollywood Movie would be Blockbuster, Superhit, Hit, Average or Flop. For this Machine Learning techniques (classification and prediction) will be applied. To make classifier or prediction model first step is the learning stage in which we need to give the training data set to train the model by applying some technique or algorithm and after that different rules are generated which helps to make a model and predict future trends in different types of organizations. Methods: All the techniques related to classification and Prediction such as Support Vector Machine(SVM), Random Forest, Decision Tree, Naïve Bayes, Logistic Regression, Adaboost, and KNN will be applied and try to find out efficient and effective results. All these functionalities can be applied with GUI Based workflows available with various categories such as data, Visualize, Model, and Evaluate. Result: To make classifier or prediction model first step is learning stage in which we need to give the training data set to train the model by applying some technique or algorithm and after that different rules are generated which helps to make a model and predict future trends in different types of organizations Conclusion: This paper focuses on Comparative Analysis that would be performed based on different parameters such as Accuracy, Confusion Matrix to identify the best possible model for predicting the movie Success. By using Advertisement Propaganda, they can plan for the best time to release the movie according to the predicted success rate to gain higher benefits. Discussion: Data Mining is the process of discovering different patterns from large data sets and from that various relationships are also discovered to solve various problems that come in business and helps to predict the forthcoming trends. This Prediction can help Production Houses for Advertisement Propaganda and also they can plan their costs and by assuring these factors they can make the movie more profitable.

Machine Learning Based Assembly of Fragments of Ancient Papyrus

2021 ◽  
Vol 14 (3) ◽  
pp. 1-21
Author(s):  
Roy Abitbol ◽  
Ilan Shimshoni ◽  
Jonathan Ben-Dov
Keyword(s):  
Machine Learning ◽  
Spatial Information ◽  
Real Life ◽  
Dead Sea ◽  
Machine Learning Techniques ◽  
Automated Classification ◽  
Learning Techniques ◽  
Test Batch ◽  
Validation Set ◽  
Local Edge

The task of assembling fragments in a puzzle-like manner into a composite picture plays a significant role in the field of archaeology as it supports researchers in their attempt to reconstruct historic artifacts. In this article, we propose a method for matching and assembling pairs of ancient papyrus fragments containing mostly unknown scriptures. Papyrus paper is manufactured from papyrus plants and therefore portrays typical thread patterns resulting from the plant’s stems. The proposed algorithm is founded on the hypothesis that these thread patterns contain unique local attributes such that nearby fragments show similar patterns reflecting the continuations of the threads. We posit that these patterns can be exploited using image processing and machine learning techniques to identify matching fragments. The algorithm and system which we present support the quick and automated classification of matching pairs of papyrus fragments as well as the geometric alignment of the pairs against each other. The algorithm consists of a series of steps and is based on deep-learning and machine learning methods. The first step is to deconstruct the problem of matching fragments into a smaller problem of finding thread continuation matches in local edge areas (squares) between pairs of fragments. This phase is solved using a convolutional neural network ingesting raw images of the edge areas and producing local matching scores. The result of this stage yields very high recall but low precision. Thus, we utilize these scores in order to conclude about the matching of entire fragments pairs by establishing an elaborate voting mechanism. We enhance this voting with geometric alignment techniques from which we extract additional spatial information. Eventually, we feed all the data collected from these steps into a Random Forest classifier in order to produce a higher order classifier capable of predicting whether a pair of fragments is a match. Our algorithm was trained on a batch of fragments which was excavated from the Dead Sea caves and is dated circa the 1st century BCE. The algorithm shows excellent results on a validation set which is of a similar origin and conditions. We then tried to run the algorithm against a real-life set of fragments for which we have no prior knowledge or labeling of matches. This test batch is considered extremely challenging due to its poor condition and the small size of its fragments. Evidently, numerous researchers have tried seeking matches within this batch with very little success. Our algorithm performance on this batch was sub-optimal, returning a relatively large ratio of false positives. However, the algorithm was quite useful by eliminating 98% of the possible matches thus reducing the amount of work needed for manual inspection. Indeed, experts that reviewed the results have identified some positive matches as potentially true and referred them for further investigation.

scholarly journals Formulating Modes of Cooperative Leaning for Education for Sustainable Development

2021 ◽  
Vol 13 (6) ◽  
pp. 3465
Author(s):  
Jordi Colomer ◽  
Dolors Cañabate ◽  
Brigita Stanikūnienė ◽  
Remigijus Bubnys
Keyword(s):  
Sustainable Development ◽  
Real Life ◽  
Education For Sustainable Development ◽  
Professional Life ◽  
Global Challenges ◽  
Life Problems ◽  
The Face ◽  
Contemporary Education ◽  
The Impact

In the face of today’s global challenges, the practice and theory of contemporary education inevitably focuses on developing the competences that help individuals to find meaningfulness in their societal and professional life, to understand the impact of local actions on global processes and to enable them to solve real-life problems [...]

scholarly journals A Data-Driven Method for the Temporal Estimation of Soil Water Potential and Its Application for Shallow Landslides Prediction

Water ◽  
2021 ◽  
Vol 13 (9) ◽  
pp. 1208
Author(s):  
Massimiliano Bordoni ◽  
Fabrizio Inzaghi ◽  
Valerio Vivaldi ◽  
Roberto Valentino ◽  
Marco Bittelli ◽  
...  
Keyword(s):  
Machine Learning ◽  
Water Potential ◽  
Soil Water ◽  
Temporal Trends ◽  
Soil Water Potential ◽  
Shallow Landslides ◽  
Water Dynamics ◽  
Data Driven ◽  
Machine Learning Techniques ◽  
Physically Based

Soil water potential is a key factor to study water dynamics in soil and for estimating the occurrence of natural hazards, as landslides. This parameter can be measured in field or estimated through physically-based models, limited by the availability of effective input soil properties and preliminary calibrations. Data-driven models, based on machine learning techniques, could overcome these gaps. The aim of this paper is then to develop an innovative machine learning methodology to assess soil water potential trends and to implement them in models to predict shallow landslides. Monitoring data since 2012 from test-sites slopes in Oltrepò Pavese (northern Italy) were used to build the models. Within the tested techniques, Random Forest models allowed an outstanding reconstruction of measured soil water potential temporal trends. Each model is sensitive to meteorological and hydrological characteristics according to soil depths and features. Reliability of the proposed models was confirmed by correct estimation of days when shallow landslides were triggered in the study areas in December 2020, after implementing the modeled trends on a slope stability model, and by the correct choice of physically-based rainfall thresholds. These results confirm the potential application of the developed methodology to estimate hydrological scenarios that could be used for decision-making purposes.

scholarly journals Classification of multiwavelength transients with Machine Learning

2020 ◽  
Author(s):  
K Sooknunan ◽  
M Lochner ◽  
Bruce A Bassett ◽  
H V Peiris ◽  
R Fender ◽  
...  
Keyword(s):  
Machine Learning ◽  
Small Sample ◽  
Light Curves ◽  
Machine Learning Techniques ◽  
Optical Data ◽  
Test Time ◽  
Test Accuracy ◽  
Training Set ◽  
The Impact

Abstract With the advent of powerful telescopes such as the Square Kilometer Array and the Vera C. Rubin Observatory, we are entering an era of multiwavelength transient astronomy that will lead to a dramatic increase in data volume. Machine learning techniques are well suited to address this data challenge and rapidly classify newly detected transients. We present a multiwavelength classification algorithm consisting of three steps: (1) interpolation and augmentation of the data using Gaussian processes; (2) feature extraction using wavelets; (3) classification with random forests. Augmentation provides improved performance at test time by balancing the classes and adding diversity into the training set. In the first application of machine learning to the classification of real radio transient data, we apply our technique to the Green Bank Interferometer and other radio light curves. We find we are able to accurately classify most of the eleven classes of radio variables and transients after just eight hours of observations, achieving an overall test accuracy of 78%. We fully investigate the impact of the small sample size of 82 publicly available light curves and use data augmentation techniques to mitigate the effect. We also show that on a significantly larger simulated representative training set that the algorithm achieves an overall accuracy of 97%, illustrating that the method is likely to provide excellent performance on future surveys. Finally, we demonstrate the effectiveness of simultaneous multiwavelength observations by showing how incorporating just one optical data point into the analysis improves the accuracy of the worst performing class by 19%.

Sign in / Sign up

Export Citation Format

Share Document