A MACHINE LEARNING PIPELINE ARTICULATING SATELLITE IMAGERY AND OPENSTREETMAP FOR ROAD DETECTION

Abstract. Satellite imagery from earth observation missions enable processing big data to gather information about the world. Automatizing the creation of maps that reflect ground truth is a desirable outcome that would aid decision makers to take adequate actions in alignment with the United Nations Sustainable Development Goals. In order to harness the power that the availability of the new generation of satellites enable, it is necessary to implement techniques capable of handling annotations for the massive volume and variability of high spatial resolution imagery for further processing. However, the availability of public datasets for training machine learning models for image segmentation plays an important role for scalability.This work focuses on bridging remote sensing and computer vision by providing an open source based pipeline for generating machine learning training datasets for road detection in an area of interest. The proposed pipeline addresses road detection as a binary classification problem using road annotations existing in OpenStreetMap for creating masks. For this case study, Planet images of 3m resolution are used for creating a training dataset for road detection in Kenya.

Download Full-text

Advancing Stress Detection Methodology with Deep Learning Techniques Targeting UX Evaluation in AAL Scenarios: Applying Embeddings for Categorical Variables

Electronics ◽

10.3390/electronics10131550 ◽

2021 ◽

Vol 10 (13) ◽

pp. 1550

Author(s):

Alexandros Liapis ◽

Evanthia Faliagka ◽

Christos P. Antonopoulos ◽

Georgios Keramidas ◽

Nikolaos Voros

Keyword(s):

Machine Learning ◽

Deep Learning ◽

User Experience ◽

Electrodermal Activity ◽

Binary Classification ◽

Research Question ◽

Classification Problem ◽

Categorical Variables ◽

Stress Detection ◽

Software Failures

Physiological measurements have been widely used by researchers and practitioners in order to address the stress detection challenge. So far, various datasets for stress detection have been recorded and are available to the research community for testing and benchmarking. The majority of the stress-related available datasets have been recorded while users were exposed to intense stressors, such as songs, movie clips, major hardware/software failures, image datasets, and gaming scenarios. However, it remains an open research question if such datasets can be used for creating models that will effectively detect stress in different contexts. This paper investigates the performance of the publicly available physiological dataset named WESAD (wearable stress and affect detection) in the context of user experience (UX) evaluation. More specifically, electrodermal activity (EDA) and skin temperature (ST) signals from WESAD were used in order to train three traditional machine learning classifiers and a simple feed forward deep learning artificial neural network combining continues variables and entity embeddings. Regarding the binary classification problem (stress vs. no stress), high accuracy (up to 97.4%), for both training approaches (deep-learning, machine learning), was achieved. Regarding the stress detection effectiveness of the created models in another context, such as user experience (UX) evaluation, the results were quite impressive. More specifically, the deep-learning model achieved a rather high agreement when a user-annotated dataset was used for validation.

Download Full-text

Programmatic Labeling of Dark Data for Artificial Intelligence in Spatial Informatics

10.5194/egusphere-egu21-16326 ◽

2021 ◽

Author(s):

Jason Meil

Keyword(s):

Machine Learning ◽

Binary Classification ◽

Joint Probability ◽

Ground Truth ◽

Training Data ◽

Data Sources ◽

Joint Probability Distribution ◽

Weak Supervision ◽

Response Variable ◽

High Degree

Data preparation process generally consumes up to 80% of the Data Scientists time, with 60% of that being attributed to cleaning and labeling data.[1]&#160; Our solution is to use automated pipelines to prepare, annotate, and catalog data. The first step upon ingestion, especially in the case of real world&#8212;unstructured and unlabeled datasets&#8212;is to leverage Snorkel, a tool specifically designed around a paradigm to rapidly create, manage, and model training data. Configured properly, Snorkel can be leveraged to temper this labeling bottle-neck through a process called weak supervision. Weak supervision uses programmatic labeling functions&#8212;heuristics, distant supervision, SME or knowledge base&#8212;scripted in python to generate &#8220;noisy labels&#8221;. The function traverses the entirety of the dataset and feeds the labeled data into a generative&#8212;conditionally probabilistic&#8212;model. The function of this model is to output the distribution of each response variable and predict the conditional probability based on a joint probability distribution algorithm. This is done by comparing the various labeling functions and the degree to which their outputs are congruent to each other. A single labeling function that has a high degree of congruence with other labeling functions will have a high degree of learned accuracy, that is, the fraction of predictions that the model got right. Conversely, single labeling functions that have a low degree of congruence with other functions will have low learned accuracy. Each prediction is then combined by the estimated weighted accuracy, whereby the predictions of the higher learned functions are counted multiple times. The result yields a transformation from a binary classification of 0 or 1 to a fuzzy label between 0 and 1&#8212; there is &#8220;x&#8221; probability that based on heuristic &#8220;n&#8221;, the response variable is &#8220;y&#8221;. The addition of data to this generative model multi-class inference will be made on the response variables positive, negative, or abstain, assigning probabilistic labels to potentially millions of data points. Thus, we have generated a discriminative ground truth for all further labeling efforts and have improved the scalability of our models. Labeling functions can be applied to unlabeled data to further machine learning efforts. &#160; Once our datasets are labeled and a ground truth is established, we need to persist the data into our delta lake since it combines the most performant aspects of a warehouse with the low-cost storage for data lakes. In addition, the lake can accept unstructured, semi structured, or structured data sources, and those sources can be further aggregated into raw ingestion, cleaned, and feature engineered data layers.&#160; By sectioning off the data sources into these &#8220;layers&#8221;, the data engineering portion is abstracted away from the data scientist, who can access model ready data at any time.&#160; Data can be ingested via batch or stream.&#160; &#160; The design of the entire ecosystem is to eliminate as much technical debt in machine learning paradigms as possible in terms of configuration, data collection, verification, governance, extraction, analytics, process management, resource management, infrastructure, monitoring, and post verification.&#160;

Download Full-text

Predicting Takeover Success Using Machine Learning Techniques

Journal of Business & Economics Research (JBER) ◽

10.19030/jber.v10i10.7264 ◽

2012 ◽

Vol 10 (10) ◽

pp. 547

Author(s):

Mei Zhang ◽

Gregory Johnson ◽

Jia Wang

Keyword(s):

Machine Learning ◽

Learning Community ◽

Binary Classification ◽

Classification Problem ◽

Machine Learning Techniques ◽

Success Prediction ◽

Support Vector ◽

Font Size ◽

Network Support ◽

Learning Techniques

A takeover success prediction model aims at predicting the probability that a takeover attempt will succeed by using publicly available information at the time of the announcement. We perform a thorough study using machine learning techniques to predict takeover success. Specifically, we model takeover success prediction as a binary classification problem, which has been widely studied in the machine learning community. Motivated by the recent advance in machine learning, we empirically evaluate and analyze many state-of-the-art classifiers, including logistic regression, artificial neural network, support vector machines with different kernels, decision trees, random forest, and Adaboost. The experiments validate the effectiveness of applying machine learning in takeover success prediction, and we found that the support vector machine with linear kernel and the Adaboost with stump weak classifiers perform the best for the task. The result is consistent with the general observations of these two approaches.

Download Full-text

How Does Knowledge of the AUC Constrain the Set of Possible Ground-Truth Labelings?

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33015425 ◽

2019 ◽

Vol 33 ◽

pp. 5425-5432

Author(s):

Jacob Whitehill

Keyword(s):

Machine Learning ◽

Empirical Evidence ◽

Recent Work ◽

Roc Curve ◽

Binary Classification ◽

Ground Truth ◽

Mathematical Structure ◽

Test Set ◽

Classification Tasks ◽

N Vector

Recent work on privacy-preserving machine learning has considered how datamining competitions such as Kaggle could potentially be “hacked”, either intentionally or inadvertently, by using information from an oracle that reports a classifier’s accuracy on the test set (Blum and Hardt 2015; Hardt and Ullman 2014; Zheng 2015; Whitehill 2016). For binary classification tasks in particular, one of the most common accuracy metrics is the Area Under the ROC Curve (AUC), and in this paper we explore the mathematical structure of how the AUC is computed from an n-vector of real-valued “guesses” with respect to the ground-truth labels. Under the assumption of perfect knowledge of the test set AUC c=p/q, we show how knowing c constrains the set W of possible ground-truth labelings, and we derive an algorithm both to compute the exact number of such labelings and to enumerate efficiently over them. We also provide empirical evidence that, surprisingly, the number of compatible labelings can actually decrease as n grows, until a test set-dependent threshold is reached. Finally, we show how W can be efficiently whittled down, through pairs of oracle queries, to infer all the groundtruth test labels with complete certainty.

Download Full-text

Predicting Anesthetic Infusion Events Using Machine Learning

10.21203/rs.3.rs-783161/v1 ◽

2021 ◽

Author(s):

Naoki Miyaguchi ◽

Koh Takeuchi ◽

Hisashi Kashima ◽

Mizuki Morita ◽

Hiroshi Morimatsu

Keyword(s):

Machine Learning ◽

Flow Rate ◽

Short Term Memory ◽

Binary Classification ◽

Classification Problem ◽

Clinical Findings ◽

Support Vector ◽

Learning Models ◽

Continuous Administration ◽

Machine Learning Models

Abstract Recently, research has been conducted to automatically control anesthesia using machine learning, with the aim of alleviating the shortage of anesthesiologists. In this study, we address the problem of predicting decisions made by anesthesiologists during surgery using machine learning; specifically, we formulate a decision making problem by increasing the flow rate at each time point in the continuous administration of analgesic remifentanil as a supervised binary classification problem. The experiments were conducted to evaluate the prediction performance using six machine learning models: logistic regression, support vector machine, random forest, LightGBM, artificial neural network, and long short-term memory (LSTM), using 210 case data collected during actual surgeries. The results demonstrated that when predicting the future increase in flow rate of remifentanil after 1 min, the model using LSTM was able to predict with scores of 0.659 for sensitivity, 0.732 for specificity, and 0.753 for ROC-AUC; this demonstrates the potential to predict the decisions made by anesthesiologists using machine learning. Furthermore, we examined the importance and contribution of the features of each model using shapley additive explanations—a method for interpreting predictions made by machine learning models. The trends indicated by the results were partially consistent with known clinical findings.

Download Full-text

An Improved Binary Relevance Algorithm for Multi-Label Classification

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.536-537.394 ◽

2014 ◽

Vol 536-537 ◽

pp. 394-398 ◽

Cited By ~ 2

Author(s):

Tao Guo ◽

Gui Yang Li

Keyword(s):

Machine Learning ◽

Binary Classification ◽

Classification Problem ◽

Learning Task ◽

Binary Classifier ◽

Classification Problems ◽

Binary Relevance ◽

Label Information ◽

Benchmark Datasets

Multi-label classification (MLC) is a machine learning task aiming to predict multiple labels for a given instance. The widely known binary relevance (BR) learns one classifier for each label without considering the correlation among labels. In this paper, an improved binary relevance algorithm (IBRAM) is proposed. This algorithm is derived form binary relevance method. It sets two layers to decompose the multi-label classification problem into L independent binary classification problems respectively. In the first layer, binary classifier is built one for each label. In the second layer, the label information from the first layer is fully used to help to generate final predicting by consider the correlation among labels. Experiments on benchmark datasets validate the effectiveness of proposed approach against other well-established methods.

Download Full-text

Collecting a Large Scale Dataset for Classifying Fake News Tweets Using Weak Supervision

Future Internet ◽

10.3390/fi13050114 ◽

2021 ◽

Vol 13 (5) ◽

pp. 114

Author(s):

Stefan Helmstetter ◽

Heiko Paulheim

Keyword(s):

Large Scale ◽

Binary Classification ◽

Classification Problem ◽

Training Dataset ◽

Fake News ◽

Weak Supervision ◽

Alternative Approach ◽

Large Scale Dataset ◽

Tweet Classification

The problem of automatic detection of fake news in social media, e.g., on Twitter, has recently drawn some attention. Although, from a technical perspective, it can be regarded as a straight-forward, binary classification problem, the major challenge is the collection of large enough training corpora, since manual annotation of tweets as fake or non-fake news is an expensive and tedious endeavor, and recent approaches utilizing distributional semantics require large training corpora. In this paper, we introduce an alternative approach for creating a large-scale dataset for tweet classification with minimal user intervention. The approach relies on weak supervision and automatically collects a large-scale, but very noisy, training dataset comprising hundreds of thousands of tweets. As a weak supervision signal, we label tweets by their source, i.e., trustworthy or untrustworthy source, and train a classifier on this dataset. We then use that classifier for a different classification target, i.e., the classification of fake and non-fake tweets. Although the labels are not accurate according to the new classification target (not all tweets by an untrustworthy source need to be fake news, and vice versa), we show that despite this unclean, inaccurate dataset, the results are comparable to those achieved using a manually labeled set of tweets. Moreover, we show that the combination of the large-scale noisy dataset with a human labeled one yields more advantageous results than either of the two alone.

Download Full-text

Towards Urban Environment Familiarity Prediction

Advances in Cartography and GIScience of the ICA ◽

10.5194/ica-adv-2-5-2019 ◽

2019 ◽

Vol 2 ◽

pp. 1-8

Author(s):

Lukas Gokl ◽

Marvin Mc Cutchan ◽

Bartosz Mazurkiewicz ◽

Paolo Fogliaroni ◽

Ioannis Giannopoulos

Keyword(s):

Machine Learning ◽

Information Exchange ◽

Municipal Government ◽

Binary Classification ◽

Ground Truth ◽

Machine Learning Algorithms ◽

Location Based Services ◽

Ground Truth Data ◽

Level Of Details

Abstract. Location Based Services (LBS) are definitely very helpful for people that interact within an unfamiliar environment, but also for those that already possess a certain level of familiarity with it. In order to avoid overwhelming familiar users with unnecessary information, the level of details offered by the LBS shall be adapted to the level of familiarity with the environment: providing more details to unfamiliar users and a lighter amount of information (that would be superfluous, if not even misleading) to the users that are more familiar with the current environment. Currently, the information exchange between the service and its users is not taking into account familiarity. Within this work, we investigate the potential of machine learning for a binary classification of environment familiarity (i.e., familiar vs unfamiliar) with the surrounding environment. For this purpose, a 3D virtual environment based on a part of Vienna, Austria was designed using datasets from the municipal government. During a navigation experiment with 22 participants we collected ground truth data in order to train four machine learning algorithms. The captured data included motion and orientation of the users as well as visual interaction with the surrounding buildings during navigation. This work demonstrates the potential of machine learning for predicting the state of familiarity as an enabling step for the implementation of LBS better tailored to the user.

Download Full-text

Evaluation Of Machine Learning Tools For Distinguishing Fraud From Error

Journal of Business & Economics Research (JBER) ◽

10.19030/jber.v11i9.8067 ◽

2013 ◽

Vol 11 (9) ◽

pp. 393

Author(s):

Mei Zhang

Keyword(s):

Machine Learning ◽

Binary Classification ◽

Classification Problem ◽

General Purpose ◽

Machine Learning Techniques ◽

Support Vector ◽

Learning Approaches ◽

Learning Tools ◽

Pure Chance ◽

Error Classification

Fraud and error are two underlying sources of misstated financial statements. Modern machine learning techniques provide a potential direction to distinguish the two factors in such statements. In this paper, a thorough evaluation is conducted evaluation on how the off-the-shelf machine learning tools perform for fraud/error classification. In particular, the task is treated as a standard binary classification problem; i.e., mapping from an input vector of financial indices to a class label which is either error or fraud. With a real dataset of financial restatements, this study empirically evaluates and analyzes five state-of-the-art classifiers, including logistic regression, artificial neural network, support vector machines, decision trees, and bagging. There are several important observations from the experimental results. First, it is observed that bagging performs the best among these commonly used general purpose machine learning tools. Second, the results show that the underlying relationship from the statement indices to the fraud/error decision is likely to be non-linear. Third, it is very challenging to distinguish error from fraud, and general machine learning approaches, though perform better than pure chance, leave much room for improvement. The results suggest that more advanced or task-specific solutions are needed for fraud/error classification.

Download Full-text

ANALYSIS OF MACHINE LEARNING ALGORITHMS FOR THE BINARY CLASSIFICATION PROBLEM

CHERKASY UNIVERSITY BULLETIN: APPLIED MATHEMATICS. INFORMATICS ◽

10.31651/2076-5886-2019-2-86-95 ◽

2020 ◽

pp. 86-95

Author(s):

Oleksandr PISKUN ◽

Keyword(s):

Machine Learning ◽

Binary Classification ◽

Learning Algorithms ◽

Classification Problem ◽

Machine Learning Algorithms ◽

Binary Classification Problem

Download Full-text