A Meta-analysis on Classification Model Performance in Real-World Datasets: An Exploratory View

2017 ◽  
Vol 31 (9-10) ◽  
pp. 715-732 ◽  
Author(s):  
David Gómez Guillén ◽  
Alfonso Rojas Espinosa
Keyword(s):  
Real World ◽  
Meta Analysis ◽  
Model Performance ◽  
Classification Model ◽  
Real World Datasets

scholarly journals A Variational Point Process Model for Social Event Sequences

2020 ◽  
Vol 34 (01) ◽  
pp. 173-180
Author(s):  
Zhen Pan ◽  
Zhenya Huang ◽  
Defu Lian ◽  
Enhong Chen
Keyword(s):  
Point Process ◽  
Real World ◽  
Process Model ◽  
Classification Model ◽  
Event Sequence ◽  
Event Sequences ◽  
Sequence Modeling ◽  
Stochastic Point Process ◽  
Feature Based ◽  
Real World Datasets

Many events occur in real-world and social networks. Events are related to the past and there are patterns in the evolution of event sequences. Understanding the patterns can help us better predict the type and arriving time of the next event. In the literature, both feature-based approaches and generative approaches are utilized to model the event sequence. Feature-based approaches extract a variety of features, and train a regression or classification model to make a prediction. Yet, their performance is dependent on the experience-based feature exaction. Generative approaches usually assume the evolution of events follow a stochastic point process (e.g., Poisson process or its complexer variants). However, the true distribution of events is never known and the performance depends on the design of stochastic process in practice. To solve the above challenges, in this paper, we present a novel probabilistic generative model for event sequences. The model is termed Variational Event Point Process (VEPP). Our model introduces variational auto-encoder to event sequence modeling that can better use the latent information and capture the distribution over inter-arrival time and types of event sequences. Experiments on real-world datasets prove effectiveness of our proposed model.

scholarly journals A General Two-stage Multi-label Ranking Framework

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Yanbing Xue ◽  
Milos Hauskrecht
Keyword(s):  
Real World ◽  
Classification Model ◽  
Stage Model ◽  
Two Stage ◽  
Label Ranking ◽  
Improved Performance ◽  
Real World Datasets ◽  
Post Hoc ◽  
Do So

In this paper we develop and study solutions for the multi-label ranking (MLR) problem. Briefly, the goal of multi-label ranking is not only to assign a set of relevant labels to a data instance but also to rank the labels according to their importance. To do so we propose a two-stage model that consists of: (1) a multi-label classification model that first selects an unordered set of labels for a data instance, and, (2) a label ordering model that orders the selected labels post-hoc in order of their importance. The advantage of such a model is that it can represent both the dependencies among labels, as well as, their importance. We evaluate the performance of our framework on both simulated and real-world datasets and show its improved performance compared to the existing multiple-label ranking solutions.

scholarly journals PI-Bully: Personalized Cyberbullying Detection with Peer Influence

2019 ◽  
Author(s):  
Lu Cheng ◽  
Jundong Li ◽  
Yasin Silva ◽  
Deborah Hall ◽  
Huan Liu
Keyword(s):  
Peer Influence ◽  
Real World ◽  
Classification Model ◽  
Learning Models ◽  
Individual User ◽  
Collaborative Environment ◽  
Accurate Detection ◽  
Real World Datasets ◽  
Cyberbullying Detection

Cyberbullying has become one of the most pressing online risks for adolescents and has raised serious concerns in society. Recent years have witnessed a surge in research aimed at developing principled learning models to detect cyberbullying behaviors. These efforts have primarily focused on building a single generic classification model to differentiate bullying content from normal (non-bullying) content among all users. These models treat users equally and overlook idiosyncratic information about users that might facilitate the accurate detection of cyberbullying. In this paper, we propose a personalized cyberbullying detection framework, PI-Bully, that draws on empirical findings from psychology highlighting unique characteristics of victims and bullies and peer influence from like-minded users as predictors of cyberbullying behaviors. Our framework is novel in its ability to model peer influence in a collaborative environment and tailor cyberbullying prediction for each individual user. Extensive experimental evaluations on real-world datasets corroborate the effectiveness of the proposed framework.

Safety and efficacy of direct acting oral anticoagulants and vitamin K antagonists in nonvalvular atrial fibrillation – a network meta-analysis of real-world data

VASA ◽  
2019 ◽  
Vol 48 (2) ◽  
pp. 134-147 ◽  
Author(s):  
Mirko Hirschl ◽  
Michael Kundi
Keyword(s):  
Real World ◽  
Meta Analysis ◽  
Vitamin K Antagonists ◽  
Oral Anticoagulants ◽  
Efficacy And Safety ◽  
Nonvalvular Atrial Fibrillation ◽  
Real World Data ◽  
Hazard Ratios ◽  
Direct Acting ◽  
Event Rates

Abstract. Background: In randomized controlled trials (RCTs) direct acting oral anticoagulants (DOACs) showed a superior risk-benefit profile in comparison to vitamin K antagonists (VKAs) for patients with nonvalvular atrial fibrillation. Patients enrolled in such studies do not necessarily reflect the whole target population treated in real-world practice. Materials and methods: By a systematic literature search, 88 studies including 3,351,628 patients providing over 2.9 million patient-years of follow-up were identified. Hazard ratios and event-rates for the main efficacy and safety outcomes were extracted and the results for DOACs and VKAs combined by network meta-analysis. In addition, meta-regression was performed to identify factors responsible for heterogeneity across studies. Results: For stroke and systemic embolism as well as for major bleeding and intracranial bleeding real-world studies gave virtually the same result as RCTs with higher efficacy and lower major bleeding risk (for dabigatran and apixaban) and lower risk of intracranial bleeding (all DOACs) compared to VKAs. Results for gastrointestinal bleeding were consistently better for DOACs and hazard ratios of myocardial infarction were significantly lower in real-world for dabigatran and apixaban compared to RCTs. By a ranking analysis we found that apixaban is the safest anticoagulant drug, while rivaroxaban closely followed by dabigatran are the most efficacious. Risk of bias and heterogeneity was assessed and had little impact on the overall results. Analysis of effect modification could guide the clinical decision as no single DOAC was superior/inferior to the others under all conditions. Conclusions: DOACs were at least as efficacious as VKAs. In terms of safety endpoints, DOACs performed better under real-world conditions than in RCTs. The current real-world data showed that differences in efficacy and safety, despite generally low event rates, exist between DOACs. Knowledge about these differences in performance can contribute to a more personalized medicine.

scholarly journals Predicting Future Occurrence of Acute Hypotensive Episodes Using Noninvasive and Invasive Features

2021 ◽  
Vol 186 (Supplement_1) ◽  
pp. 445-451
Author(s):  
Yifei Sun ◽  
Navid Rashedi ◽  
Vikrant Vaze ◽  
Parikshit Shah ◽  
Ryan Halter ◽  
...  
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Real World ◽  
Short Term Memory ◽  
Model Performance ◽  
Learning Technologies ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
K Nearest Neighbor ◽  
Continuous Map

ABSTRACT Introduction Early prediction of the acute hypotensive episode (AHE) in critically ill patients has the potential to improve outcomes. In this study, we apply different machine learning algorithms to the MIMIC III Physionet dataset, containing more than 60,000 real-world intensive care unit records, to test commonly used machine learning technologies and compare their performances. Materials and Methods Five classification methods including K-nearest neighbor, logistic regression, support vector machine, random forest, and a deep learning method called long short-term memory are applied to predict an AHE 30 minutes in advance. An analysis comparing model performance when including versus excluding invasive features was conducted. To further study the pattern of the underlying mean arterial pressure (MAP), we apply a regression method to predict the continuous MAP values using linear regression over the next 60 minutes. Results Support vector machine yields the best performance in terms of recall (84%). Including the invasive features in the classification improves the performance significantly with both recall and precision increasing by more than 20 percentage points. We were able to predict the MAP with a root mean square error (a frequently used measure of the differences between the predicted values and the observed values) of 10 mmHg 60 minutes in the future. After converting continuous MAP predictions into AHE binary predictions, we achieve a 91% recall and 68% precision. In addition to predicting AHE, the MAP predictions provide clinically useful information regarding the timing and severity of the AHE occurrence. Conclusion We were able to predict AHE with precision and recall above 80% 30 minutes in advance with the large real-world dataset. The prediction of regression model can provide a more fine-grained, interpretable signal to practitioners. Model performance is improved by the inclusion of invasive features in predicting AHE, when compared to predicting the AHE based on only the available, restricted set of noninvasive technologies. This demonstrates the importance of exploring more noninvasive technologies for AHE prediction.

scholarly journals Effectiveness of transfer learning for enhancing tumor classification with a convolutional neural network on frozen sections

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Young-Gon Kim ◽  
Sungchul Kim ◽  
Cristina Eunbee Cho ◽  
In Hye Song ◽  
Hee Jin Lee ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Frozen Section ◽  
Medical Center ◽  
External Validation ◽  
Model Performance ◽  
Classification Model ◽  
Training Dataset

AbstractFast and accurate confirmation of metastasis on the frozen tissue section of intraoperative sentinel lymph node biopsy is an essential tool for critical surgical decisions. However, accurate diagnosis by pathologists is difficult within the time limitations. Training a robust and accurate deep learning model is also difficult owing to the limited number of frozen datasets with high quality labels. To overcome these issues, we validated the effectiveness of transfer learning from CAMELYON16 to improve performance of the convolutional neural network (CNN)-based classification model on our frozen dataset (N = 297) from Asan Medical Center (AMC). Among the 297 whole slide images (WSIs), 157 and 40 WSIs were used to train deep learning models with different dataset ratios at 2, 4, 8, 20, 40, and 100%. The remaining, i.e., 100 WSIs, were used to validate model performance in terms of patch- and slide-level classification. An additional 228 WSIs from Seoul National University Bundang Hospital (SNUBH) were used as an external validation. Three initial weights, i.e., scratch-based (random initialization), ImageNet-based, and CAMELYON16-based models were used to validate their effectiveness in external validation. In the patch-level classification results on the AMC dataset, CAMELYON16-based models trained with a small dataset (up to 40%, i.e., 62 WSIs) showed a significantly higher area under the curve (AUC) of 0.929 than those of the scratch- and ImageNet-based models at 0.897 and 0.919, respectively, while CAMELYON16-based and ImageNet-based models trained with 100% of the training dataset showed comparable AUCs at 0.944 and 0.943, respectively. For the external validation, CAMELYON16-based models showed higher AUCs than those of the scratch- and ImageNet-based models. Model performance for slide feasibility of the transfer learning to enhance model performance was validated in the case of frozen section datasets with limited numbers.

Time-Efficient Ensemble Learning with Sample Exchange for Edge Computing

2021 ◽  
Vol 21 (3) ◽  
pp. 1-17
Author(s):  
Wu Chen ◽  
Yong Yu ◽  
Keke Gai ◽  
Jiamou Liu ◽  
Kim-Kwang Raymond Choo
Keyword(s):  
Ensemble Learning ◽  
Real World ◽  
Interaction Mechanism ◽  
Training Model ◽  
Edge Computing ◽  
Learning Techniques ◽  
Multi Agent ◽  
Real World Datasets ◽  
Entire Dataset ◽  
Exchange Data

In existing ensemble learning algorithms (e.g., random forest), each base learner’s model needs the entire dataset for sampling and training. However, this may not be practical in many real-world applications, and it incurs additional computational costs. To achieve better efficiency, we propose a decentralized framework: Multi-Agent Ensemble. The framework leverages edge computing to facilitate ensemble learning techniques by focusing on the balancing of access restrictions (small sub-dataset) and accuracy enhancement. Specifically, network edge nodes (learners) are utilized to model classifications and predictions in our framework. Data is then distributed to multiple base learners who exchange data via an interaction mechanism to achieve improved prediction. The proposed approach relies on a training model rather than conventional centralized learning. Findings from the experimental evaluations using 20 real-world datasets suggest that Multi-Agent Ensemble outperforms other ensemble approaches in terms of accuracy even though the base learners require fewer samples (i.e., significant reduction in computation costs).

Sign in / Sign up

Export Citation Format

Share Document