scholarly journals Maintaining AUC and H-measure over time

2021 ◽  
Author(s):  
Nikolaj Tatti
Keyword(s):  
Convex Hull ◽  
Roc Curve ◽  
Search Tree ◽  
Alternative Measure ◽  
Step Process ◽  
Running Time ◽  
Data Points ◽  
Two Measures ◽  
A Minor ◽  
Over Time

AbstractMeasuring the performance of a classifier is a vital task in machine learning. The running time of an algorithm that computes the measure plays a very small role in an offline setting, for example, when the classifier is being developed by a researcher. However, the running time becomes more crucial if our goal is to monitor the performance of a classifier over time. In this paper we study three algorithms for maintaining two measures. The first algorithm maintains area under the ROC curve (AUC) under addition and deletion of data points in $$\mathcal {O} \mathopen {}\left( \log n\right)$$ O log n time. This is done by maintaining the data points sorted in a self-balanced search tree. In addition, we augment the search tree that allows us to query the ROC coordinates of a data point in $$\mathcal {O} \mathopen {}\left( \log n\right)$$ O log n time. In doing so we are able to maintain AUC in $$\mathcal {O} \mathopen {}\left( \log n\right)$$ O log n time. Our next two algorithms involve in maintaining H-measure, an alternative measure based on the ROC curve. Computing the measure is a two-step process: first we need to compute a convex hull of the ROC curve, followed by a sum over the convex hull. We demonstrate that we can maintain the convex hull using a minor modification of the classic convex hull maintenance algorithm. We then show that under certain conditions, we can compute the H-measure exactly in $$\mathcal {O} \mathopen {}\left( \log ^2 n\right)$$ O log 2 n time, and if the conditions are not met, then we can estimate the H-measure in $$\mathcal {O} \mathopen {}\left( (\log n + \epsilon ^{-1})\log n\right)$$ O ( log n + ϵ - 1 ) log n time. We show empirically that our methods are significantly faster than the baselines.

scholarly journals TreeMerge: a new method for improving the scalability of species tree estimation methods

2019 ◽  
Vol 35 (14) ◽  
pp. i417-i426 ◽  
Author(s):  
Erin K Molloy ◽  
Tandy Warnow
Keyword(s):  
Large Scale ◽  
Species Tree ◽  
New Method ◽  
Divide And Conquer ◽  
Supplementary Information ◽  
Estimation Methods ◽  
Running Time ◽  
Tree Estimation ◽  
Computationally Intensive ◽  
A Minor

Abstract Motivation At RECOMB-CG 2018, we presented NJMerge and showed that it could be used within a divide-and-conquer framework to scale computationally intensive methods for species tree estimation to larger datasets. However, NJMerge has two significant limitations: it can fail to return a tree and, when used within the proposed divide-and-conquer framework, has O(n5) running time for datasets with n species. Results Here we present a new method called ‘TreeMerge’ that improves on NJMerge in two ways: it is guaranteed to return a tree and it has dramatically faster running time within the same divide-and-conquer framework—only O(n2) time. We use a simulation study to evaluate TreeMerge in the context of multi-locus species tree estimation with two leading methods, ASTRAL-III and RAxML. We find that the divide-and-conquer framework using TreeMerge has a minor impact on species tree accuracy, dramatically reduces running time, and enables both ASTRAL-III and RAxML to complete on datasets (that they would otherwise fail on), when given 64 GB of memory and 48 h maximum running time. Thus, TreeMerge is a step toward a larger vision of enabling researchers with limited computational resources to perform large-scale species tree estimation, which we call Phylogenomics for All. Availability and implementation TreeMerge is publicly available on Github (http://github.com/ekmolloy/treemerge). Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Contrasting Monetary and Multidimensional Poverty Measures in a Low-Income Sub-Saharan African Country

2020 ◽  
Vol 151 (2) ◽  
pp. 547-574 ◽  
Author(s):  
Lukas Salecker ◽  
Anar K. Ahmadov ◽  
Leyla Karimli
Keyword(s):  
Risk Factors ◽  
Low Income ◽  
Multidimensional Poverty ◽  
Poverty Measurement ◽  
Sub Saharan Africa ◽  
Analytical Strategy ◽  
Poverty Risk ◽  
Two Measures ◽  
Sub Saharan ◽  
Over Time

AbstractDespite significant progress in poverty measurement, few studies have undertaken an in-depth comparison of monetary and multidimensional measures in the context of low-income countries and fewer still in Sub-Saharan Africa. Yet the differences can be particularly consequential in these settings. We address this gap by applying a distinct analytical strategy to the case of Rwanda. Using data from two waves of the Rwandan Integrated Household Living Conditions Survey, we combine comparing poverty rates cross-sectionally and over time, examining the overlaps and differences in the two measures, investigating poverty rates within population sub-groups, and estimating several statistical models to assess the differences between the two measures in identifying poverty risk factors. We find that using a monetary measure alone does not capture high incidence of multidimensional poverty in both waves, that it is possible to be multidimensional poor without being monetary poor, and that using a monetary measure alone overlooks significant change in multidimensional poverty over time. The two measures also differ in which poverty risk factors they put emphasis on. Relying only on monetary measures in low-income sub-Saharan Africa can send inaccurate signals to policymakers regarding the optimal design of social policies as well as monitoring their effectiveness.

Improving the Red-Black tree delete algorithm

2021 ◽  
Author(s):  
ZEGOUR Djamel Eddine
Keyword(s):  
Data Structure ◽  
Search Tree ◽  
Binary Search ◽  
Binary Search Tree ◽  
Search Performance ◽  
Running Time ◽  
Standard Algorithm ◽  
Color Changes ◽  
Red Black Tree ◽  
Symbol Tables

Abstract Today, Red-Black trees are becoming a popular data structure typically used to implement dictionaries, associative arrays, symbol tables within some compilers (C++, Java …) and many other systems. In this paper, we present an improvement of the delete algorithm of this kind of binary search tree. The proposed algorithm is very promising since it colors differently the tree while reducing color changes by a factor of about 29%. Moreover, the maintenance operations re-establishing Red-Black tree balance properties are reduced by a factor of about 11%. As a consequence, the proposed algorithm saves about 4% on running time when insert and delete operations are used together while conserving search performance of the standard algorithm.

scholarly journals Visualizing Belief in Meritocracy, 1930–2010

2018 ◽  
Vol 4 ◽  
pp. 237802311881180 ◽  
Author(s):  
Jonathan J. B. Mijs
Keyword(s):  
Survey Data ◽  
Representative Sample ◽  
Western World ◽  
Birth Cohorts ◽  
Popular Belief ◽  
Time Period ◽  
Level Data ◽  
Two Measures ◽  
Time Periods ◽  
Over Time

In this figure I describe the long trend in popular belief in meritocracy across the Western world between 1930 and 2010. Studying trends in attitudes is limited by the paucity of survey data that can be compared across countries and over time. Here, I show how to complement survey waves with cohort-level data. Repeated surveys draw on a representative sample of the population to describe the typical beliefs held by citizens in a given country and period. Leveraging the fact that citizens surveyed in a given year were born in different time-periods allows for a comparison of beliefs across birth cohorts. The latter overlaps with the former, but considerably extends the time period covered by the data. Taken together, the two measures give a “triangulated” longitudinal record of popular belief in meritocracy. I find that in most countries, popular belief in meritocracy is (much) stronger for more recent periods and cohorts.

scholarly journals The perception of biopsychosocial impacts of COVID-19 during lockdown restrictions over time in the UK – a mixed methods study

2021 ◽  
Vol 4 (Special2) ◽  
pp. 402-414
Author(s):  
Samuel Grimwood ◽  
Kaz Stuart ◽  
Ruth Browning ◽  
Elaine Bidmead ◽  
Thea Winn-Reed
Keyword(s):  
Social Issues ◽  
Negative Impact ◽  
Mixed Methods Study ◽  
Biological Domain ◽  
Data Points ◽  
Integrated Recovery ◽  
Biopsychosocial Health ◽  
The Uk ◽  
The Impact ◽  
Over Time

Background: The COVID-19 pandemic has profoundly impacted the health of individuals physically, mentally, and socially. This study aims to gain a deeper understanding of this impact across the pandemic from a biopsychosocial stance. Methods: A survey created by the research team was employed between November 2020 and February 2021 across social media, relevant organizations, and networks. The survey incorporated 5-time points across the different stages of the pandemic, covering biological, psychological, and social. There were 5 items for each survey (Very Positive affect to Very Negative affect), and analysis was undertaken using SPSS version 16. Descriptive statistics and non-parametric Friedman and Wilcoxon Tests, as well as correlations between the three domains, were implemented. Results: This study included 164 participants (77.0% female and 35.0% male) across 24 out of 38 counties in the UK. The impact of COVID-19 on biological domain was significant across the five data points χ2(4) = 63.99, p < 0.001, psychological χ2(4) = 118.939, p <0.001 and socially χ2(4) = 186.43, p <0.001. Between the 5 data points, 4 out of 5 had a negative impact, however between the first stage of lockdown and the easing of restrictions, findings for biological (Z=-2.35, p <0.05), psychological (Z=-6.61, p < 0.001), and socially (Z = -8.61, p <0.001) were positive. Negative correlations between the three domains across the pandemic are apparent, but in later stages, the biological domain had a positive correlation r = 0.52, p < 0.001. Conclusion: The data shows a negative impact from the self-reported perception of wellbeing from a biopsychosocial stance over time, as well as perceiving the three domains to interact negatively. To address these biopsychosocial issues, the research implies a place-based integrated recovery effort is needed, addressing biological, psychological, and social issues simultaneously. Further research should investigate biopsychosocial health among a more generalizable population.

Sphericity Evaluation Based on Minimum Circumscribed Sphere Method

2012 ◽  
Vol 433-440 ◽  
pp. 3146-3151 ◽  
Author(s):  
Fan Wu Meng ◽  
Chun Guang Xu ◽  
Juan Hao ◽  
Ding Guo Xiao
Keyword(s):  
Convex Hull ◽  
Early Stage ◽  
Measured Data ◽  
Efficient Approach ◽  
Critical Data ◽  
Material Condition ◽  
Data Points ◽  
Point Set ◽  
Data Point

The search of sphericity evaluation is a time-consuming work. The minimum circumscribed sphere (MCS) is suitable for the sphere with the maximum material condition. An algorithm of sphericity evaluation based on the MCS is introduced. The MCS of a measured data point set is determined by a small number of critical data points according to geometric criteria. The vertices of the convex hull are the candidates of these critical data points. Two theorems are developed to solve the sphericity evaluation problems. The validated results show that the proposed strategy offers an effective way to identify the critical data points at the early stage of computation and gives an efficient approach to solve the sphericity problems.

The Influence of Inhomogeneous Input Data from Different Waves on Predictive Model Development for COVID-19 ICU Patients (Preprint)

2021 ◽  
Author(s):  
Sebastian Johannes Fritsch ◽  
Konstantin Sharafutdinov ◽  
Moein Einollahzadeh Samadi ◽  
Gernot Marx ◽  
Andreas Schuppert ◽  
...  
Keyword(s):  
Machine Learning ◽  
Convex Hull ◽  
Prediction Models ◽  
Model Development ◽  
Predictive Performance ◽  
Support Vector ◽  
Good Prediction ◽  
The Impact ◽  
Second Wave ◽  
Over Time

BACKGROUND During the course of the COVID-19 pandemic, a variety of machine learning models were developed to predict different aspects of the disease, such as long-term causes, organ dysfunction or ICU mortality. The number of training datasets used has increased significantly over time. However, these data now come from different waves of the pandemic, not always addressing the same therapeutic approaches over time as well as changing outcomes between two waves. The impact of these changes on model development has not yet been studied. OBJECTIVE The aim of the investigation was to examine the predictive performance of several models trained with data from one wave predicting the second wave´s data and the impact of a pooling of these data sets. Finally, a method for comparison of different datasets for heterogeneity is introduced. METHODS We used two datasets from wave one and two to develop several predictive models for mortality of the patients. Four classification algorithms were used: logistic regression (LR), support vector machine (SVM), random forest classifier (RF) and AdaBoost classifier (ADA). We also performed a mutual prediction on the data of that wave which was not used for training. Then, we compared the performance of models when a pooled dataset from two waves was used. The populations from the different waves were checked for heterogeneity using a convex hull analysis. RESULTS 63 patients from wave one (03-06/2020) and 54 from wave two (08/2020-01/2021) were evaluated. For both waves separately, we found models reaching sufficient accuracies up to 0.79 AUROC (95%-CI 0.76-0.81) for SVM on the first wave and up 0.88 AUROC (95%-CI 0.86-0.89) for RF on the second wave. After the pooling of the data, the AUROC decreased relevantly. In the mutual prediction, models trained on second wave´s data showed, when applied on first wave´s data, a good prediction for non-survivors but an insufficient classification for survivors. The opposite situation (training: first wave, test: second wave) revealed the inverse behaviour with models correctly classifying survivors and incorrectly predicting non-survivors. The convex hull analysis for the first and second wave populations showed a more inhomogeneous distribution of underlying data when compared to randomly selected sets of patients of the same size. CONCLUSIONS Our work demonstrates that a larger dataset is not a universal solution to all machine learning problems in clinical settings. Rather, it shows that inhomogeneous data used to develop models can lead to serious problems. With the convex hull analysis, we offer a solution for this problem. The outcome of such an analysis can raise concerns if the pooling of different datasets would cause inhomogeneous patterns preventing a better predictive performance.

Guidance for Custom Alarm Design: You Don’t Have to “Crank it up to 11”

2019 ◽  
Vol 8 (1) ◽  
pp. 142-145
Author(s):  
Morgan E. Reynolds ◽  
Michael F. Rayo ◽  
Morgan Fitzgerald ◽  
Mahmoud Abdel - Rasoul ◽  
Susan D. Moffatt - Bruce
Keyword(s):  
Longitudinal Studies ◽  
First Impressions ◽  
Change Over Time ◽  
Daily Exposure ◽  
Real Work ◽  
Two Measures ◽  
The Relationship ◽  
Over Time

Changes in alarm perception and response after prolonged daily exposure is not well studied due to the difficulties in setting up rigorous longitudinal studies in real work domains. A prime example of this is the absence of research studying how conveyed urgency and identifiability of auditory alarms change over time. We conducted a three-year study to understand how alarm performance with respect to these two measures changed over time, ostensibly due to prolonged nurse exposure. Gaining a better understanding of the relationship between these two aspects of a sound’s sensory dimension could be extremely valuable to acoustical alarm designers, as it allows them to anticipate changes in the sounds’ sensory performance over time, and not be overly sensitive to first impressions of the auditory alarm set.

scholarly journals Repeatability and frequency of in-paddock sheep walk-over weights: implications for individual animal management

2014 ◽  
Vol 54 (2) ◽  
pp. 207 ◽  
Author(s):  
D. J. Brown ◽  
D. B. Savage ◽  
G. N. Hinch
Keyword(s):  
Radio Frequency Identification ◽  
Null Hypothesis ◽  
Average Reference ◽  
Animal Management ◽  
Data Points ◽  
Frequency Identification ◽  
The Mean ◽  
Gut Fill ◽  
On Farm ◽  
Over Time

Sheep liveweight is an indicator of nutritional status, and its measure may be used as an aid to nutritional management. When walk-over weighing (WOW), a remote weighing concept for grazing sheep, is combined with radio frequency identification (RFID), resulting ‘RFID-linked WOW’ data may enable the liveweight of individual sheep to be tracked over time. We investigated whether RFID-linked WOW data is sufficiently repeatable and frequent to generate individual liveweight estimates with 95% confidence intervals (95% CI) of <2 kg (a sufficient level of error to account for fluctuating gut fill) for a flock within timeframes suitable for management (1-day and 5-day timeframes). Four flocks of sheep were used to generate RFID-linked WOW datasets. RFID-linked WOW data were organised into three groups: raw (unfiltered), coarse filtered (remove all sheep-weights outside the flock’s liveweight range), and fine filtered (remove all sheep-weights outside a 25% range of a recent flock average reference liveweight). The repeatability of raw (unfiltered) RFID-linked WOW data was low (0.20), while a coarse (0.46) and fine (0.76) data filter improved repeatability. The 95% CI of raw RFID-linked WOW data was 27 kg, and was decreased by a coarse (11 kg) and fine (6 kg) data filter. Increasing the number of raw, coarse and fine-filtered data points to 190, 30 and 12 sheep-weights, respectively, decreased the 95% CI to <2 kg. The mean cumulative percentage of sheep achieving >11 fine-filtered RFID-linked WOW sheep-weights within a 1-day and 5-day timeframe was 0 and 10%, respectively. The null hypothesis was accepted: RFID-linked WOW data had low repeatability and was unable to generate liveweight estimates with a 95% CI of less than 2 kg within a suitable timeframe. Therefore, at this stage, RFID-linked WOW is not recommended for on-farm decision making of individual sheep.

Sign in / Sign up

Export Citation Format

Share Document