Comparing Genomic Prediction Models by Means of Cross Validation

In the two decades of continuous development of genomic selection, a great variety of models have been proposed to make predictions from the information available in dense marker panels. Besides deciding which particular model to use, practitioners also need to make many minor choices for those parameters in the model which are not typically estimated by the data (so called “hyper-parameters”). When the focus is placed on predictions, most of these decisions are made in a direction sought to optimize predictive accuracy. Here we discuss and illustrate using publicly available crop datasets the use of cross validation to make many such decisions. In particular, we emphasize the importance of paired comparisons to achieve high power in the comparison between candidate models, as well as the need to define notions of relevance in the difference between their performances. Regarding the latter, we borrow the idea of equivalence margins from clinical research and introduce new statistical tests. We conclude that most hyper-parameters can be learnt from the data by either minimizing REML or by using weakly-informative priors, with good predictive results. In particular, the default options in a popular software are generally competitive with the optimal values. With regard to the performance assessments themselves, we conclude that the paired k-fold cross validation is a generally applicable and statistically powerful methodology to assess differences in model accuracies. Coupled with the definition of equivalence margins based on expected genetic gain, it becomes a useful tool for breeders.

Download Full-text

Measuring the illicit cigarette market in Mexico: a cross validation of two methodologies

Tobacco Control ◽

10.1136/tobaccocontrol-2019-055449 ◽

2020 ◽

pp. tobaccocontrol-2019-055449 ◽

Cited By ~ 3

Author(s):

Belen Saenz de Miera Juarez ◽

Luz Myriam Reynales-Shigematsu ◽

Michal Stoklosa ◽

Kevin Welding ◽

Jeffrey Drope

Keyword(s):

Survey Data ◽

Cross Validation ◽

Tobacco Industry ◽

Statistical Tests ◽

Survey Methods ◽

Cigarette Use ◽

Illicit Trade ◽

Face To Face ◽

The Difference ◽

Cigarette Packs

ObjectiveTo compare two methods to estimate the magnitude of the illicit cigarette trade in Mexico and to contrast these results with tobacco industry figures.MethodsWe used two survey methods: a smoker survey and a discarded pack survey. Data were collected in eight major cities in Mexico between November and December 2017. A total of 2396 face-to-face interviews to adult smokers were conducted and 8204 discarded packs were collected. To determine whether cigarette packs were intended for the Mexican market, we analysed pack features required by Mexican regulations and self-reported brands of the last purchase. Standard statistical tests to compare proportions were employed. Correlates of illicit cigarette use were also analysed.ResultsThe share of cigarettes not intended for the Mexican market was 8.8% based on the analysis of discarded packs and 7.6% based on the survey of smokers, that is, the difference was small and only borderline significant overall (p=0.055). Also, both results were lower than those presented by the tobacco industry (16.6%). However, differences across methods were statistically significant for various cities.ConclusionOur results suggest that the optimal practice for the study of illicit trade is to cross validate estimates using both the smoker survey and the littered pack survey. If resources are limited, however, our findings indicate that either method could be used because both yield similar overall results, as longs as the potential biases are considered. Also, consistent with findings from other studies, our results suggest that the tobacco industry exaggerates the scope of illicit cigarette trade.

Download Full-text

Utilizing Statistical Tests for Comparing Machine Learning Algorithms

Kurdistan Journal of Applied Research ◽

10.24017/science.2021.1.8 ◽

2021 ◽

pp. 69-74

Author(s):

Hozan Khalid Hamarashid

Keyword(s):

Machine Learning ◽

Prediction Models ◽

Hypothesis Test ◽

Statistical Tests ◽

Learning Algorithms ◽

Performance Measure ◽

Machine Learning Algorithms ◽

Statistical Hypothesis ◽

The Mean ◽

The Difference

The mean result of machine learning models is determined by utilizing k-fold cross-validation. The algorithm with the best average performance should surpass those with the poorest. But what if the difference in average outcomes is the consequence of a statistical anomaly? To conduct whether or not the mean result differences between two algorithms is genuine then statistical hypothesis test is utilized. Using statistical hypothesis testing, this study will demonstrate how to compare machine learning algorithms. The output of several machine learning algorithms or simulation pipelines is compared during model selection. The model that performs the best based on your performance measure becomes the last model, which can be utilized to make predictions on new data. With classification and regression prediction models it can be conducted by utilizing traditional machine learning and deep learning methods. The difficulty is to identify whether or not the difference between two models is accurate.

Download Full-text

Observations on the Use of Crash Modification Factor-Corrected Crash Prediction Models to Identify Sites with Promise

Transportation Research Record Journal of the Transportation Research Board ◽

10.3141/2635-09 ◽

2017 ◽

Vol 2635 (1) ◽

pp. 71-78

Author(s):

Mark J. Poppe

Keyword(s):

Prediction Models ◽

Crash Prediction ◽

Network Screening ◽

Modification Factor ◽

Crash Modification Factor ◽

The Difference ◽

Definition Of ◽

A Site ◽

Crash Prediction Models ◽

Simple Models

The Highway Safety Manual (HSM) provides guidance on the application of crash prediction models for network screening, evaluation of alternative designs, and evaluation of implemented safety improvements. A variety of models are available. They may be classified as network screening level (or simple models), project level [or crash modification factor (CMF)–corrected models], and evaluation-level models. The use of crash prediction models to identify sites with promise of safety improvement is often based on assessing the difference between the expected number of crashes ( Ne) at a site and the predicted number of crashes ( Np) for similar sites within the population. A large difference between Ne and Np may denote a safety problem and be used to identify and rank sites with promise. The HSM indicates that CMF-corrected models may be used for network screening purposes. However, issues arise in the analysis regarding the definition of similar sites. When CMF-corrected models are used, the definition of similar sites changes with changes in the CMF adjustments particular to each site. Use of ( Ne − Np) for identifying and ranking sites with promise does not work well when the analysis uses a CMF-corrected estimate of Np. A large combined CMF will increase Np and thereby decrease the value ( Ne − Np). But a large combined CMF may point to a site with promise. This paper examines this phenomenon in detail, reviews a case study, and suggests that simple models may be preferable for identifying sites with promise.

Download Full-text

The Distinction between [P] and [S]

10.23943/princeton/9780691161006.003.0018 ◽

2017 ◽

Author(s):

Galen Strawson

Keyword(s):

Personal Identity ◽

Radical Change ◽

Legal Responsibility ◽

The Difference ◽

Definition Of ◽

Over Time

This chapter examines the difference between John Locke's definition of a person [P], considered as a kind of thing, and his definition of a subject of experience of a certain sophisticated sort [S]. It first discusses the equation [P] = [S], where [S] is assumed to be a continuing thing that is able to survive radical change of substantial realization, as well as Locke's position about consciousness in relation to [P]'s identity or existence over time as [S]. It argues that Locke is not guilty of circularity because he is not proposing consciousness as the determinant of [S]'s identity over time, but only of [S]'s moral and legal responsibility over time. Finally, it suggests that the terms “Person” and “Personal identity” pull apart, in Locke's scheme of things, but in a perfectly coherent way.

Download Full-text

Need of Human Rights as a Better Way of Life

Think India ◽

10.26643/think-india.v22i3.8075 ◽

2019 ◽

Vol 22 (3) ◽

pp. 72-83

Author(s):

Tushar Kadian

Keyword(s):

Human Rights ◽

Basic Needs ◽

Research Paper ◽

Human Being ◽

Way Of Life ◽

Labour Unions ◽

The People ◽

The Subject ◽

Definition Of ◽

Made In

Actually, basic needs postulates securing of the elementary conditions of existence to every human being. Despite of the practical and theoretical importance of the subject the greatest irony is non- availability of any universal preliminary definition of the concept of basic needs. Moreover, this becomes the reason for unpredictability of various political programmes aiming at providing basic needs to the people. The shift is necessary for development of this or any other conception. No labour reforms could be made in history till labours were treated as objects. Its only after they were started being treating as subjects, labour unions were allowed to represent themselves in strategy formulations that labour reforms could become a reality. The present research paper highlights the basic needs of Human Rights in life.

Download Full-text

L'anglicisme dans la langue française

Digital Press Social Sciences and Humanities ◽

10.29037/digitalpress.43286 ◽

2019 ◽

Vol 3 ◽

pp. 00013

Author(s):

Danny Susanto

Keyword(s):

English Language ◽

Oil Industry ◽

English Word ◽

Public Institutions ◽

Font Size ◽

The World ◽

French Speaking ◽

The Difference ◽

Definition Of ◽

Bibliographic Study

The purpose of this study is to analyze the phenomenon known as “anglicism”: a loan made to the English language by another language. Anglicism arose either from the adoption of an English word as a result of a translation defect despite the existence of an equivalent term in the language of the speaker, or from a wrong translation, as a word-by-word translation. Said phenomenon is very common nowadays and most languages of the world including making use of some linguistic concepts such as anglicism, neologism, syntax, morphology etc, this article addresses various aspects related to Anglicisms in French through a bibliographic study: the definition of Anglicism, the origin of Anglicisms in French and the current situation, the areas most affected by Anglicism, the different categories of Anglicism, the difference between French Anglicism in France and French-speaking Canada, the attitude of French-speaking society towards to the Anglicisms and their efforts to stop this phenomenon. The study shows that the areas affected are, among others, trade, travel, parliamentary and judicial institutions, sports, rail, industrial production and most recently film, industrial production, sport, oil industry, information technology, science and technology. Various initiatives have been implemented either by public institutions or by individuals who share concerns about the increasingly felt threat of the omnipresence of Anglicism in everyday life.

Download Full-text

Bioactivity Prediction Based on Matched Molecular Pair and Matched Molecular Series Methods

Current Pharmaceutical Design ◽

10.2174/1381612826666200427111309 ◽

2020 ◽

Vol 26 (33) ◽

pp. 4195-4205

Author(s):

Xiaoyu Ding ◽

Chen Cui ◽

Dingyan Wang ◽

Jihui Zhao ◽

Mingyue Zheng ◽

...

Keyword(s):

Prediction Model ◽

Large Scale ◽

Prediction Models ◽

Predictive Accuracy ◽

Lead Optimization ◽

Consensus Method ◽

Molecular Pair ◽

Bioactivity Prediction ◽

Compound Synthesis ◽

Consensus Modeling

Background: Enhancing a compound’s biological activity is the central task for lead optimization in small molecules drug discovery. However, it is laborious to perform many iterative rounds of compound synthesis and bioactivity tests. To address the issue, it is highly demanding to develop high quality in silico bioactivity prediction approaches, to prioritize such more active compound derivatives and reduce the trial-and-error process. Methods: Two kinds of bioactivity prediction models based on a large-scale structure-activity relationship (SAR) database were constructed. The first one is based on the similarity of substituents and realized by matched molecular pair analysis, including SA, SA_BR, SR, and SR_BR. The second one is based on SAR transferability and realized by matched molecular series analysis, including Single MMS pair, Full MMS series, and Multi single MMS pairs. Moreover, we also defined the application domain of models by using the distance-based threshold. Results: Among seven individual models, Multi single MMS pairs bioactivity prediction model showed the best performance (R2 = 0.828, MAE = 0.406, RMSE = 0.591), and the baseline model (SA) produced the most lower prediction accuracy (R2 = 0.798, MAE = 0.446, RMSE = 0.637). The predictive accuracy could further be improved by consensus modeling (R2 = 0.842, MAE = 0.397 and RMSE = 0.563). Conclusion: An accurate prediction model for bioactivity was built with a consensus method, which was superior to all individual models. Our model should be a valuable tool for lead optimization.

Download Full-text

Prognostic implication of CT-FFR based functional SYNTAX score in patients with de novo three-vessel disease

European Heart Journal - Cardiovascular Imaging ◽

10.1093/ehjci/jeaa256 ◽

2020 ◽

Author(s):

Hong Yan Qiao ◽

Jian Hua Li ◽

U Joseph Schoepf ◽

Richard R Bayer ◽

Fiona C Tinnefeld ◽

...

Keyword(s):

De Novo ◽

Predictive Accuracy ◽

Computed Tomographic Angiography ◽

Syntax Score ◽

Cardiac Events ◽

Fractional Flow ◽

Computed Tomographic ◽

Flow Reserve ◽

The Difference ◽

Functional Syntax

Abstract Aims This study was aimed at investigating whether a machine learning (ML)-based coronary computed tomographic angiography (CCTA) derived fractional flow reserve (CT-FFR) SYNTAX score (SS), ‘Functional SYNTAX score’ (FSSCTA), would predict clinical outcome in patients with three-vessel coronary artery disease (CAD). Methods and results The SS based on CCTA (SSCTA) and ICA (SSICA) were retrospectively collected in 227 consecutive patients with three-vessel CAD. FSSCTA was calculated by combining the anatomical data with functional data derived from a ML-based CT-FFR assessment. The ability of each score system to predict major adverse cardiac events (MACE) was compared. The difference between revascularization strategies directed by the anatomical SS and FSSCTA was also assessed. Two hundred and twenty-seven patients were divided into two groups according to the SSCTA cut-off value of 22. After determining FSSCTA for each patient, 22.9% of patients (52/227) were reclassified to a low-risk group (FSSCTA ≤ 22). In the low- vs. intermediate-to-high (>22) FSSCTA group, MACE occurred in 3.2% (4/125) vs. 34.3% (35/102), respectively (P < 0.001). The independent predictors of MACE were FSSCTA (OR = 1.21, P = 0.001) and diabetes (OR = 2.35, P = 0.048). FSSCTA demonstrated a better predictive accuracy for MACE compared with SSCTA (AUC: 0.81 vs. 0.75, P = 0.01) and SSICA (0.81 vs. 0.75, P < 0.001). After FSSCTA was revealed, 52 patients initially referred for CABG based on SSCTA would have been changed to PCI. Conclusion Recalculating SS by incorporating lesion-specific ischaemia as determined by ML-based CT-FFR is a better predictor of MACE in patients with three-vessel CAD. Additionally, the use of FSSCTA may alter selected revascularization strategies in these patients.

Download Full-text

Assessing elderly’s functional balance and mobility via analyzing data from waist-mounted tri-axial wearable accelerometers in timed up and go tests

BMC Medical Informatics and Decision Making ◽

10.1186/s12911-021-01463-4 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Lisha Yu ◽

Yang Zhao ◽

Hailiang Wang ◽

Tien-Lung Sun ◽

Terrence E. Murphy ◽

...

Keyword(s):

Predictive Models ◽

Cross Validation ◽

Motor Coordination ◽

Predictive Accuracy ◽

Short Form ◽

Community Dwelling ◽

Regularized Regression ◽

Balance Assessment ◽

Timed Up And Go ◽

Functional Balance

Abstract Background Poor balance has been cited as one of the key causal factors of falls. Timely detection of balance impairment can help identify the elderly prone to falls and also trigger early interventions to prevent them. The goal of this study was to develop a surrogate approach for assessing elderly’s functional balance based on Short Form Berg Balance Scale (SFBBS) score. Methods Data were collected from a waist-mounted tri-axial accelerometer while participants performed a timed up and go test. Clinically relevant variables were extracted from the segmented accelerometer signals for fitting SFBBS predictive models. Regularized regression together with random-shuffle-split cross-validation was used to facilitate the development of the predictive models for automatic balance estimation. Results Eighty-five community-dwelling older adults (72.12 ± 6.99 year) participated in our study. Our results demonstrated that combined clinical and sensor-based variables, together with regularized regression and cross-validation, achieved moderate-high predictive accuracy of SFBBS scores (mean MAE = 2.01 and mean RMSE = 2.55). Step length, gender, gait speed and linear acceleration variables describe the motor coordination were identified as significantly contributed variables of balance estimation. The predictive model also showed moderate-high discriminations in classifying the risk levels in the performance of three balance assessment motions in terms of AUC values of 0.72, 0.79 and 0.76 respectively. Conclusions The study presented a feasible option for quantitatively accurate, objectively measured, and unobtrusively collected functional balance assessment at the point-of-care or home environment. It also provided clinicians and elderly with stable and sensitive biomarkers for long-term monitoring of functional balance.

Download Full-text

Size-adjusted aortic valve area: refining the definition of severe aortic stenosis

European Heart Journal - Cardiovascular Imaging ◽

10.1093/ehjci/jeaa295 ◽

2020 ◽

Author(s):

Branka Vulesevic ◽

Naozumi Kubota ◽

Ian G Burwash ◽

Claire Cimadevilla ◽

Sarah Tubiana ◽

...

Keyword(s):

Aortic Valve ◽

Predictive Accuracy ◽

Severe Aortic Stenosis ◽

Aortic Valve Area ◽

Obese Patients ◽

Derivation Cohort ◽

Wide Range ◽

Definition Of ◽

Diagnostic And Prognostic Value ◽

Valve Area

Abstract Aims Severe aortic valve stenosis (AS) is defined by an aortic valve area (AVA) <1 cm2 or an AVA indexed to body surface area (BSA) <0.6 cm/m2, despite little evidence supporting the latter approach and important intrinsic limitations of BSA indexation. We hypothesized that AVA indexed to height (H) might be more applicable to a wide range of populations and body morphologies and might provide a better predictive accuracy. Methods and results In 1298 patients with degenerative AS and preserved ejection fraction from three different countries and continents (derivation cohort), we aimed to establish an AVA/H threshold that would be equivalent to 1.0 cm2 for defining severe AS. In a distinct prospective validation cohort of 395 patients, we compared the predictive accuracy of AVA/BSA and AVA/H. Correlations between AVA and AVA/BSA or AVA/H were excellent (all R2 > 0.79) but greater with AVA/H. Regressions lines were markedly different in obese and non-obese patients with AVA/BSA (P < 0.0001) but almost identical with AVA/H (P = 0.16). AVA/BSA values that corresponded to an AVA of 1.0 cm2 were markedly different in obese and non-obese patients (0.48 and 0.59 cm2/m2) but not with AVA/H (0.61 cm2/m for both). Agreement for the diagnosis of severe AS (AVA < 1 cm2) was significantly higher with AVA/H than with AVA/BSA (P < 0.05). Similar results were observed across the three countries. An AVA/H cut-off value of 0.6 cm2/m [HR = 8.2(5.6–12.1)] provided the best predictive value for the occurrence of AS-related events [absolute AVA of 1 cm2: HR = 7.3(5.0–10.7); AVA/BSA of 0.6 cm2/m2 HR = 6.7(4.4–10.0)]. Conclusion In a large multinational/multiracial cohort, AVA/H was better correlated with AVA than AVA/BSA and a cut-off value of 0.6 cm2/m provided a better diagnostic and prognostic value than 0.6 cm2/m2. Our results suggest that severe AS should be defined as an AVA < 1 cm2 or an AVA/H < 0.6 cm2/m rather than a BSA-indexed value of 0.6 cm2/m2.

Download Full-text