Use of Double Cross-Validation and Bootstrap Methods to Estimate Replicability of Results of Multiple Regression

1998 ◽  
Vol 86 (3_suppl) ◽  
pp. 1143-1152 ◽  
Author(s):  
Rebecca P. Ang
Keyword(s):  
Multiple Regression ◽  
Cross Validation ◽  
Statistical Significance ◽  
Significance Testing ◽  
Data Set ◽  
Bootstrap Methods ◽  
Bootstrap Procedure ◽  
Microcomputer Program ◽  
Validation Procedure ◽  
Double Cross

Researchers have all too often ignored replicability of results because they overly rely on significance testing. This is a misinformed view because statistical significance does not evaluate the importance or replicability of a result. This paper focuses on two methods of assessing replicability of results, double cross-validation and bootstrap procedures. Selected variables from Hughes, Cavell, and Grossman's (1997) data set of 112 cases are used to illustrate all these techniques as applied to the interpretation of multiple regression results. One statistical computer package SPSS is used for the double cross-validation procedure, and a 1987 microcomputer program package of Lunneborg is used to demonstrate the bootstrap procedure.

scholarly journals Development and double cross-validation of new spot urine sodium equation to predict 24-h urine sodium in the Malaysian population

2021 ◽  
Vol 40 (S1) ◽  
Author(s):  
Fatimah Othman ◽  
Rashidah Ambak ◽  
Mohd Azahadi Omar ◽  
Suzana Shahar ◽  
Noor Safiza Mohd Nor ◽  
...  
Keyword(s):  
Cross Validation ◽  
Sodium Intake ◽  
Study Groups ◽  
Urine Collection ◽  
Urine Sodium ◽  
Data Set ◽  
Spot Urine ◽  
Malaysian Population ◽  
New Equation ◽  
Double Cross

Abstract Background Monitoring sodium intake through 24-h urine collection sample is recommended, but the implementation of this method can be difficult. The objective of this study was to develop and validate an equation using spot urine concentration to predict 24-h sodium excretion in the Malaysian population. Methods This was a Malaysian Community Salt Study (MyCoSS) sub-study, which was conducted from October 2017 to March 2018. Out of 798 participants in the MyCoSS study who completed 24-h urine collection, 768 of them have collected one-time spot urine the following morning. They were randomly assigned into two groups to form separate spot urine equations. The final spot urine equation was derived from the entire data set after confirming the stability of the equation by double cross-validation in both study groups. Newly derived spot urine equation was developed using the coefficients from the multiple linear regression test. A Bland-Altman plot was used to measure the mean bias and limits of agreement between estimated and measured 24-h urine sodium. The estimation of sodium intake using the new equation was compared with other established equations, namely Tanaka and INTERSALT. Results The new equation showed the least mean bias between measured and predicted sodium, − 0.35 (− 72.26, 71.56) mg/day compared to Tanaka, 629.83 (532.19, 727.47) mg/day and INTERSALT, and 360.82 (284.34, 437.29) mg/day. Predicted sodium measured from the new equation showed greater correlation with measured sodium (r = 0.50) compared to Tanaka (r =0.24) and INTERSALT (r = 0.44), P < 0.05. Conclusion Our newly developed equation from spot urine can predict least mean bias of sodium intake among the Malaysian population when 24-h urine sodium collection is not feasible.

scholarly journals Seasonal Patterns of Spore Dispersal of Phaeosphaeria spp. and Stagonospora spp.

Plant Disease ◽  
1998 ◽  
Vol 82 (2) ◽  
pp. 187-194 ◽  
Author(s):  
E. Arseniuk ◽  
T. Góral ◽  
A. L. Scharen
Keyword(s):  
Multiple Regression ◽  
Air Temperature ◽  
Regression Models ◽  
Cross Validation ◽  
Weather Data ◽  
Time Interval ◽  
Weather Factors ◽  
Airborne Spores ◽  
Data Set ◽  
Microscope Slides

The spatial and temporal patterns of discharge and dissemination of airborne spores of Phaeosphaeria spp. and Stagonospora spp. were studied. Both ascospores and pycnidiospores of the pathogens were deposited at various densities on microscope slides used as spore samplers. The maximum deposition of the spores was observed during the period of August to October. A multiple regression analysis was used to determine which weather factors significantly explained the variation measured in the numbers of ascospores that settled on microscope slides. Rainfall, air temperature, and relative air humidity were influential in the release of Phaeosphaeria spp. ascospores into the air. The amount of airborne ascospores was a function of the variables and remained largely under their control. The liberation of ascospores was favored by air temperature above 0°C, rainfall greater than 1 mm, and high relative humidity. The range of atmospheric conditions stimulating air dispersal of ascospores was wider than that for pycnidiospores. Pycnidiospores were sampled only during rainy days. Their release was affected adversely by air temperature below 5°C. Multiple regression models based on weather data were developed and verified for their predictive ability and accuracy by jackknife and cross-validation procedures, as well as by comparisons of observed and predicted mean numbers of deposited ascospores per microscope slide after a substitution of each period data set with a set of data of the other respective time interval. The numbers of airborne ascospores predicted by the regression models were in a good agreement with the observed values. The jackknife and cross-validation techniques allowed use of the limited data sets for both the parameter estimation and validation processes in a development of simulation models. The airborne inoculum appeared to be omnipresent over cereal areas year round, except during periods with freezing temperatures and a snow cover. Such an omnipresence of inoculum of the pathogens poses a danger to crops and could be of importance in the epidemiology of Stagonospora (= Septoria) blotches under the climatic conditions of central Poland.

scholarly journals Tumor Response Prediction in 90Y Radioembolization with PET-based Radiomics Features and Absorbed Dose Metrics

2020 ◽  
Author(s):  
Lise Wei ◽  
Can Cui ◽  
Jiarui Xu ◽  
Ravi Kaza ◽  
Issam El Naqa ◽  
...  
Keyword(s):  
Cross Validation ◽  
Cox Regression ◽  
Statistical Significance ◽  
Absorbed Dose ◽  
Phantom Study ◽  
Operating Characteristics ◽  
Data Set ◽  
Progression Model ◽  
Dose Metrics ◽  
Combined Models

Abstract Purpose To evaluate whether lesion radiomics features and absorbed dose metrics extracted from post-therapy 90Y PET can be integrated to better predict outcomes in microsphere radioembolization of liver malignancies. Methods Given the noisy nature of 90Y PET, first, a liver phantom study with repeated acquisitions and varying reconstruction parameters was used to identify a subset of robust radiomics features for the patient analysis. In 36 radioembolization procedures, 90Y PET/CT was performed within a couple of hours to extract 46 radiomics features and estimate absorbed dose in 105 primary and metastatic liver lesions. Robust radiomics modeling was based on bootstrapped multivariate logistic regression with shrinkage regularization (LASSO) and Cox regression with LASSO. Nested cross-validation and bootstrap resampling were used for optimal parameter/feature selection and for guarding against overfitting risks. Spearman rank correlation was used to analyze feature associations. Area under the receiver-operating characteristics curve (AUC) was used for lesion response (at first follow-up) analysis while Kaplan-Meier plots and c-index were used to assess progression model performance. Models with absorbed dose only, radiomics only and combined models were developed to predict lesion outcome.Results The phantom study identified 15/46 reproducible and robust radiomics features that were subsequently used in the patient models. A lesion response model with zone percentage (ZP) and mean absorbed dose achieved an AUC of 0.729 (95%CI: 0.702-0.758) and a progression model with zone size nonuniformity (ZSN) and absorbed dose achieved a c-index of 0.803 (95% CI: 0.790-0.815) on nested cross validation (CV). Although the combined models outperformed the radiomics only and absorbed dose only models, statistical significance was not achieved with the current limited data set to establish expected superiority.Conclusion We have developed new lesion-level response and progression models using textural radiomics features, derived from 90Y PET combined with mean absorbed dose for predicting outcome in radioembolization. These encouraging, but limited results, will need further validation in independent and larger datasets prior to any clinical adoption.

scholarly journals Twenty steps towards an adequate inferential interpretation of p-values

2018 ◽  
Author(s):  
Norbert Hirschauer ◽  
Sven Grüner ◽  
Oliver Mußhoff ◽  
Claudia Becker
Keyword(s):  
Regression Analysis ◽  
Bayesian Statistics ◽  
Multiple Regression Analysis ◽  
Multiple Regression ◽  
Statistical Significance ◽  
Significance Testing ◽  
Statistical Significance Testing ◽  
P Values ◽  
Logical Consistency ◽  
Categorical Reasoning

We suggest twenty immediately actionable steps to reduce widespread inferential errors related to “statistical significance testing.” Our propositions refer first to the theoretical preconditions for using p-values. They furthermore include wording guidelines as well as structural and operative advice of how to present results, especially in multiple regression analysis. Our propositions aim at fostering the logical consistency of inferential arguments by avoiding false categorical reasoning. They are not aimed at dispensing with p-values or completely replacing frequentist approaches by Bayesian statistics.

scholarly journals Twenty steps towards an adequate inferential interpretation of p-values

2017 ◽  
Author(s):  
Norbert Hirschauer ◽  
Oliver Mußhoff ◽  
Claudia Becker ◽  
Sven Grüner
Keyword(s):  
Regression Analysis ◽  
Bayesian Statistics ◽  
Multiple Regression Analysis ◽  
Multiple Regression ◽  
Statistical Significance ◽  
Significance Testing ◽  
Statistical Significance Testing ◽  
P Values ◽  
Logical Consistency ◽  
Categorical Reasoning

We suggest twenty immediately actionable steps to reduce widespread inferential errors related to “statistical significance testing.” Our propositions refer first to the theoretical preconditions for using p-values. They furthermore include wording guidelines as well as structural and operative advice of how to present results, especially in multiple regression analysis. Our propositions aim at fostering the logical consistency of inferential arguments by avoiding false categorical reasoning. They are not aimed at dispensing with p-values or completely replacing frequentist approaches by Bayesian statistics.

scholarly journals Tumor Response Prediction in 90Y Radioembolization with PET-based Radiomics Features and Absorbed Dose Metrics

2020 ◽  
Author(s):  
Lise Wei ◽  
Can Cui ◽  
Jiarui Xu ◽  
Ravi Kaza ◽  
Issam El Naqa ◽  
...  
Keyword(s):  
Cross Validation ◽  
Cox Regression ◽  
Statistical Significance ◽  
Absorbed Dose ◽  
Phantom Study ◽  
Operating Characteristics ◽  
Data Set ◽  
Progression Model ◽  
Dose Metrics ◽  
Combined Models

Abstract Purpose Evaluate whether lesion radiomics features and absorbed dose metrics extracted from post-therapy 90Y PET can be integrated to better predict outcome in microsphere radioembolization of liver malignancies. Methods Given the noisy nature of 90Y PET, first, a liver phantom study with repeated acquisitions and varying reconstruction parameters was used to identify a subset of robust radiomics features for the patient analysis. In 36 radioembolization procedures, 90Y PET/CT was performed within a couple of hours to extract 46 radiomics features and estimate absorbed dose in 105 primary and metastatic liver lesions. Robust radiomics modeling was based on bootstrapped multivariate logistic regression with shrinkage regularization (LASSO) and Cox regression with LASSO. Nested cross-validation and bootstrap resampling were used for optimal parameter/feature selection and for guarding against overfitting risks. Spearman rank correlation was used to analyze feature associations. Area under the receiver-operating characteristics curve (AUC) was used for lesion response (at first follow-up) analysis while Kaplan-Meier plots and c-index were used to assess progression model performance. Models with absorbed dose only, radiomics only and combined models were developed to predict lesion outcome.Results The phantom study identified 15/46 reproducible and robust radiomics features that were subsequently used in the patient models. A lesion response model with zone percentage (ZP) and mean absorbed dose achieved an AUC of 0.729 (95%CI: 0.702-0.758) and a progression model with zone size nonuniformity (ZSN) and absorbed dose achieved a c-index of 0.803 (95% CI: 0.790-0.815) on nested cross validation (CV). The combined models outperformed the radiomics only and absorbed dose only models, but statistical significance was not achieved with this limited data set.Conclusion We have developed new lesion-level response and progression models using textural radiomics features, derived from 90Y PET combined with mean absorbed dose for predicting outcome in radioembolization. These encouraging results will need further validation in independent datasets prior to clinical adoption.

Twenty Steps Towards an Adequate Inferential Interpretation of p-Values in Econometrics

2019 ◽  
Vol 239 (4) ◽  
pp. 703-721 ◽  
Author(s):  
Norbert Hirschauer ◽  
Sven Grüner ◽  
Oliver Mußhoff ◽  
Claudia Becker
Keyword(s):  
Regression Analysis ◽  
Bayesian Statistics ◽  
Multiple Regression Analysis ◽  
Multiple Regression ◽  
Statistical Significance ◽  
Significance Testing ◽  
Statistical Significance Testing ◽  
P Values ◽  
Logical Consistency ◽  
Categorical Reasoning

Abstract We suggest twenty immediately actionable steps to reduce widespread inferential errors related to “statistical significance testing.” Our propositions refer to the theoretical preconditions for using p-values. They furthermore include wording guidelines as well as structural and operative advice on how to present results, especially in research based on multiple regression analysis, the working horse of empirical economists. Our propositions aim at fostering the logical consistency of inferential arguments by avoiding false categorical reasoning. They are not aimed at dispensing with p-values or completely replacing frequentist approaches by Bayesian statistics.

scholarly journals Twenty steps towards an adequate inferential interpretation of p-values

2018 ◽  
Author(s):  
Norbert Hirschauer ◽  
Sven Grüner ◽  
Oliver Mußhoff ◽  
Claudia Becker
Keyword(s):  
Regression Analysis ◽  
Bayesian Statistics ◽  
Multiple Regression Analysis ◽  
Multiple Regression ◽  
Statistical Significance ◽  
Significance Testing ◽  
Statistical Significance Testing ◽  
P Values ◽  
Logical Consistency ◽  
Categorical Reasoning

We suggest twenty immediately actionable steps to reduce widespread inferential errors related to “statistical significance testing.” Our propositions refer first to the theoretical preconditions for using p-values. They furthermore include wording guidelines as well as structural and operative advice of how to present results, especially in multiple regression analysis. Our propositions aim at fostering the logical consistency of inferential arguments by avoiding false categorical reasoning. They are not aimed at dispensing with p-values or completely replacing frequentist approaches by Bayesian statistics.

scholarly journals Twenty steps towards an adequate inferential interpretation of p-values

2019 ◽  
Author(s):  
Norbert Hirschauer ◽  
Sven Grüner ◽  
Oliver Mußhoff ◽  
Claudia Becker
Keyword(s):  
Regression Analysis ◽  
Bayesian Statistics ◽  
Multiple Regression Analysis ◽  
Multiple Regression ◽  
Statistical Significance ◽  
Significance Testing ◽  
Statistical Significance Testing ◽  
P Values ◽  
Logical Consistency ◽  
Categorical Reasoning

We suggest twenty immediately actionable steps to reduce widespread inferential errors related to “statistical significance testing.” Our propositions refer first to the theoretical preconditions for using p-values. They furthermore include wording guidelines as well as structural and operative advice on how to present results, especially in research based on multiple regression analysis, the working horse of empirical economists. Our propositions aim at fostering the logical consistency of inferential arguments by avoiding false categorical reasoning. They are not aimed at dispensing with p-values or completely replacing frequentist approaches by Bayesian statistics.

scholarly journals Tumor response prediction in 90Y radioembolization with PET-based radiomics features and absorbed dose metrics

2020 ◽  
Vol 7 (1) ◽  
Author(s):  
Lise Wei ◽  
Can Cui ◽  
Jiarui Xu ◽  
Ravi Kaza ◽  
Issam El Naqa ◽  
...  
Keyword(s):  
Cross Validation ◽  
Cox Regression ◽  
Statistical Significance ◽  
Absorbed Dose ◽  
Phantom Study ◽  
Operating Characteristics ◽  
Data Set ◽  
Progression Model ◽  
Dose Metrics ◽  
Combined Models

Abstract Purpose To evaluate whether lesion radiomics features and absorbed dose metrics extracted from post-therapy 90Y PET can be integrated to better predict outcomes in microsphere radioembolization of liver malignancies Methods Given the noisy nature of 90Y PET, first, a liver phantom study with repeated acquisitions and varying reconstruction parameters was used to identify a subset of robust radiomics features for the patient analysis. In 36 radioembolization procedures, 90Y PET/CT was performed within a couple of hours to extract 46 radiomics features and estimate absorbed dose in 105 primary and metastatic liver lesions. Robust radiomics modeling was based on bootstrapped multivariate logistic regression with shrinkage regularization (LASSO) and Cox regression with LASSO. Nested cross-validation and bootstrap resampling were used for optimal parameter/feature selection and for guarding against overfitting risks. Spearman rank correlation was used to analyze feature associations. Area under the receiver-operating characteristics curve (AUC) was used for lesion response (at first follow-up) analysis while Kaplan-Meier plots and c-index were used to assess progression model performance. Models with absorbed dose only, radiomics only, and combined models were developed to predict lesion outcome. Results The phantom study identified 15/46 reproducible and robust radiomics features that were subsequently used in the patient models. A lesion response model with zone percentage (ZP) and mean absorbed dose achieved an AUC of 0.729 (95% CI 0.702–0.758), and a progression model with zone size nonuniformity (ZSN) and absorbed dose achieved a c-index of 0.803 (95% CI 0.790–0.815) on nested cross-validation (CV). Although the combined models outperformed the radiomics only and absorbed dose only models, statistical significance was not achieved with the current limited data set to establish expected superiority. Conclusion We have developed new lesion-level response and progression models using textural radiomics features, derived from 90Y PET combined with mean absorbed dose for predicting outcome in radioembolization. These encouraging, but limited results, will need further validation in independent and larger datasets prior to any clinical adoption.

Sign in / Sign up

Export Citation Format

Share Document