scholarly journals "Bring More Data!" – A Good Advice? Removing Separation in Logistic Regression by Increasing Sample Size

2019 ◽  
Author(s):  
Hana Šinkovec ◽  
Angelika Geroldinger ◽  
Georg Heinze
Keyword(s):  
Logistic Regression ◽  
Sample Size ◽  
Original Data ◽  
Parameter Estimates ◽  
Small Data ◽  
Data Sets ◽  
Data Set ◽  
Logistic Regression Models ◽  
Good Advice ◽  
The Cost

The parameters of logistic regression models are usually obtained by the method of maximum likelihood (ML). However, in analyses of small data sets or data sets with unbalanced outcomes or exposures, ML parameter estimates may not exist. This situation has been termed “separation” as the two outcome groups are separated by the values of a covariate or a linear combination of covariates. To overcome the problem of non-existing ML parameter estimates, applying Firth’s correction (FC) was proposed. In practice, however, a principal investigator might be advised to “bring more data” in order to solve a separation issue. We illustrate the problem by means of an examples from colorectal cancer screening and ornithology. It is unclear if such an increasing sample size (ISS) strategy that keeps sampling new observations until separation is removed improves estimation compared to applying FC to the original data set. We performed an extensive simulation study where the main focus was to estimate the cost-adjusted relative efficiency of ML combined with ISS compared to FC. FC yielded reasonably small root mean squared errors and proved to be the more efficient estimator. Given our findings, we propose not to adapt the sample size when separation is encountered but to use FC as the default method of analysis whenever the number of observations or outcome events is critically low.

scholarly journals Bring More Data!—A Good Advice? Removing Separation in Logistic Regression by Increasing Sample Size

2019 ◽  
Vol 16 (23) ◽  
pp. 4658 ◽  
Author(s):  
Šinkovec ◽  
Geroldinger ◽  
Heinze
Keyword(s):  
Logistic Regression ◽  
Sample Size ◽  
Original Data ◽  
Parameter Estimates ◽  
Small Data ◽  
Data Sets ◽  
Data Set ◽  
Logistic Regression Models ◽  
Good Advice ◽  
The Cost

The parameters of logistic regression models are usually obtained by the method of maximum likelihood (ML). However, in analyses of small data sets or data sets with unbalanced outcomes or exposures, ML parameter estimates may not exist. This situation has been termed ‘separation’ as the two outcome groups are separated by the values of a covariate or a linear combination of covariates. To overcome the problem of non-existing ML parameter estimates, applying Firth’s correction (FC) was proposed. In practice, however, a principal investigator might be advised to ‘bring more data’ in order to solve a separation issue. We illustrate the problem by means of examples from colorectal cancer screening and ornithology. It is unclear if such an increasing sample size (ISS) strategy that keeps sampling new observations until separation is removed improves estimation compared to applying FC to the original data set. We performed an extensive simulation study where the main focus was to estimate the cost-adjusted relative efficiency of ML combined with ISS compared to FC. FC yielded reasonably small root mean squared errors and proved to be the more efficient estimator. Given our findings, we propose not to adapt the sample size when separation is encountered but to use FC as the default method of analysis whenever the number of observations or outcome events is critically low.

scholarly journals Upstrap for estimating power and sample size in complex models

2021 ◽  
Author(s):  
Marta Karas ◽  
Ciprian Crainiceanu
Keyword(s):  
Sample Size ◽  
Effect Size ◽  
Sample Size Calculation ◽  
Cluster Randomized Trial ◽  
Original Data ◽  
Binary Outcomes ◽  
Data Sets ◽  
Size Estimation ◽  
Data Set ◽  
Cluster Randomized

Power and sample size calculation are major components of statistical analyses. The upstrap resampling method introduced by Crainiceanu and Crainiceanu (2018) was proposed as a general solution to this problem but has not been assessed in numerical experiments. We evaluate the power and sample size estimation properties of the upstrap for data sets that are larger or smaller than the original data set. We also expand the scope of upstrap and propose a solution to estimate the power to detect: (1) an effect size observed in the data; and (2) an effect size chosen by a researcher. Simulations include the following scenarios: (a) one- and two-sample t-tests; (b) linear regression with both Gaussian and binary outcomes; (c) multilevel mixed effects models with both Gaussian and binary outcomes. We illustrate the approach using a reanalysis of a cluster randomized trial of malaria transmission. The accompanying software and data are publicly available.

scholarly journals Synthesising artificial patient-level data for Open Science - an evaluation of five methods

2020 ◽  
Author(s):  
Michael Allen ◽  
Andrew Salmon
Keyword(s):  
Breast Cancer ◽  
Logistic Regression ◽  
Synthetic Data ◽  
Original Data ◽  
Classification Model ◽  
Data Sets ◽  
List Type ◽  
Breast Cancer Data ◽  
Data Set ◽  
Cancer Data

ABSTRACTBackgroundOpen science is a movement seeking to make scientific research accessible to all, including publication of code and data. Publishing patient-level data may, however, compromise the confidentiality of that data if there is any significant risk that data may later be associated with individuals. Use of synthetic data offers the potential to be able to release data that may be used to evaluate methods or perform preliminary research without risk to patient confidentiality.MethodsWe have tested five synthetic data methods:A technique based on Principal Component Analysis (PCA) which samples data from distributions derived from the transformed data.Synthetic Minority Oversampling Technique, SMOTE which is based on interpolation between near neighbours.Generative Adversarial Network, GAN, an artificial neural network approach with competing networks - a discriminator network trained to distinguish between synthetic and real data., and a generator network trained to produce data that can fool the discriminator network.CT-GAN, a refinement of GANs specifically for the production of structured tabular synthetic data.Variational Auto Encoders, VAE, a method of encoding data in a reduced number of dimensions, and sampling from distributions based on the encoded dimensions.Two data sets are used to evaluate the methods:The Wisconsin Breast Cancer data set, a histology data set where all features are continuous variables.A stroke thrombolysis pathway data set, a data set describing characteristics for patients where a decision is made whether to treat with clot-busting medication. Features are mostly categorical, binary, or integers.Methods are evaluated in three ways:The ability of synthetic data to train a logistic regression classification model.A comparison of means and standard deviations between original and synthetic data.A comparison of covariance between features in the original and synthetic data.ResultsUsing the Wisconsin Breast Cancer data set, the original data gave 98% accuracy in a logistic regression classification model. Synthetic data sets gave between 93% and 99% accuracy. Performance (best to worst) was SMOTE > PCA > GAN > CT-GAN = VAE. All methods produced a high accuracy in reproducing original data means and stabdard deviations (all R-square > 0.96 for all methods and data classes). CT-GAN and VAE suffered a significant loss of covariance between features in the synthetic data sets.Using the Stroke Pathway data set, the original data gave 82% accuracy in a logistic regression classification model. Synthetic data sets gave between 66% and 82% accuracy. Performance (best to worst) was SMOTE > PCA > CT-GAN > GAN > VAE. CT-GAN and VAE suffered loss of covariance between features in the synthetic data sets, though less pronounced than with the Wisconsin Breast Cancer data set.ConclusionsThe pilot work described here shows, as proof of concept, that synthetic data may be produced, which is of sufficient quality to publish with open methodology, to allow people to better understand and test methodology. The quality of the synthetic data also gives promise of data sets that may be used for screening of ideas, or for research project (perhaps especially in an education setting).More work is required to further refine and test methods across a broader range of patient-level data sets.

scholarly journals Using self-supervised feature learning to improve the use of pulse oximeter signals to predict paediatric hospitalization

2021 ◽  
Vol 6 ◽  
pp. 248
Author(s):  
Paul Mwaniki ◽  
Timothy Kamanu ◽  
Samuel Akech ◽  
Dustin Dunsmuir ◽  
J. Mark Ansermino ◽  
...  
Keyword(s):  
Machine Learning ◽  
Logistic Regression ◽  
Deep Learning ◽  
Pulse Oximeter ◽  
Regression Models ◽  
Data Sets ◽  
Learning Models ◽  
Data Set ◽  
Logistic Regression Models ◽  
End To End

Background: The success of many machine learning applications depends on knowledge about the relationship between the input data and the task of interest (output), hindering the application of machine learning to novel tasks. End-to-end deep learning, which does not require intermediate feature engineering, has been recommended to overcome this challenge but end-to-end deep learning models require large labelled training data sets often unavailable in many medical applications. In this study, we trained machine learning models to predict paediatric hospitalization given raw photoplethysmography (PPG) signals obtained from a pulse oximeter. We trained self-supervised learning (SSL) for automatic feature extraction from PPG signals and assessed the utility of SSL in initializing end-to-end deep learning models trained on a small labelled data set with the aim of predicting paediatric hospitalization.Methods: We compared logistic regression models fitted using features extracted using SSL with end-to-end deep learning models initialized either randomly or using weights from the SSL model. We also compared the performance of SSL models trained on labelled data alone (n=1,031) with SSL trained using both labelled and unlabelled signals (n=7,578). Results: The SSL model trained on both labelled and unlabelled PPG signals produced features that were more predictive of hospitalization compared to the SSL model trained on labelled PPG only (AUC of logistic regression model: 0.78 vs 0.74). The end-to-end deep learning model had an AUC of 0.80 when initialized using the SSL model trained on all PPG signals, 0.77 when initialized using SSL trained on labelled data only, and 0.73 when initialized randomly. Conclusions: This study shows that SSL can improve the classification of PPG signals by either extracting features required by logistic regression models or initializing end-to-end deep learning models. Furthermore, SSL can leverage larger unlabelled data sets to improve performance of models fitted using small labelled data sets.

PSVI-8 Meta-regression Analysis to Determine the Relationship Between Growing Pig Body Weight and Variation

2021 ◽  
Vol 99 (Supplement_1) ◽  
pp. 218-219
Author(s):  
Andres Fernando T Russi ◽  
Mike D Tokach ◽  
Jason C Woodworth ◽  
Joel M DeRouchey ◽  
Robert D Goodband ◽  
...  
Keyword(s):  
Body Weight ◽  
Regression Analysis ◽  
Sample Size ◽  
Polynomial Regression ◽  
Data Sets ◽  
Regression Equations ◽  
Prediction Equations ◽  
Data Set ◽  
Rate Of Increase ◽  
The Relationship

Abstract The swine industry has been constantly evolving to select animals with improved performance traits and to minimize variation in body weight (BW) in order to meet packer specifications. Therefore, understanding variation presents an opportunity for producers to find strategies that could help reduce, manage, or deal with variation of pigs in a barn. A systematic review and meta-analysis was conducted by collecting data from multiple studies and available data sets in order to develop prediction equations for coefficient of variation (CV) and standard deviation (SD) as a function of BW. Information regarding BW variation from 16 papers was recorded to provide approximately 204 data points. Together, these data included 117,268 individually weighed pigs with a sample size that ranged from 104 to 4,108 pigs. A random-effects model with study used as a random effect was developed. Observations were weighted using sample size as an estimate for precision on the analysis, where larger data sets accounted for increased accuracy in the model. Regression equations were developed using the nlme package of R to determine the relationship between BW and its variation. Polynomial regression analysis was conducted separately for each variation measurement. When CV was reported in the data set, SD was calculated and vice versa. The resulting prediction equations were: CV (%) = 20.04 – 0.135 × (BW) + 0.00043 × (BW)2, R2=0.79; SD = 0.41 + 0.150 × (BW) - 0.00041 × (BW)2, R2 = 0.95. These equations suggest that there is evidence for a decreasing quadratic relationship between mean CV of a population and BW of pigs whereby the rate of decrease is smaller as mean pig BW increases from birth to market. Conversely, the rate of increase of SD of a population of pigs is smaller as mean pig BW increases from birth to market.

scholarly journals Classification of jujube defects in small data sets based on transfer learning

2021 ◽  
Author(s):  
Jianping Ju ◽  
Hong Zheng ◽  
Xiaohang Xu ◽  
Zhongyuan Guo ◽  
Zhaohui Zheng ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Loss Function ◽  
Training Model ◽  
Parameter Distribution ◽  
Test Accuracy ◽  
Small Data ◽  
Data Sets ◽  
Data Set ◽  
Small Data Sets

AbstractAlthough convolutional neural networks have achieved success in the field of image classification, there are still challenges in the field of agricultural product quality sorting such as machine vision-based jujube defects detection. The performance of jujube defect detection mainly depends on the feature extraction and the classifier used. Due to the diversity of the jujube materials and the variability of the testing environment, the traditional method of manually extracting the features often fails to meet the requirements of practical application. In this paper, a jujube sorting model in small data sets based on convolutional neural network and transfer learning is proposed to meet the actual demand of jujube defects detection. Firstly, the original images collected from the actual jujube sorting production line were pre-processed, and the data were augmented to establish a data set of five categories of jujube defects. The original CNN model is then improved by embedding the SE module and using the triplet loss function and the center loss function to replace the softmax loss function. Finally, the depth pre-training model on the ImageNet image data set was used to conduct training on the jujube defects data set, so that the parameters of the pre-training model could fit the parameter distribution of the jujube defects image, and the parameter distribution was transferred to the jujube defects data set to complete the transfer of the model and realize the detection and classification of the jujube defects. The classification results are visualized by heatmap through the analysis of classification accuracy and confusion matrix compared with the comparison models. The experimental results show that the SE-ResNet50-CL model optimizes the fine-grained classification problem of jujube defect recognition, and the test accuracy reaches 94.15%. The model has good stability and high recognition accuracy in complex environments.

A Comparison Study of Mahalanobis-Taguchi System and Neural Network for Multivariate Pattern Recognition

2005 ◽  
Author(s):  
Jungeui Hong ◽  
Elizabeth A. Cudney ◽  
Genichi Taguchi ◽  
Rajesh Jugulum ◽  
Kioumars Paryani ◽  
...  
Keyword(s):  
Neural Network ◽  
Small Data ◽  
Data Sets ◽  
Comparison Study ◽  
Data Set ◽  
Set Size ◽  
Breast Cancer Study ◽  
Discriminant Ability ◽  
Small Data Sets ◽  
Multivariate Pattern

The Mahalanobis-Taguchi System is a diagnosis and predictive method for analyzing patterns in multivariate cases. The goal of this study is to compare the ability of the Mahalanobis-Taguchi System and a neural network to discriminate using small data sets. We examine the discriminant ability as a function of data set size using an application area where reliable data is publicly available. The study uses the Wisconsin Breast Cancer study with nine attributes and one class.

Association Between Opioid Use and Diabetes in Patients With Ankle Fracture Repair

2021 ◽  
pp. 107110072110581
Author(s):  
Wenye Song ◽  
Naohiro Shibuya ◽  
Daniel C. Jupiter
Keyword(s):  
Diabetes Mellitus ◽  
Logistic Regression ◽  
Ankle Fracture ◽  
Regression Models ◽  
Fracture Repair ◽  
Opioid Use ◽  
Data Set ◽  
Opioid Prescription ◽  
Logistic Regression Models ◽  
Patients With Diabetes

Background: Ankle fractures in patients with diabetes mellitus have long been recognized as a challenge to practicing clinicians. Ankle fracture patients with diabetes may experience prolonged healing, higher risk of hardware failure, an increased risk of wound dehiscence and infection, and higher pain scores pre- and postoperatively, compared to patients without diabetes. However, the duration of opioid use among this patient cohort has not been previously evaluated. The purpose of this study is to retrospectively compare the time span of opioid utilization between ankle fracture patients with and without diabetes mellitus. Methods: We conducted a retrospective cohort study using our institution’s TriNetX database. A total of 640 ankle fracture patients were included in the analysis, of whom 73 had diabetes. All dates of opioid use for each patient were extracted from the data set, including the first and last date of opioid prescription. Descriptive analysis and logistic regression models were employed to explore the differences in opioid use between patients with and without diabetes after ankle fracture repair. A 2-tailed P value of .05 was set as the threshold for statistical significance. Results: Logistic regression models revealed that patients with diabetes are less likely to stop using opioids within 90 days, or within 180 days, after repair compared to patients without diabetes. Female sex, neuropathy, and prefracture opioid use are also associated with prolonged opioid use after ankle fracture repair. Conclusion: In our study cohort, ankle fracture patients with diabetes were more likely to require prolonged opioid use after fracture repair. Level of Evidence: Level III, prognostic.

scholarly journals Embedding Undersampling Rotation Forest for Imbalanced Problem

2018 ◽  
Vol 2018 ◽  
pp. 1-15 ◽  
Author(s):  
Huaping Guo ◽  
Xiaoyu Diao ◽  
Hongbing Liu
Keyword(s):  
Imbalanced Data ◽  
Feature Space ◽  
Original Data ◽  
Training Set ◽  
Data Set ◽  
Minority Class ◽  
Rotation Forest ◽  
Novel Method ◽  
Individual Classifier ◽  
The Cost

Rotation Forest is an ensemble learning approach achieving better performance comparing to Bagging and Boosting through building accurate and diverse classifiers using rotated feature space. However, like other conventional classifiers, Rotation Forest does not work well on the imbalanced data which are characterized as having much less examples of one class (minority class) than the other (majority class), and the cost of misclassifying minority class examples is often much more expensive than the contrary cases. This paper proposes a novel method called Embedding Undersampling Rotation Forest (EURF) to handle this problem (1) sampling subsets from the majority class and learning a projection matrix from each subset and (2) obtaining training sets by projecting re-undersampling subsets of the original data set to new spaces defined by the matrices and constructing an individual classifier from each training set. For the first method, undersampling is to force the rotation matrix to better capture the features of the minority class without harming the diversity between individual classifiers. With respect to the second method, the undersampling technique aims to improve the performance of individual classifiers on the minority class. The experimental results show that EURF achieves significantly better performance comparing to other state-of-the-art methods.

scholarly journals Examination of Different Item Response Theory Models on Tests Composed of Testlets

2017 ◽  
Vol 6 (4) ◽  
pp. 113
Author(s):  
Esin Yilmaz Kogar ◽  
Hülya Kelecioglu
Keyword(s):  
Item Response Theory ◽  
Sample Size ◽  
Item Response ◽  
Data Sets ◽  
Meaningful Difference ◽  
Response Theory ◽  
Size Change ◽  
Data Set ◽  
Testlet Response Theory ◽  
Item Response Theory Models

The purpose of this research is to first estimate the item and ability parameters and the standard error values related to those parameters obtained from Unidimensional Item Response Theory (UIRT), bifactor (BIF) and Testlet Response Theory models (TRT) in the tests including testlets, when the number of testlets, number of independent items, and sample size change, and then to compare the obtained results. Mathematic test in PISA 2012 was employed as the data collection tool, and 36 items were used to constitute six different data sets containing different numbers of testlets and independent items. Subsequently, from these constituted data sets, three different sample sizes of 250, 500 and 1000 persons were selected randomly. When the findings of the research were examined, it was determined that, generally the lowest mean error values were those obtained from UIRT, and TRT yielded a mean of error estimation lower than that of BIF. It was found that, under all conditions, models which take into consideration the local dependency have provided a better model-data compatibility than UIRT, generally there is no meaningful difference between BIF and TRT, and both models can be used for those data sets. It can be said that when there is a meaningful difference between those two models, generally BIF yields a better result. In addition, it has been determined that, in each sample size and data set, item and ability parameters and correlations of errors of the parameters are generally high.

Sign in / Sign up

Export Citation Format

Share Document