Comparison of Weighted Lag Adaptive LASSO with Autometrics for Covariate Selection and Forecasting Using Time-Series Data

In order to reduce the dimensionality of parameter space and enhance out-of-sample forecasting performance, this research compares regularization techniques with Autometrics in time-series modeling. We mainly focus on comparing weighted lag adaptive LASSO (WLAdaLASSO) with Autometrics, but as a benchmark, we estimate other popular regularization methods LASSO, AdaLASSO, SCAD, and MCP. For analytical comparison, we implement Monte Carlo simulation and assess the performance of these techniques in terms of out-of-sample Root Mean Square Error, Gauge, and Potency. The comparison is assessed with varying autocorrelation coefficients and sample sizes. The simulation experiment indicates that, compared to Autometrics and other regularization approaches, the WLAdaLASSO outperforms the others in covariate selection and forecasting, especially when there is a greater linear dependency between predictors. In contrast, the computational efficiency of Autometrics decreases with a strong linear dependency between predictors. However, under the large sample and weak linear dependency between predictors, the Autometrics potency ⟶ 1 and gauge ⟶ α. In contrast, LASSO, AdaLASSO, SCAD, and MCP select more covariates and possess higher RMSE than Autometrics and WLAdaLASSO. To compare the considered techniques, we made the Generalized Unidentified Model for covariate selection and out-of-sample forecasting for the trade balance of Pakistan. We train the model on 1985–2015 observations and 2016–2020 observations as test data for the out-of-sample forecast.

Flexible, non-parametric modeling using regularized neural networks

Non-parametric, additive models are able to capture complex data dependencies in a flexible, yet interpretable way. However, choosing the format of the additive components often requires non-trivial data exploration. Here, as an alternative, we propose PrAda-net, a one-hidden-layer neural network, trained with proximal gradient descent and adaptive lasso. PrAda-net automatically adjusts the size and architecture of the neural network to reflect the complexity and structure of the data. The compact network obtained by PrAda-net can be translated to additive model components, making it suitable for non-parametric statistical modelling with automatic model selection. We demonstrate PrAda-net on simulated data, where we compare the test error performance, variable importance and variable subset identification properties of PrAda-net to other lasso-based regularization approaches for neural networks. We also apply PrAda-net to the massive U.K. black smoke data set, to demonstrate how PrAda-net can be used to model complex and heterogeneous data with spatial and temporal components. In contrast to classical, statistical non-parametric approaches, PrAda-net requires no preliminary modeling to select the functional forms of the additive components, yet still results in an interpretable model representation.

Doubly robust adaptive LASSO for effect modifier discovery

Effect modification occurs when the effect of a treatment on an outcome differsaccording to the level of some pre-treatment variable (the effect modifier). Assessing an effect modifier is not a straight-forward task even for a subject matter expert. In this paper, we propose a two-stageprocedure to automatically selecteffect modifying variables in a Marginal Structural Model (MSM) with a single time point exposure based on the two nuisance quantities (the conditionaloutcome expectation and propensity score). We highlight the performance of our proposal in a simulation study. Finally, to illustrate tractability of our proposed methods, we apply them to analyze a set of pregnancy data. We estimate the conditional expected difference in the counterfactual birth weight if all women were exposed to inhaled corticosteroids during pregnancy versus the counterfactual birthweight if all women were not, using data from asthma medications during pregnancy.

New adaptive lasso approaches for variable selection in automated pharmacovigilance signal detection

Background Adverse effects of drugs are often identified after market introduction. Post-marketing pharmacovigilance aims to detect them as early as possible and relies on spontaneous reporting systems collecting suspicious cases. Signal detection tools have been developed to mine these large databases and counts of reports are analysed with disproportionality methods. To address disproportionality method biases, recent methods apply to individual observations taking into account all exposures for the same patient. In particular, the logistic lasso provides an efficient variable selection framework, yet the choice of the regularization parameter is a challenging issue and the lasso variable selection may give inconsistent results. Methods We propose a new signal detection methodology based on the adaptive lasso. We derived two new adaptive weights from (i) a lasso regression using the Bayesian Information Criterion (BIC), and (ii) the class-imbalanced subsampling lasso (CISL), an extension of stability selection. The BIC is used in the adaptive lasso stage for variable selection. We performed an extensive simulation study and an application to real data, where we compared our methods to the existing adaptive lasso, and recent detection approaches based on lasso regression or propensity scores in high dimension. For both studies, we evaluate the methods in terms of false discoveries and sensitivity. Results In the simulations and the application, both proposed adaptive weights show equivalent or better performances than the other competitors, with an advantage for the CISL-based adaptive weights. CISL and lasso regression using BIC are solid alternatives. Conclusion Our proposed adaptive lasso is an appealing methodology for signal detection in pharmacovigilance. Although we cannot rely on test theory, our approaches show a low and stable False Discovery Rate in all simulation settings. All methods evaluated in this work are implemented in the adapt4pv R package.

An adaptive shortest-solution guided decimation approach to sparse high-dimensional linear regression

High-dimensional linear regression model is the most popular statistical model for high-dimensional data, but it is quite a challenging task to achieve a sparse set of regression coefficients. In this paper, we propose a simple heuristic algorithm to construct sparse high-dimensional linear regression models, which is adapted from the shortest-solution guided decimation algorithm and is referred to as ASSD. This algorithm constructs the support of regression coefficients under the guidance of the shortest least-squares solution of the recursively decimated linear models, and it applies an early-stopping criterion and a second-stage thresholding procedure to refine this support. Our extensive numerical results demonstrate that ASSD outperforms LASSO, adaptive LASSO, vector approximate message passing, and two other representative greedy algorithms in solution accuracy and robustness. ASSD is especially suitable for linear regression problems with highly correlated measurement matrices encountered in real-world applications.

Introducing LASSO-type penalisation to generalised joint regression modelling for count data

In this work, we propose an extension of the versatile joint regression framework for bivariate count responses of the package by Marra and Radice (R package version 0.2-3, 2020) by incorporating an (adaptive) LASSO-type penalty. The underlying estimation algorithm is based on a quadratic approximation of the penalty. The method enables variable selection and the corresponding estimates guarantee shrinkage and sparsity. Hence, this approach is particularly useful in high-dimensional count response settings. The proposal’s empirical performance is investigated in a simulation study and an application on FIFA World Cup football data.

Complete Blood Count Parameters Outperform Putative Inflammatory Markers in Predicting COVID-19 Mortality: A Multimodal Machine Learning Model

Introduction: SARS-CoV-2 evoked immunodysregulation drives inflammation, morbidity, and mortality across COVID-19 presentation spectrum. We sought to identify baseline cell counts and proportions reported with a complete blood count (CBC) that contribute independent information to a model predicting mortality in hospitalized patients with laboratory confirmed SARS-CoV-2 infection. Such a model may complement or improve presentation risk stratification informed by putative inflammatory markers. Methods: Our retrospective design, analyses and interpretations followed constructs detailed in the Strengthening the Reporting of Observational Studies in Epidemiology reporting guideline. Under IRB exemption, discharge medical electronic health records underwent extraction of administrative and clinical data. Demographics, anthropometrics, vital signs, laboratory test and ICD-10-CM-based Elixhauser comorbidity categories were included. Univariate logistic regression was used to identify CBC parameters and attendant ratios associated (p<.05) with hospital mortality. Generalized regression with adaptive LASSO modeling was used to evaluate explanatory probability while eliminating collinearities in identified CBC parameters (individual and ratio) associated with mortality while controlling age, sex, race, baseline vital signs, Elixhauser comorbidities and COVID-19 epoch quarters / treatment. Additional analysis with Bootstrap Forest (BF) was employed to evaluate aggregated synergies and retain parameters that optimized generalized RSquared representing multivariate prediction accuracy and explained variance proportion (EV%) in mortality provided by each variable. Further BF analysis was used to examine relative magnitude of EV% versus putative COVID-19 inflammatory markers. CBC variables included in final BF model were temporally parsed in 24h intervals then pooled when measured after 120h since first vital sign at hospitalization. Results were averaged when a patient underwent multiple assays within an interval. A two-way ANOVA was employed to compare survival vs. non-survival pathways. Results: Among patients consecutively discharged between March 14, 2020 through May 31, 2021, 208 (10 %) of 2153 died. Survivor vs. non-survivor patient and clinical characteristics are summarized in Table 1. CBC parameters identified as independently associated with hospital mortality included WBC, lymphocytes, bands, segmented neutrophils, monocytes, and RDW-CV. (Table 2) Ratios of CBC parameters associated with mortality included AMC/ALC and APC/ALC (Table 2). Results of BF EF% modeling including CBC parameters respectively without (Rsquare = 0.65) and with (Rsquare = 0.70) inclusion of putative inflammatory markers are illustrated in Figure 1a and 1b. Inflammatory markers alone exhibited lowest Rsquare (0.52) (Figure 1c). Figure 2 illustrates temporal kinetics of modeled CBC parameters across hospitalization. Intergroup differences at baseline were sustained, save for RDW-CV after 5-days. Conclusions: Machine learning approaches identified several CBC parameters measured at presentation that when modeled with putative COVID-19 inflammatory markers, enhanced early prediction of hospital mortality. CBC parameters are usually more often measured compared to other inflammatory markers that show COVID-19 severity and serve as an easily obtainable source of information to determine which patients may require a higher level of care before clinical symptoms follow. This includes progression to critical illness and hospital mortality. We recommend that CBC parameters, especially bands, APC/ALC ratio and AMC/ALC ratio be considered for baseline risk stratification of COVID-19 severity, as these trends are sustained at least 5-days after hospitalization. Figure 1 Figure 1. Disclosures No relevant conflicts of interest to declare.

Incorporating biological networks into high-dimensional Bayesian survival analysis using an ICM/M algorithm

The Cox proportional hazards model has been widely used in cancer genomic research that aims to identify genes from high-dimensional gene expression space associated with the survival time of patients. With the increase in expertly curated biological pathways, it is challenging to incorporate such complex networks in fitting a high-dimensional Cox model. This paper considers a Bayesian framework that employs the Ising prior to capturing relations among genes represented by graphs. A spike-and-slab prior is also assigned to each of the coefficients for the purpose of variable selection. The iterated conditional modes/medians (ICM/M) algorithm is proposed for the implementation for Cox models. The ICM/M estimates hyperparameters using conditional modes and obtains coefficients through conditional medians. This procedure produces some coefficients that are exactly zero, making the model more interpretable. Comparisons of the ICM/M and other regularized Cox models were carried out with both simulated and real data. Compared to lasso, adaptive lasso, elastic net, and DegreeCox, the ICM/M yielded more parsimonious models with consistent variable selection. The ICM/M model also provided a smaller number of false positives than the other methods and showed promising results in terms of predictive accuracy. In terms of computing times among the network-aware methods, the ICM/M algorithm is substantially faster than DegreeCox even when incorporating a large complex network. The implementation of the ICM/M algorithm for Cox regression model is provided in R package icmm, available on the Comprehensive R Archive Network (CRAN).

Likelihood theory for the graph Ornstein-Uhlenbeck process

We consider the problem of modelling restricted interactions between continuously-observed time series as given by a known static graph (or network) structure. For this purpose, we define a parametric multivariate Graph Ornstein-Uhlenbeck (GrOU) process driven by a general Lévy process to study the momentum and network effects amongst nodes, effects that quantify the impact of a node on itself and that of its neighbours, respectively. We derive the maximum likelihood estimators (MLEs) and their usual properties (existence, uniqueness and efficiency) along with their asymptotic normality and consistency. Additionally, an Adaptive Lasso approach, or a penalised likelihood scheme, infers both the graph structure along with the GrOU parameters concurrently and is shown to satisfy similar properties. Finally, we show that the asymptotic theory extends to the case when stochastic volatility modulation of the driving Lévy process is considered.