scholarly journals A Traveler’s Guide to the Multiverse: Promises, Pitfalls, and a Framework for the Evaluation of Analytic Decisions

2021 ◽  
Vol 4 (1) ◽  
pp. 251524592095492
Author(s):  
Marco Del Giudice ◽  
Steven W. Gangestad
Keyword(s):  
Degrees Of Freedom ◽  
A Priori ◽  
Simulated Data ◽  
Style Analysis ◽  
Data Set ◽  
Scant Attention ◽  
Equivalence Type ◽  
The Impact ◽  
Biased Estimates

Decisions made by researchers while analyzing data (e.g., how to measure variables, how to handle outliers) are sometimes arbitrary, without an objective justification for choosing one alternative over another. Multiverse-style methods (e.g., specification curve, vibration of effects) estimate an effect across an entire set of possible specifications to expose the impact of hidden degrees of freedom and/or obtain robust, less biased estimates of the effect of interest. However, if specifications are not truly arbitrary, multiverse-style analyses can produce misleading results, potentially hiding meaningful effects within a mass of poorly justified alternatives. So far, a key question has received scant attention: How does one decide whether alternatives are arbitrary? We offer a framework and conceptual tools for doing so. We discuss three kinds of a priori nonequivalence among alternatives—measurement nonequivalence, effect nonequivalence, and power/precision nonequivalence. The criteria we review lead to three decision scenarios: Type E decisions (principled equivalence), Type N decisions (principled nonequivalence), and Type U decisions (uncertainty). In uncertain scenarios, multiverse-style analysis should be conducted in a deliberately exploratory fashion. The framework is discussed with reference to published examples and illustrated with the help of a simulated data set. Our framework will help researchers reap the benefits of multiverse-style methods while avoiding their pitfalls.

Voting in Open Economies

2007 ◽  
Vol 40 (3) ◽  
pp. 283-306 ◽  
Author(s):  
Timothy Hellwig ◽  
David Samuels
Keyword(s):  
Market Integration ◽  
Open Economies ◽  
Global Market ◽  
Policy Implications ◽  
Set Covering ◽  
Policy Outcomes ◽  
Data Set ◽  
Electoral Accountability ◽  
Scant Attention ◽  
The Impact

What are the electoral consequences of global market integration? Although recent discussions of politics and markets have much to say on globalization’s implications for policy outcomes, the impact of market integration on representative democracy has received scant attention. This article addresses this omission. We extend the globalization literature to develop two competing hypotheses regarding the influence of open economies on electoral accountability. Predictions are tested using a new data set covering elections from 75 countries over 27 years. Results support a government constraint hypothesis: Exposure to the world economy weakens connections between economic performance and support for political incumbents. By redirecting concerns from the policy implications of globalization and toward its electoral consequences, findings highlight the influence of voter perceptions and of vote-seeking politicians in the politics of globalization.

The Impact of Site Sample Size on the Reconstruction of Culture Histories

2011 ◽  
Vol 76 (3) ◽  
pp. 547-572 ◽  
Author(s):  
Charles Perreault
Keyword(s):  
Simulated Data ◽  
Archaeological Sites ◽  
Cultural Tradition ◽  
Data Sets ◽  
Data Set ◽  
Rate Of Spread ◽  
Accuracy And Precision ◽  
Historical Processes ◽  
The Impact ◽  
Simulated Data Sets

I examine how our capacity to produce accurate culture-historical reconstructions changes as more archaeological sites are discovered, dated, and added to a data set. More precisely, I describe, using simulated data sets, how increases in the number of known sites impact the accuracy and precision of our estimations of (1) the earliest and (2) latest date of a cultural tradition, (3) the date and (4) magnitude of its peak popularity, as well as (5) its rate of spread and (6) disappearance in a population. I show that the accuracy and precision of inferences about these six historical processes are not affected in the same fashion by changes in the number of known sites. I also consider the impact of two simple taphonomic site destruction scenarios on the results. Overall, the results presented in this paper indicate that unless we are in possession of near-total samples of sites, and can be certain that there are no taphonomic biases in the universe of sites to be sampled, we will make inferences of varying precision and accuracy depending on the aspect of a cultural trait’s history in question.

scholarly journals Hypoglycemia and Clinical Outcomes in Hospitalized Patients With Diabetes: Does Association With Adverse Outcomes Remain When Number of Glucose Tests Performed Is Accounted For?

2017 ◽  
Vol 11 (4) ◽  
pp. 720-723 ◽  
Author(s):  
Gregory C. Jones ◽  
Joseph G. Timmons ◽  
Scott G. Cunningham ◽  
Stephen J. Cleland ◽  
Christopher A. R. Sainsbury
Keyword(s):  
Length Of Stay ◽  
Simulated Data ◽  
Economic Modeling ◽  
Adverse Outcomes ◽  
Health Board ◽  
Minor Effect ◽  
Data Set ◽  
Sample Number ◽  
Mathematical Association ◽  
The Impact

Background: Hypoglycemia is associated with increased length of stay in hospital patients, but previous studies have not considered the confounding effect of increased hypoglycemia detection associated with increased capillary blood glucose (CBG) measurement in prolonged admissions. We aimed to determine the effect of recorded hypoglycemia on length of stay of hospital inpatients (LOS) when this mathematical association is subtracted. Methods: CBG data were analyzed for inpatients within our health board area (01/2009-01/2015). A simulated CBG data set was generated for each patient with an identical sampling frequency to the measured CBG data set. The mathematical component of increased LOS was determined using the simulated data set. Subtraction of this confounding mathematical association was used to provide measurement of the true clinical association between recorded hypoglycemia (CBG < 4 mmol [< 72mg/dl]) and LOS. Results: A total of 196 962 admissions of 52 475 individuals with known diabetes were analyzed. 68 809 admissions of 29 551 individuals had >4 CBG measurements made and were included in analysis. After subtraction of the mathematical association of increased sample number, the clinical effect of recorded hypoglycemia is reduced—but persists—compared to previous studies. 1-2 days with recorded hypoglycemia has a relatively minor effect on LOS. LOS increases rapidly if there are ≥3 days with recorded hypoglycemia, with an increase of 0.75 days LOS per additional day with hypoglycemia. Conclusions: This technique increases accuracy of economic modeling of the impact of hypoglycemia on health care systems. This could assist study of the impact of hypoglycemia on other outcomes by factoring for bias of increased sample numbers.

scholarly journals Inverse modelling of European CH<sub>4</sub> emissions during 2006–2012 using different inverse models and reassessed atmospheric observations

2017 ◽  
Author(s):  
Peter Bergamaschi ◽  
Ute Karstens ◽  
Alistair J. Manning ◽  
Marielle Saunois ◽  
Aki Tsuruta ◽  
...  
Keyword(s):  
A Priori ◽  
Lower Troposphere ◽  
Inverse Modelling ◽  
Top Down ◽  
Data Set ◽  
Bottom Up ◽  
Ch4 Emissions ◽  
In Situ Data ◽  
Inverse Models ◽  
The Impact

Abstract. We present inverse modelling (top-down) estimates of European methane (CH4) emissions for 2006–2012 based on a new quality-controlled and harmonized in-situ data set from 18 European atmospheric monitoring stations. We applied an ensemble of seven inverse models and performed four inversion experiments, investigating the impact of different sets of stations and the use of a priori information on emissions. The inverse models infer total CH4 emissions of 26.7 (20.2–29.7) Tg CH4 yr−1 (mean, 10th and 90th percentiles from all inversions) for the EU-28 for 2006–2012 from the four inversion experiments. For comparison, total anthropogenic CH4 emissions reported to UNFCCC (bottom-up, based on statistical data and emissions factors) amount to only 21.3 Tg CH4 yr−1 (2006) to 18.8 Tg CH4 yr−1 (2012). A potential explanation for the higher range of top-down estimates compared to bottom-up inventories could be the contribution from natural sources, such as peatlands, wetlands, and wet soils. Based on seven different wetland inventories from the Wetland and Wetland CH4 Inter-comparison of Models Project (WETCHIMP) total wetland emissions of 4.3 (2.3–8.2) CH4 yr−1 from EU-28 are estimated. The hypothesis of significant natural emissions is supported by the finding that several inverse models yield significant seasonal cycles of derived CH4 emissions with maxima in summer, while anthropogenic CH4 emissions are assumed to have much lower seasonal variability. Furthermore, we investigate potential biases in the inverse models by comparison with regular aircraft profiles at four European sites and with vertical profiles obtained during the Infrastructure for Measurement of the European Carbon Cycle (IMECC) aircraft campaign. We present a novel approach to estimate the biases in the derived emissions, based on the comparison of simulated and measured enhancements of CH4 compared to the background, integrated over the entire boundary layer and over the lower troposphere. This analysis identifies regional biases for several models at the aircraft profile sites in France, Hungary and Poland.

scholarly journals Assessment of the TROPOMI tropospheric NO<sub>2</sub> product based on airborne APEX observations

2020 ◽  
Author(s):  
Frederik Tack ◽  
Alexis Merlaud ◽  
Marian-Daniel Iordache ◽  
Gaia Pinardi ◽  
Ermioni Dimitropoulou ◽  
...  
Keyword(s):  
Remote Sensing ◽  
A Priori ◽  
Horizontal Distribution ◽  
Time Interval ◽  
Short Time Interval ◽  
Vertical Column ◽  
Data Set ◽  
Urban Regions ◽  
Tropospheric No2 ◽  
The Impact

Abstract. Sentinel-5 Precursor (S-5P), launched in October 2017, carrying the TROPOspheric Monitoring Instrument (TROPOMI) nadir-viewing spectrometer, is the first mission of the Copernicus Programme dedicated to the monitoring of air quality, climate, and ozone. In the presented study, the TROPOMI tropospheric nitrogen dioxide (NO2) L2 product (OFFL v1.03.01; 3.5 km × 7 km at nadir observations) has been validated over strongly polluted urban regions by comparison with coincident high-resolution Airborne Prism EXperiment (APEX) remote sensing observations (~75 m × 120 m). Satellite products can be optimally assessed based on (APEX) airborne remote sensing observations as a large amount of satellite pixels can be fully mapped at high accuracy and in a relatively short time interval, reducing the impact of spatio-temporal mismatches. In the framework of the S5PVAL-BE campaign, the APEX imaging spectrometer has been deployed during four mapping flights (26–29 June 2019) over the two largest urban regions in Belgium, i.e. Brussels and Antwerp, in order to map the horizontal distribution of tropospheric NO2. For each flight, 10 to 20 TROPOMI pixels were fully covered by approximately 2800 to 4000 APEX measurements within each TROPOMI pixel. The TROPOMI and APEX NO2 vertical column density (VCD) retrieval schemes are similar in concept. Overall for the ensemble of the four flights, the standard TROPOMI NO2 VCD product is well correlated (R = 0.92) but biased negatively by −1.2 ± 1.2 × 1015 molec cm−2 or −14 % ± 12 %, on average, with respect to coincident APEX NO2 retrievals. When replacing the coarse 1° × 1° TM5-MP a priori NO2 profiles by NO2 profile shapes from the CAMS regional CTM ensemble at 0.1° × 0.1°, the slope increases by 11 % to 0.93, and the bias is reduced to −0.1 ± 1.0 × 1015 molec cm−2 or −1.0 % ± 12 %. When the absolute value of the difference is taken, the bias is 1.3 × 1015 molec cm−2 or 16 %, and 0.7 × 1015 molec cm−2 or 9 % on average, when comparing APEX NO2 VCDs with TM5-MP-based and CAMS-based NO2 VCDs, respectively. Both sets of retrievals are well within the accuracy requirement of a maximum bias of 25–50 % for the TROPOMI tropospheric NO2 product for all individual compared pixels. Additionally, the APEX data set allows the study of TROPOMI subpixel variability and impact of signal smoothing due to its finite satellite pixel size, typically coarser than fine-scale gradients in the urban NO2 field. The amount of underestimation of peak plume values and overestimation of urban background values in the TROPOMI data is in the order of 1–2 × 1015 molec cm−2 on average, or 10 %–20 %, in case of an urban scene.

scholarly journals Structural and statistical properties of the collocation technique for error characterization

2012 ◽  
Vol 19 (1) ◽  
pp. 69-80 ◽  
Author(s):  
S. Zwieback ◽  
K. Scipal ◽  
W. Dorigo ◽  
W. Wagner
Keyword(s):  
Remote Sensing ◽  
Expert Knowledge ◽  
Simulated Data ◽  
Data Sources ◽  
Data Sets ◽  
Data Set ◽  
Formidable Challenge ◽  
Error Characterization ◽  
Collocation Technique ◽  
The Impact

Abstract. The validation of geophysical data sets (e.g. derived from models, exploration techniques or remote sensing) presents a formidable challenge as all products are inherently different and subject to errors. The collocation technique permits the retrieval of the error variances of different data sources without the need to specify one data set as a reference. In addition calibration constants can be determined to account for biases and different dynamic ranges. The method is frequently applied to the study and comparison of remote sensing, in-situ and modelled data, particularly in hydrology and oceanography. Previous studies have almost exclusively focussed on the validation of three data sources; in this paper it is shown how the technique generalizes to an arbitrary number of data sets. It turns out that only parts of the covariance structure can be resolved by the collocation technique, thus emphasizing the necessity of expert knowledge for the correct validation of geophysical products. Furthermore the bias and error variance of the estimators are derived with particular emphasis on the assumptions necessary for establishing those characteristics. Important properties of the method, such as the structural deficiencies, dependence of the accuracy on the number of measurements and the impact of violated assumptions, are illustrated by application to simulated data.

scholarly journals Model-Based Geostatistics from a Bayesian Perspective: Investigating Area-to-Point Kriging with Small Data Sets

2019 ◽  
Vol 52 (3) ◽  
pp. 397-423
Author(s):  
Luc Steinbuch ◽  
Thomas G. Orton ◽  
Dick J. Brus
Keyword(s):  
Prior Distribution ◽  
Crop Yields ◽  
Simulated Data ◽  
Monte Carlo Sampling ◽  
Model Parameters ◽  
Small Data ◽  
Data Sets ◽  
Data Set ◽  
Minimal Impact ◽  
The Impact

AbstractArea-to-point kriging (ATPK) is a geostatistical method for creating high-resolution raster maps using data of the variable of interest with a much lower resolution. The data set of areal means is often considerably smaller ($$<\,50 $$<50 observations) than data sets conventionally dealt with in geostatistical analyses. In contemporary ATPK methods, uncertainty in the variogram parameters is not accounted for in the prediction; this issue can be overcome by applying ATPK in a Bayesian framework. Commonly in Bayesian statistics, posterior distributions of model parameters and posterior predictive distributions are approximated by Markov chain Monte Carlo sampling from the posterior, which can be computationally expensive. Therefore, a partly analytical solution is implemented in this paper, in order to (i) explore the impact of the prior distribution on predictions and prediction variances, (ii) investigate whether certain aspects of uncertainty can be disregarded, simplifying the necessary computations, and (iii) test the impact of various model misspecifications. Several approaches using simulated data, aggregated real-world point data, and a case study on aggregated crop yields in Burkina Faso are compared. The prior distribution is found to have minimal impact on the disaggregated predictions. In most cases with known short-range behaviour, an approach that disregards uncertainty in the variogram distance parameter gives a reasonable assessment of prediction uncertainty. However, some severe effects of model misspecification in terms of overly conservative or optimistic prediction uncertainties are found, highlighting the importance of model choice or integration into ATPK.

scholarly journals Benefits of assimilating thin sea-ice thickness from SMOS-Ice into the TOPAZ system

2016 ◽  
Author(s):  
Jiping Xie ◽  
Francois Counillon ◽  
Laurent Bertino ◽  
Xiangshan Tian-Kunze ◽  
Lars Kaleschke
Keyword(s):  
Sea Ice ◽  
Degrees Of Freedom ◽  
Winter Season ◽  
The Arctic ◽  
Ice Thickness ◽  
Data Sets ◽  
Data Set ◽  
Sea Ice Thickness ◽  
Ice Concentration ◽  
The Impact

Abstract. An observation product for thin sea ice thickness (SMOS-Ice) is derived from the brightness temperature data of the European Space Agency's (ESA) Soil Moisture and Ocean Salinity (SMOS) Mission, and available in real-time at daily frequency during the winter season. In this study, we investigate the benefit of assimilating SMOS-Ice into the TOPAZ system. TOPAZ is a coupled ocean-sea ice forecast system that assimilates SST, altimetry data, temperature and salinity profiles, ice concentration, and ice drift with the Ensemble Kalman Filter (EnKF). The conditions for assimilation of sea ice thickness thinner than 0.4 m are favorable, as observations are reliable below this threshold and their probability distribution is comparable to that of the model. Two paralleled runs of TOPAZ have been performed respectively in March and November 2014, with assimilation of thin sea ice thickness (thinner than 0.4 m) in addition to the standard ice and ocean observational data sets. It is found that the RMSD of thin sea-ice thickness is reduced by 11 % in March and 22 % in November suggesting that SMOS-Ice has a larger impact during the beginning of freezing season. There is a slight improvement of the ice concentration and no degradation of the ocean variables. The Degrees of Freedom for Signal (DFS) indicate that the SMOS-Ice contents important information (> 20 % of the impact of all observations) for some areas in the Arctic. The areas of largest impact are the Kara Sea, the Canadian archipelago, the Baffin Bay, the Beaufort Sea and the Greenland Sea. This study suggests that SMOS-Ice is a good complementary data set that can be safely included in the TOPAZ system as it improves the ice thickness and the ice concentration but does not degrade other quantities. Keywords: SMOS-Ice; EnKF; OSE; thin sea-ice thickness; DFS;

scholarly journals Accounting for the bin structure of data removes bias when fitting size spectra

2020 ◽  
Vol 636 ◽  
pp. 19-33 ◽  
Author(s):  
AM Edwards ◽  
JPW Robinson ◽  
JL Blanchard ◽  
JK Baum ◽  
MJ Plank
Keyword(s):  
Simulated Data ◽  
Likelihood Method ◽  
Size Spectrum ◽  
Trawl Survey ◽  
Power Law Distribution ◽  
Data Types ◽  
Data Set ◽  
Size Spectra ◽  
High Resolution Data ◽  
Biased Estimates

Size spectra are recommended tools for detecting the response of marine communities to fishing or to management measures. A size spectrum succinctly describes how a property, such as abundance or biomass, varies with body size in a community. Required data are often collected in binned form, such as numbers of individuals in 1 cm length bins. Numerous methods have been employed to fit size spectra, but most give biased estimates when tested on simulated data, and none account for the data’s bin structure (breakpoints of bins). Here, we used 8 methods to fit an annual size-spectrum exponent, b, to an example data set (30 yr of the North Sea International Bottom Trawl Survey). The methods gave conflicting conclusions regarding b declining (the size spectrum steepening) through time, and so any resulting advice to ecosystem managers will be highly dependent upon the method used. Using simulated data, we showed that ignoring the bin structure gives biased estimates of b, even for high-resolution data. However, our extended likelihood method, which explicitly accounts for the bin structure, accurately estimated b and its confidence intervals, even for coarsely collected data. We developed a novel visualisation method that accounts for the bin structure and associated uncertainty, provide recommendations concerning different data types and have created an R package (sizeSpectra) to reproduce all results and encourage use of our methods. This work is also relevant to wider applications where a power-law distribution (the underlying distribution for a size spectrum) is fitted to binned data.

scholarly journals Empirical Evidence on the Impact of City Taxes

2016 ◽  
Vol 4 (2) ◽  
pp. 65
Author(s):  
Charles Swenson
Keyword(s):  
Los Angeles ◽  
A Priori ◽  
The United States ◽  
Tax Rates ◽  
Significant Cost ◽  
Data Set ◽  
Business Tax ◽  
Government Data ◽  
The Impact ◽  
Municipal Level

An important issue is whether cities can influence their own economic growths through municipal-level tax policy. There is little evidence on this to date, nor is there a clear a priori answer to this question. This study first documents municipal business tax rates across the United States, and finds they are a relatively significant cost to business. Next, using very a unique and precise government data set, the study examines the economic impacts of two previous tax cuts in Los Angeles and finds that these cuts generally resulted in growth in both the number of jobs and establishments.

Sign in / Sign up

Export Citation Format

Share Document