Environmental epidemiology in a crossfire

Two tendencies have emerged in environmental epidemiology that hamper the translation of research findings into prevention of environmental hazards. One is the increased focus on highlighting weaknesses of epidemiology research that is clearly meant to explain away the research conclusions and weaken their possible implications for interventions to control environmental hazards. Another is the voluminous amount of information sharing that involves a substantial amount of misinformation, as part of the ongoing infodemic. In this light, the appearance of the catalogue of doubt-raising strategies, indeed the worst practices of scientific inference, is good news. Collected under the auspices of the International Network for Epidemiology in Policy, it serves to illustrate the range of possible (and impossible) forms of critique that may be raised on behalf of vested interests or other groups who for some reason disagree with the epidemiological conclusions. We believe that this systematic list will be useful in our field and help to identify critiques of policy options that are hidden and sometimes suppressed in weighing the epidemiological evidence.

The epistemic consequences of pragmatic value-laden scientific inference

In this work, we explore the epistemic import of the value-ladenness of Neyman-Pearson’s Theory of Testing Hypotheses (N-P) by reconstructing and extending Daniel Steel’s argument for the legitimate influence of pragmatic values on scientific inference. We focus on how to properly understand N-P’s pragmatic value-ladenness and the epistemic reliability of N-P. We develop an account of the twofold influence of pragmatic values on N-P’s epistemic reliability and replicability. We refer to these two distinguished aspects as “direct” and “indirect”. We discuss the replicability of experiments in terms of the indirect aspect and the replicability of outcomes in terms of the direct aspect. We argue that the influence of pragmatic values is beneficial to N-P’s epistemic reliability and replicability indirectly. We show that while the direct influence of pragmatic values can be beneficial, its negative effects on reliability and replicability are also unavoidable in some cases, with the direct and indirect aspects possibly being incongruent.

Postscript

This postscript reflects on the role of parsimony in the future of systematics. Under the view of systematics advocated in this book, the exuberantly messy data of biological diversity are organized into a clear and coherent explanatory framework through the application of the principle of parsimony. The principle of common cause, the principle of cause and effect, and the principle of uniformitarianism are all applications of the principle of parsimony to the explanation of events unfolding in time. Thus, parsimony is not merely an old-fashioned phylogenetic method that has been superceded by purportedly more powerful and sophisticated statistical tools: it is the epistemological key to evaluating empirical evidence and discovering orderly patterns in the world to the extent that our perceptions allow. Ultimately, the success of every scientific inference and prediction relating to empirical phenomena in the world hinges upon parsimony.

Fragmentation and Old Evidence

Bayesian confirmation theory is our best formal framework for describing inductive reasoning. The problem of old evidence is a particularly difficult one for confirmation theory, because it suggests that this framework fails to account for central and important cases of inductive reasoning and scientific inference. I show that we can appeal to the fragmentation of doxastic states to solve this problem for confirmation theory. This fragmentation solution is independently well-motivated because of the success of fragmentation in solving other problems. I also argue that the fragmentation solution is preferable to other solutions to the problem of old evidence. These other solutions are already committed to something like fragmentation, but suffer from difficulties due to their additional commitments. If these arguments are successful, Bayesian confirmation theory is saved from the problem of old evidence, and the argument for fragmentation is bolstered by its ability to solve yet another problem.

Doing Science: Methods and Philosophy

The content of this dissertation spans four years of work, which was carried out in the Netherlands (Tilburg University and University of Amsterdam) and Italy (University of Turin). It is part of the ERC project “Making Scientific Inference More Objective” led by professor Jan Sprenger, for which philosophy of science and empirical research were combined. The dissertation can be summarized as a small set of modest attempts to contribute to improving scientific practice. Each of these attempts was geared towards either increasing understanding of a particular problem or making a contribution to how science can be practiced. The general focus was on philosophical nuance while remaining methodologically practicable. The five papers contained in this dissertation are both methodologically and philosophically diverse. The first three (Chapters 2 through 4) are more empirical in nature and are focused on understanding and evaluating how science is practiced: a meta-analysis of semantic intuitions research in experimental philosophy; a systematic review on essay literature on the null hypothesis significance test; and an experiment on how teams of statisticians analyze the same data. The last two (Chapters 5 and 6) are focused on the improvement of scientific practice by providing tools for the improvement of empirical research with a strong philosophical foundation: a practicable and testable definition of scientific objectivity and a Bayesian operationalization of Popper’s concept of a severe test.

How Computational Modeling Can Force Theory Building in Psychological Science

Psychology endeavors to develop theories of human capacities and behaviors on the basis of a variety of methodologies and dependent measures. We argue that one of the most divisive factors in psychological science is whether researchers choose to use computational modeling of theories (over and above data) during the scientific-inference process. Modeling is undervalued yet holds promise for advancing psychological science. The inherent demands of computational modeling guide us toward better science by forcing us to conceptually analyze, specify, and formalize intuitions that otherwise remain unexamined—what we dub open theory. Constraining our inference process through modeling enables us to build explanatory and predictive theories. Here, we present scientific inference in psychology as a path function in which each step shapes the next. Computational modeling can constrain these steps, thus advancing scientific inference over and above the stewardship of experimental practice (e.g., preregistration). If psychology continues to eschew computational modeling, we predict more replicability crises and persistent failure at coherent theory building. This is because without formal modeling we lack open and transparent theorizing. We also explain how to formalize, specify, and implement a computational model, emphasizing that the advantages of modeling can be achieved by anyone with benefit to all.

A Call for Greater Modesty in Psychology and Cognitive Neuroscience

Exaggerated claims and low levels of reproducibility are commonplace in psychology and cognitive neuroscience, due to an incentive structure that demands “newsworthy” results. My overall argument here is that in addition to methodological reform, greater modesty is required across all levels - from individual researchers to the systems that govern science (e.g., editors, reviewers, grant panels, hiring committees) - to redirect expectations regarding what psychological and brain science can effectively deliver. Empirical work and the reform agenda should pivot away from making big claims on narrow evidence bases or single tools and focus on the limitations of our individual efforts, as well as how we can work together to build ways of thinking that enable integration and synthesis across multiple modalities and levels of description. I outline why modesty matters for science including the reform agenda, provide some practical steps that we can take to embrace modesty, rebut common misconceptions of what modesty means for science, and present some limitations of the approach. Ultimately, by presenting a more sober view of our capacities and achievements, whilst placing work within a wider context that respects the complexity of the human brain, we will bolster the fidelity of scientific inference and thus help in a small way to generate a firmer footing upon which to build a cumulative science.

Stability of SARS-CoV-2 phylogenies

The SARS-CoV-2 pandemic has led to unprecedented, nearly real-time genetic tracing due to the rapid community sequencing response. Researchers immediately leveraged these data to infer the evolutionary relationships among viral samples and to study key biological questions, including whether host viral genome editing and recombination are features of SARS-CoV-2 evolution. This global sequencing effort is inherently decentralized and must rely on data collected by many labs using a wide variety of molecular and bioinformatic techniques. There is thus a strong possibility that systematic errors associated with lab—or protocol—specific practices affect some sequences in the repositories. We find that some recurrent mutations in reported SARS-CoV-2 genome sequences have been observed predominantly or exclusively by single labs, co-localize with commonly used primer binding sites and are more likely to affect the protein-coding sequences than other similarly recurrent mutations. We show that their inclusion can affect phylogenetic inference on scales relevant to local lineage tracing, and make it appear as though there has been an excess of recurrent mutation or recombination among viral lineages. We suggest how samples can be screened and problematic variants removed, and we plan to regularly inform the scientific community with our updated results as more SARS-CoV-2 genome sequences are shared (https://virological.org/t/issues-with-sars-cov-2-sequencing-data/473 and https://virological.org/t/masking-strategies-for-sars-cov-2-alignments/480). We also develop tools for comparing and visualizing differences among very large phylogenies and we show that consistent clade- and tree-based comparisons can be made between phylogenies produced by different groups. These will facilitate evolutionary inferences and comparisons among phylogenies produced for a wide array of purposes. Building on the SARS-CoV-2 Genome Browser at UCSC, we present a toolkit to compare, analyze and combine SARS-CoV-2 phylogenies, find and remove potential sequencing errors and establish a widely shared, stable clade structure for a more accurate scientific inference and discourse.