Comparing Theories with the Ising Model of Explanatory Coherence (IMEC)

Theories are among the most important tools of science. Lewin (1943) already noted “[t]here is nothing as practical as a good theory”. Although psychologists discussed problems of theory in their discipline for a long time, weak theories are still widespread in most subfields. One possible reason for this is that psychologists lack the tools to systematically assess the quality of their theories. Thagard (1989) developed a computational model for formal theory evaluation based on the concept of explanatory coherence. However, there are possible improvements to Thagard’s (1989) model and it is not available in software that psychologists typically use. Therefore, we developed a new implementation of explanatory coherence based on the Ising model. We demonstrate the capabilities of this new Ising Model of Explanatory Coherence (IMEC) on several examples from psychology and other sciences. It is also available in the R-package IMEC so that it can help scientists to evaluate the quality of their theories in practice.

The Struggle is Real: Challenges and Solutions in Theory Building

Strong theories are sorely lacking in the applied social sciences, especially in psychology. Elko Fried identifies fundamental problems that are common in social science research and explains how these problems manifest themselves in the literature, impede scientific progress, and contribute to the lack of theory building. He covers a lot of ground: weak theories, the conflation of theoretical and statistical models, problematic inferences, equating latent variables with psychological constructs, and more. As intelligence researchers, we are all too familiar with these kinds of problems; they have been sources of confusion and barriers to progress for over a century. A decade ago, the continued lack of concern over these issues in the field of intelligence became our major concern, and our motivation to develop a new approach. Suffice to say, Elko Fried is music to our ears.

How to Build A Strong Theoretical Foundation

It is our theories that shape how we see the world and the questions we ask of it. Fried argues that psychological science is plagued by weak theories, and that there is a real need for building and testing strong theories that include formal models. I agree. Fried calls for better training in the construction of formal theory, and I enthusiastically agree with this as well. However, I am concerned that the road to establish such a training in program will be long and hard. Fried ends his piece with tentative optimism, but little in the way of concrete proposals. Here I’ll outline what I think some of the necessary changes are and why implementing them will be challenging. I’ll conclude with some thoughts on how to overcome those challenges. Constructing good strong theories requires integration of the skills currently possessed by psychological scientists with (1) increased interdisciplinarity, (2) increased technical prowess, and (3) increased philosophical scrutiny.

Lack of theory building and testing impedes progress in the factor and network literature

The last decade has brought reforms to improve methodological practices, with the goal to increase the reliability and replicability of effects. However, explanations of effects remain scarce, and a growing chorus of scholars argues that the replicability crisis has distracted from a crisis of theory. In the same decade, the empirical literature using factor and network models has grown rapidly. I discuss three ways in which this literature falls short of theory building and testing. First, statistical and theoretical models are conflated, leading to invalid inferences such as the existence of psychological constructs based on factor models, or recommendations for clinical interventions based on network models. I demonstrate this inferential gap in a simulation study on statistical equivalence: excellent model fit does little to corroborate a theory, regardless of quality or quantity of data. Second, researchers fail to explicate theories about psychological constructs, but use implicit causal beliefs to guide inferences. These latent theories have led to problematic best practices in psychological research where inferences are drawn based on one specific causal model that is assumed, but not explicated. Third, explicated theories are often weak theories: narrative and imprecise descriptions vulnerable to hidden assumptions and unknowns. They fail to make clear predictions, and it remains unclear whether statistical effects corroborate such theories or not. Weak theories are immune to refutation or revision. I argue that these three challenges to theory building and testing are common and harmful, and impede theory formation, failure, and reform. A renewed focus on theoretical psychology and formal models offers a way forward.

PROOF-THEORETIC STRENGTHS OF WEAK THEORIES FOR POSITIVE INDUCTIVE DEFINITIONS

In this article the lightface ${\rm{\Pi }}_1^1$-Comprehension axiom is shown to be proof-theoretically strong even over ${\rm{RCA}}_0^{\rm{*}}$, and we calibrate the proof-theoretic ordinals of weak fragments of the theory ${\rm{I}}{{\rm{D}}_1}$ of positive inductive definitions over natural numbers. Conjunctions of negative and positive formulas in the transfinite induction axiom of ${\rm{I}}{{\rm{D}}_1}$ are shown to be weak, and disjunctions are strong. Thus we draw a boundary line between predicatively reducible and impredicative fragments of ${\rm{I}}{{\rm{D}}_1}$.