Scaling up experimental social, behavioral, and economic science

The standard experimental paradigm in the social, behavioral, and economic sciences is extremely limited. Although recent advances in digital technologies and crowdsourcing services allow individual experiments to be deployed and run faster than in traditional physical labs, a majority of experiments still focus on one-off results that do not generalize easily to real-world contexts or even to other variations of the same experiment. As a result, there exist few universally acknowledged findings, and even those are occasionally overturned by new data. We argue that to achieve replicable, generalizable, scalable and ultimately useful social and behavioral science, a fundamental rethinking of the model of virtual-laboratory style experiments is required. Not only is it possible to design and run experiments that are radically different in scale and scope than was possible in an era of physical labs; this ability allows us to ask fundamentally different types of questions than have been asked historically of lab studies. We posit, however, that taking full advantage of this new and exciting potential will require four major changes to the infrastructure, methodology, and culture of experimental science: (1) significant investments in software design and participant recruitment, (2) innovations in experimental design and analysis of experimental data, (3) adoption of new models of collaboration, and (4) a new understanding of the nature and role of theory in experimental social and behavioral science. We conclude that the path we outline, although ambitious, is well within the power of current technology and has the potential to facilitate a new class of scientific advances in social, behavioral and economic studies.This paper emerged from discussions at a workshop held by the Computational Social Science Lab at the University of Pennsylvania in January 2020. The work was supported by James and Jane Manzi Analytics Fund and the Alfred P. Sloan Foundation.

A collaborative clinical case conference model for teaching social and behavioral science in medicine: an action research study

Background Effective social and behavioral sciences teaching in medical education requires integration with clinical experience, as well as collaboration between social and behavioral sciences experts and clinical faculty. However, teaching models for achieving this integration have not been adequately established, nor has the collaboration process been described. This study aims to propose a collaborative clinical case conference model to integrate social and behavioral sciences and clinical experience. Additionally, we describe how social and behavioral science experts and clinical faculty collaborate during the development of the teaching method. Methods A team of medical teachers and medical anthropologists planned for the development of a case conference based on action research methodology. The initial model was planned for a 3-h session, similar to a Clinicopathological Conference (CPC) structure. We evaluated each session based on field notes taken by medical anthropologists and post-session questionnaires that surveyed participants’ reactions and points of improvement. Based on the evaluation, a reflective meeting was held to discuss revisions for the next trial. We incorporated the development process into undergraduate medical curricula in clinical years and in a postgraduate and continuous professional development session for residents and certified family physicians in Japan. We repeated the plan-act-observe-reflection process more than 15 times between 2015 and 2018. Results The development of the collaborative clinical case conference model is summarized in three phases: Quasi-CPC, Interactive, and Co-constructive with unique structures and underlying paradigms. The model successfully contributed to promoting the participants’ recognition of the clinical significance of social and behavioral sciences. The case preparation entailed unique and significant learning of how social and behavioral sciences inform clinical practice. The model development process promoted the mutual understanding between clinical faculty and anthropologists, which might function as faculty development for teachers involved in social and behavioral sciences teaching in medical education. Conclusions The application of appropriate conference models and awareness of their underlying paradigms according to educational situations promotes the integration of social and behavioral sciences with clinical medicine education. Faculty development regarding social and behavioral sciences in medical education should focus on collaboration with scholars with different paradigmatic orientations.

Firm Behavior in the Face of Severe Weather: Economic Analysis between Probabilistic and Deterministic Warnings

Tornadoes cause billions of dollars in damage and over 100 fatalities on average annually. Yet, an indirect cost to these storms is found in lost sales and/or lost productivity from responding to over 2000 warnings per year. This project responds to the Weather Research and Forecasting Innovation Act of 2017, H.R. 353, which calls for the use of social and behavioral science to study and improve storm warning systems. Our goal is to provide an analysis of cost avoidance that could accrue from a change to the warning paradigm, particularly to include probabilistic hazard information at storm scales. A survey of nearly 500 firms was conducted in and near the Dallas–Fort Worth metropolitan area asking questions about experience with tornadoes, sources of information for severe weather, expected cost of responding to tornado warnings, and how the firm would respond to either deterministic or probabilistic warnings. We find a dramatic change from deterministic warnings compared to the proposed probabilistic and that a probabilistic information system produces annual cost avoidance in a range of $2.3–$7.6 billion (U.S. dollars) compared to the current deterministic warning paradigm.

Abundance

Chapter 1 situates the contemporary focus of this book in historical perspective by summarizing the main findings from studies of previous eras that had a massive surge in the amount of information available. Moreover, it critically examines the key contributions from social and behavioral science scholarship on information overload. In addition, it further articulates the conceptual framework that is initially introduced in the preface and that constitutes the analytical apparatus of the book. It also describes the research design adopted to gather the data necessary to answer the questions posed in the preface. Finally, it provides an outline of the book.

How Many Replicators Does It Take to Achieve Reliability? Investigating Researcher Variability in a Crowdsourced Replication

The paper reports findings from a crowdsourced replication. Eighty-four replicator teams attempted to verify results reported in an original study by running the same models with the same data. The replication involved an experimental condition. A “transparent” group received the original study and code, and an “opaque” group received the same underlying study but with only a methods section and description of the regression coefficients without size or significance, and no code. The transparent group mostly verified the original study (95.5%), while the opaque group had less success (89.4%). Qualitative investigation of the replicators’ workflows reveals many causes of non-verification. Two categories of these causes are hypothesized, routine and non-routine. After correcting non-routine errors in the research process to ensure that the results reflect a level of quality that should be present in ‘real-world’ research, the rate of verification was 96.1% in the transparent group and 92.4% in the opaque group. Two conclusions follow: (1) Although high, the verification rate suggests that it would take a minimum of three replicators per study to achieve replication reliability of at least 95% confidence assuming ecological validity in this controlled setting, and (2) like any type of scientific research, replication is prone to errors that derive from routine and undeliberate actions in the research process. The latter suggests that idiosyncratic researcher variability might provide a key to understanding part of the “reliability crisis” in social and behavioral science and is a reminder of the importance of transparent and well documented workflows.

COVID-19 exposed why translational social science is long overdue

The COVID-19 pandemic presented a unique challenge to social and behavioral science. Until very recently, and for almost a year, behavioral (non-pharmaceutical) measures were the only tools available to fight the pandemic: wearing masks, frequent handwashing, and social distancing from crowds and private gatherings. Despite the life-saving potential of these measures, a large number of people worldwide contested and undermined these recommendations, exacerbating the pandemic. With no biomedical solutions on the horizon for a prolonged time, the ball was in our court. The challenge was singularly within our realm of expertise. This was the time for social and behavioral science to shine. Did we rise to the occasion? Amid growing reflections about the (un)met challenges posed by the pandemic, and lessons (not) learned, it is time for reckoning.

A Collaborative Clinical Case Conference Model for Teaching Social and Behavioral Science in Medicine: An Action Research Study

Background: Effective social and behavioral sciences teaching in medical education requires integration with clinical experience, as well as collaboration with social and behavioral sciences experts and clinical faculties. However, teaching models for achieving this integration have not been adequately established, nor has the collaboration process been described. This study aims to propose a collaborative clinical case conference model to integrate social and behavioral sciences and clinical experience. Additionally, we describe how social and behavioral science experts and clinical faculties collaborate during the development of the teaching method.Methods: A team of clinical teachers and medical anthropologists planned for the development of a case conference based on action research methodology. The initial model planned for a 3-hour session, similar to the Clinicopathological Conference structure. We evaluated each session based on fieldnotes taken by medical anthropologists, and post-session questionnaires that surveyed participants’ reactions and points of improvements. Based on the evaluation, a reflective meeting was held to discuss revisions for the next trial. We incorporated the development process into undergraduate medical curricula in clinical years and in a postgraduate and continuous professional development session for residents and certified family physicians in Japan. We repeated the plan-act-observe-reflection process more than 15 times between 2015 and 2018.Results: The development of the collaborative clinical case conference model is summarized in three phases: Quasi-CPC, Interactive, and Co-constructive with unique structures and underlying paradigms. The model successfully contributed to promoting the participants’ recognition of the clinical significance of social and behavioral sciences. The case preparation entailed a unique, significant learning of how social and behavioral sciences inform clinical practice. The model development process promoted the mutual understanding between clinical faculties and anthropologists, which might function as faculty development for teachers involved in social and behavioral sciences teaching in medical education. Conclusions: The application of fitting conference models and awareness of their underlying paradigm according to educational situations could promote the integration of social and behavioral sciences with clinical medicine education. Faculty development in social and behavioral sciences in medical education should focus on collaboration with scholars with different paradigmatic orientations.