scholarly journals External Validity

2021 ◽  
Vol 24 (1) ◽  
pp. 365-393
Author(s):  
Michael G. Findley ◽  
Kyosuke Kikuta ◽  
Michael Denly
Keyword(s):  
Causal Inference ◽  
Scientific Inquiry ◽  
External Validity ◽  
Reporting Standards ◽  
Social Scientists ◽  
Target Populations ◽  
The Core ◽  
Evaluative Criteria ◽  
Generalized Knowledge ◽  
Equal Interest

External validity captures the extent to which inferences drawn from a given study's sample apply to a broader population or other target populations. Social scientists frequently invoke external validity as an ideal, but they rarely attempt to make rigorous, credible external validity inferences. In recent years, methodologically oriented scholars have advanced a flurry of work on various components of external validity, and this article reviews and systematizes many of those insights. We first clarify the core conceptual dimensions of external validity and introduce a simple formalization that demonstrates why external validity matters so critically. We then organize disparate arguments about how to address external validity by advancing three evaluative criteria: model utility, scope plausibility, and specification credibility. We conclude with a practical aspiration that scholars supplement existing reporting standards to include routine discussion of external validity. It is our hope that these evaluation and reporting standards help rebalance scientific inquiry, such that the current obsession with causal inference is complemented with an equal interest in generalized knowledge.

Improving the External Validity of Conjoint Analysis: The Essential Role of Profile Distribution

2021 ◽  
pp. 1-27 ◽  
Author(s):  
Brandon de la Cuesta ◽  
Naoki Egami ◽  
Kosuke Imai
Keyword(s):  
Conjoint Analysis ◽  
Real World ◽  
External Validity ◽  
Causal Effect ◽  
The Other ◽  
Estimation Methods ◽  
Social Scientists ◽  
Actual Distribution ◽  
Open Source Software Package ◽  
Profile Distribution

Abstract Conjoint analysis has become popular among social scientists for measuring multidimensional preferences. When analyzing such experiments, researchers often focus on the average marginal component effect (AMCE), which represents the causal effect of a single profile attribute while averaging over the remaining attributes. What has been overlooked, however, is the fact that the AMCE critically relies upon the distribution of the other attributes used for the averaging. Although most experiments employ the uniform distribution, which equally weights each profile, both the actual distribution of profiles in the real world and the distribution of theoretical interest are often far from uniform. This mismatch can severely compromise the external validity of conjoint analysis. We empirically demonstrate that estimates of the AMCE can be substantially different when averaging over the target profile distribution instead of uniform. We propose new experimental designs and estimation methods that incorporate substantive knowledge about the profile distribution. We illustrate our methodology through two empirical applications, one using a real-world distribution and the other based on a counterfactual distribution motivated by a theoretical consideration. The proposed methodology is implemented through an open-source software package.

scholarly journals Defining and Measuring News Media Quality: Comparing the Content Perspective and the Audience Perspective

2021 ◽  
pp. 194016122199966
Author(s):  
Philipp Bachmann ◽  
Mark Eisenegger ◽  
Diana Ingenhoff
Keyword(s):  
News Media ◽  
Social Scientists ◽  
Content Quality ◽  
Media System ◽  
Evaluative Criteria ◽  
Political Implications ◽  
Multimethod Approach ◽  
Media Systems ◽  
Measurement Approach

High-quality news is important, not only for its own sake but also for its political implications. However, defining, operationalizing, and measuring news media quality is difficult, because evaluative criteria depend upon beliefs about the ideal society, which are inherently contested. This conceptual and methodological paper outlines important considerations for defining news media quality before developing and applying a multimethod approach to measure it. We refer to Giddens' notion of double hermeneutics, which reveals that the ways social scientists understand constructs inevitably interact with the meanings of these constructs shared by people in society. Reflecting the two-way relationship between society and social sciences enables us to recognize news media quality as a dynamic, contingent, and contested construct and, at the same time, to reason our understanding of news media quality, which we derive from Habermas' ideal of deliberative democracy. Moreover, we investigate the Swiss media system to showcase our measurement approach in a repeated data collection from 2017 to 2020. We assess the content quality of fifty news media outlets using four criteria derived from the deliberative ideal ( N = 20,931 and 18,559 news articles and broadcasting items, respectively) and compare the results with those from two representative online surveys ( N = 2,169 and 2,159 respondents). The high correlations between both methods show that a deliberative understanding of news media quality is anchored in Swiss society and shared by audiences. This paper shall serve as a showcase to reflect and measure news media quality across other countries and media systems.

scholarly journals Introduction: Physics, Technology, and Technics during the Interwar Period

2018 ◽  
Vol 31 (3) ◽  
pp. 251-261
Author(s):  
Shaul Katzir
Keyword(s):  
World War Ii ◽  
Cold War ◽  
Interwar Period ◽  
Scientific Inquiry ◽  
Government Funding ◽  
World War ◽  
The Core ◽  
Historical Processes ◽  
The University ◽  
University Industry

Historians, philosophers, and physicists portray the 1920s and 1930s as a period of major theoretical breakthrough in physics, quantum mechanics, which led to the expansion of physics into the core of the atom and the growth and strengthening of the discipline. These important developments in scientific inquiry into the micro-world and light have turned historical attention away from other significant historical processes and from other equally important causes for the expansion of physics. World War II, on the other hand, is often seen as the watershed moment when physics achieved new levels of social and technical engagement at a truly industrial scale. Historians have shown that military interests and government funding have shaped physics to unprecedented degree, and according to some, to the extent of discontinuity with earlier practices of research (Forman 1987; Kevles 1990; Kaiser 2002). In this vein, Stuart Leslie wrote, “Nothing in the prewar experience fully prepared academic scientists and their institutions for the scale and scope of a wartime mobilization that would transform the university, industry, and the federal government and their mutual interrelationships” (Leslie 1993, 6). While one can never befullyready for novelties, the contributors to this issue show that developments in interwar physics did prepare participants for their cold war interactions with industry and government.

scholarly journals Adaptive Learning Systems and Interference in Causal Inference

2021 ◽  
Author(s):  
Alexander Olof Savi ◽  
Nick ten Broeke ◽  
Abe Dirk Hofman
Keyword(s):  
Causal Inference ◽  
Adaptive Learning ◽  
External Validity ◽  
Learning Systems ◽  
Learning System ◽  
Adaptive Learning System ◽  
Adaptive Learning Systems

Adaptive learning systems can be susceptible to between-subject cross-condition interference by design. This interference has important implications for the implementation and evaluation of A/B tests in such systems, as it obstructs causal inference and hurts external validity. We illustrate the problem in an Elo based adaptive learning system, discuss sources and degrees of interference, and provide solutions, using an example in the study of dropout.

scholarly journals Student Scientific Inquiry in the Core Curriculum

2013 ◽  
Vol 77 (8) ◽  
pp. 176 ◽  
Author(s):  
Georgeta D. Vaidean ◽  
Sandeep S. Vansal ◽  
Ronnie J. Moore ◽  
Stuart Feldman
Keyword(s):  
Scientific Inquiry ◽  
Core Curriculum ◽  
The Core

“My Regular Pharmacy”

2021 ◽  
pp. 177-191
Author(s):  
Νiki Glaveli
Keyword(s):  
Literature Review ◽  
Extensive Literature ◽  
Confirmatory Factor Analyses ◽  
Strategic Actions ◽  
The Core ◽  
Care Services ◽  
Evaluative Criteria ◽  
Confirmatory Factor ◽  
Average Variance ◽  
Patronage Behavior

The objectives of this chapter are (1) to develop an instrument that incorporates the core attributes of evaluative criteria that are used by patients/customers in selecting and repeatedly visiting a community pharmacy, (2) to analyze the proposed attributes in order to put forward valid dimensions of evaluative criteria, and (3) to assess the reliability of this instrument. The selection criteria incorporated in the final list were selected based on an extensive literature review and on experts' and customers' opinions. To collect the data, a survey was conducted in the area of Macedonia, Greece. In total, 223 questionnaires were collected. Confirmatory factor analyses revealed a four-factor structure of the proposed instrument namely: price, availability of products, staff competence, and pharmaceutical care services. Moreover, reliability was assessed through Cronbach's alpha coefficient and average variance extracted. The current study outcomes can guide pharmacists' strategic actions in boosting customer patronage behavior in the highly competitive pharmaceutical sector.

Group Size, Social Class, Religion, and Politics

2017 ◽  
pp. 31-51
Author(s):  
Chaim I. Waxman
Keyword(s):  
United States ◽  
New York ◽  
York City ◽  
Jewish Population ◽  
Religion And Politics ◽  
The United States ◽  
Orthodox Jewish ◽  
Social Scientists ◽  
The Core ◽  
Jewish Demography

This chapter focuses on determining the size of the Orthodox Jewish population in the United States and difficulties related to the problem of estimating the Jewish population as a whole. It analyses the acceptance of the notion of the 'core Jewish population' among social scientists and Jewish communal professionals. It also looks at major debates relating to significantly different estimates of population size among those specializing in Jewish demography. The chapter addresses questions as to whether belonging to an Orthodox synagogue makes one Orthodox, or whether being Orthodox entails matters of faith and behaviour. It cites the UJA-Federation of New York, which estimated the total Orthodox population in New York City at 493,000 in 2013.

scholarly journals Introduction to the Symposium: Causal Inference and Public Health

2019 ◽  
Vol 40 (1) ◽  
pp. 1-5 ◽  
Author(s):  
Allison E. Aiello ◽  
Lawrence W. Green
Keyword(s):  
Public Health ◽  
Causal Inference ◽  
Health Research ◽  
External Validity ◽  
Environmental Science ◽  
Social Epidemiology ◽  
Public Health Research ◽  
Health Problems ◽  
Research Findings ◽  
Internal And External Validity

Assessing the extent to which public health research findings can be causally interpreted continues to be a critical endeavor. In this symposium, we invited several researchers to review issues related to causal inference in social epidemiology and environmental science and to discuss the importance of external validity in public health. Together, this set of articles provides an integral overview of the strengths and limitations of applying causal inference frameworks and related approaches to a variety of public health problems, for both internal and external validity.

What Is the Social Responsibility of Social Scientists to Influence National Security Affairs?

2020 ◽  
pp. 0095327X2091718
Author(s):  
Yagil Levy
Keyword(s):  
National Security ◽  
Security Studies ◽  
Policy Recommendations ◽  
Relevant Research ◽  
Social Scientists ◽  
Critical Sociology ◽  
The Public ◽  
The Core ◽  
The Social ◽  
Security Affairs

Mainstream scholars of IR favor policy-relevant research, that is the agenda to influence government policymakers by offering policy recommendations. In this article, I offer a different perspective by presenting alternative arguments about social scientists’ responsibility to influence. By drawing on themes of public sociology and critical sociology, security studies and public policy, I argue that the core of this responsibility is to seek to influence policy via engagement with the public rather than with policymakers.

Symposium on Scientific Explanation in Political Science: Deduction, Explanation, and Social Scientific Inquiry

1969 ◽  
Vol 63 (4) ◽  
pp. 1233-1246
Author(s):  
John G. Gunnell
Keyword(s):  
Social Science ◽  
Philosophy Of Science ◽  
Political Science ◽  
Scientific Inquiry ◽  
Scientific Explanation ◽  
Rapid Expansion ◽  
Social Scientists ◽  
Methodological Problems ◽  
Social Scientific ◽  
The Many

The purpose here is to explore certain aspects of the philosophy of science which have serious implications both for the practice of social and political science and for understanding that practice. The current relationship between social science and the philosophy of science (or the philosophy of the social sciences) is a curious one. Despite the emergence of a considerable body of literature in philosophy which is pertinent to the methodological problems of social science, there has been a lack of osteusive ties between the two areas. A justified concern with the independence of social scientific research has contributed to a tendency toward isolation which is unfortunate in view of the proliferation of philosophical problems which necessarily attends the rapid expansion of any empirical discipline. Although in the literature of contemporary social science there are frequent references to certain works in the philosophy of science and to philosophical issues relating to methodology, these are most often in the context of bald pronouncements and shibboleths relating to the nature of science, its goals, and the character of its reasoning. But what is most disturbing about the fact that social scientists have little direct and thorough acquaintance with the philosophy of science is not merely that there has been a failure to carefully examine the many logical and epistemological assumptions which are implicit in social scientific inquiry, since this task might normally and properly be considered to be within the province of the philospher of science.

Sign in / Sign up

Export Citation Format

Share Document