scholarly journals Promises and Perils of Experimentation: The Mutual-Internal-Validity Problem

2021 ◽  
pp. 174569162097477
Author(s):  
Hause Lin ◽  
Kaitlyn M. Werner ◽  
Michael Inzlicht
Keyword(s):  
Real World ◽  
External Validity ◽  
Internal Validity ◽  
Theory Testing ◽  
The Real

Researchers run experiments to test theories, search for and document phenomena, develop theories, or advise policymakers. When testing theories, experiments must be internally valid but do not have to be externally valid. However, when experiments are used to search for and document phenomena, develop theories, or advise policymakers, external validity matters. Conflating these goals and failing to recognize their tensions with validity concerns can lead to problems with theorizing. Psychological scientists should be aware of the mutual-internal-validity problem, long recognized by experimental economists. When phenomena elicited by experiments are used to develop theories that, in turn, influence the design of theory-testing experiments, experiments and theories can become wedded to each other and lose touch with reality. They capture and explain phenomena within but not beyond the laboratory. We highlight how triangulation can address validity problems by helping experiments and theories make contact with ideas from other disciplines and the real world.

scholarly journals Promises and perils of experimentation: The mutual internal validity problem

2020 ◽  
Author(s):  
Hause Lin ◽  
Kaitlyn M. Werner ◽  
Michael Inzlicht
Keyword(s):  
Real World ◽  
External Validity ◽  
Internal Validity ◽  
Theory Testing ◽  
The Real

Researchers run experiments to test theories, search for and document phenomena, develop theories, or advise policymakers. When testing theories, experiments must be internally valid but do not have to be externally valid. However, when experiments are used to search for and document phenomena, develop theories, or advise policymakers, external validity matters. Conflating these goals and failing to recognize their tensions with validity concerns can lead to problems with theorizing. Experimenters in psychology should be aware of the mutual-internal-validity problem, long recognized by experimental economists. When phenomena elicited by experiments are used to develop theories that, in turn, influence the design of theory-testing experiments, experiments and theories can become wedded to each other and lose touch with reality. They capture and explain phenomena within but not beyond the laboratory. We highlight how triangulation can address validity problems by helping experiments and theories make contact with ideas from other disciplines and the real world.

External Validity Reconsidered

2011 ◽  
Author(s):  
Diana C. Mutz
Keyword(s):  
Political Science ◽  
Real World ◽  
External Validity ◽  
Observational Studies ◽  
Internal Validity ◽  
Conventional Wisdom ◽  
Observational Research ◽  
Convincing Case

This chapter talks about the significance of generalizability. Experimentalists often go to great lengths to argue that student or other convenience samples are not problematic in terms of external validity. Likewise, a convincing case for causality is often elusive with observational research, no matter how stridently one might argue to the contrary. The conventional wisdom is that experiments are widely valued for their internal validity, and experiments lack external validity. These assumptions are so widespread as to go without question in most disciplines, particularly those emphasizing external validity, such as political science and sociology. But observational studies, such as surveys, are still supposed to be better for purposes of maximizing external validity because this method allows studying people in real world settings.

Internal and External Validity in Ethical Reasoning

2021 ◽  
pp. 40-61
Author(s):  
James Wilson
Keyword(s):  
Real World ◽  
External Validity ◽  
Thought Experiment ◽  
Ethical Reasoning ◽  
Internal Validity ◽  
Thought Experiments ◽  
American Philosophy ◽  
Anglo American ◽  
Internal And External Validity ◽  
Rigorous Method

A particular approach to ethical reasoning has come to dominate much Anglo-American philosophy, one which assumes that the most rigorous method is to proceed by analysis of thought experiments. In thought experiments, features such as context and history are stripped away, and all factors other than those of ethical interest are stipulated to be equal. This chapter argues that even if a thought experiment produces results that are internally valid—in that it provides a genuine ethical insight about the highly controlled and simplified experimental scenario under discussion—this does not imply external validity. Just as in empirical experiments, there is a yawning gap between succeeding in the relatively easy project of establishing internal validity in a controlled and simplified context, and the more difficult one of establishing external validity in the messier and more complex real world.

Importing social preferences across contexts and the pitfall of over-generalization across theories

2012 ◽  
Vol 35 (1) ◽  
pp. 34-35 ◽  
Author(s):  
Anne C. Pisor ◽  
Daniel M. T. Fessler
Keyword(s):  
Real World ◽  
External Validity ◽  
Group Selection ◽  
Social Preferences ◽  
Cross Cultural ◽  
Strong Reciprocity ◽  
The Real

AbstractClaims regarding negative strong reciprocity do indeed rest on experiments lacking established external validity, often without even a small “menu of options.” Guala's review should prompt strong reciprocity proponents to extend the real-world validity of their work, exploring the preferences participants bring to experiments. That said, Guala's approach fails to differentiate among group selection approaches and glosses over cross-cultural variability.

scholarly journals Evaluation of the Internal and External Validity of Social Media and Mobile Technology Driven HPV Vaccination Interventions: Systematic Review Using Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) Framework

2021 ◽  
Author(s):  
Matthew Asare ◽  
Braden Popelsky ◽  
Emmanuel Akowuah ◽  
Beth A. Lanning ◽  
Jane R. Montealegre
Keyword(s):  
Social Media ◽  
Real World ◽  
External Validity ◽  
Hpv Vaccination ◽  
Internal Validity ◽  
Population Based ◽  
Sufficient Information ◽  
Population Based Study ◽  
Vaccination Rates ◽  
Effective Interventions

Social media HPV vaccination interventions show promise for increasing HPV vaccination rates. An important consideration for the implementation of effective interventions into real-world practice in the translation potential, or external validity, of the intervention. To this end, we conducted a systematic literature review to describe the current body of evidence regarding the external validity of social media HPV vaccination-related interventions. Constructs related to external validity were based on the RE-AIM (reach, effectiveness, adoption, implementation, maintenance) framework. Seventeen articles published between 2006 and 2020 met inclusion criteria. Three researchers independently coded each article using a validated RE-AIM (reach, effectiveness/efficacy, adoption, implementation, maintenance) framework. Discrepant codes were discussed with a fourth reviewer to gain consensus. Of these 17 studies, three were pilot efficacy studies, 10 were RCTs to evaluate effectiveness, one was a population-based study, and three did not explicitly state which type of study was conducted. Reflecting this distribution of study types, across all studies the mean level of reporting RE-AIM dimensions varied with reach recording 90.8%, effectiveness (72.1%), adoption (40.3%), implementation (45.6%), and maintenance (26.5%). This review suggests that while the current HPV vaccination social media-driven interventions provide sufficient information on internal validity (reach and effectiveness), few have aimed to gather data on external validity needed to translate the interventions into real world implementation. Our data suggest that implementation research is needed to move HPV vaccination-related interventions into practice. Included in this review are recommendations for enhancing the design and reporting of these HPV vaccination social media-related interventions.

scholarly journals Internal and External Validity of Social Media and Mobile Technology-Driven HPV Vaccination Interventions: Systematic Review Using the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) Framework

Vaccines ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. 197
Author(s):  
Matthew Asare ◽  
Braden Popelsky ◽  
Emmanuel Akowuah ◽  
Beth A. Lanning ◽  
Jane R. Montealegre
Keyword(s):  
Social Media ◽  
Real World ◽  
External Validity ◽  
Hpv Vaccination ◽  
Internal Validity ◽  
Population Based ◽  
Sufficient Information ◽  
Population Based Study ◽  
Vaccination Rates ◽  
Effective Interventions

Social media human papillomavirus (HPV) vaccination interventions show promise for increasing HPV vaccination rates. An important consideration for the implementation of effective interventions into real-world practice is the translation potential, or external validity, of the intervention. To this end, we conducted a systematic literature review to describe the current body of evidence regarding the external validity of social media HPV vaccination-related interventions. Constructs related to external validity were based on the reach, effectiveness, adoption, implementation, maintenance (RE-AIM) framework. Seventeen articles published between 2006 and 2020 met the inclusion criteria. Three researchers independently coded each article using a validated RE-AIM framework. Discrepant codes were discussed with a fourth reviewer to gain consensus. Of these 17 studies, 3 were pilot efficacy studies, 10 were randomized controlled trials (RCTs) to evaluate effectiveness, 1 was a population-based study, and 3 did not explicitly state which type of study was conducted. Reflecting this distribution of study types, across all studies the mean level of reporting RE-AIM dimensions varied with reach recording 90.8%, effectiveness (72.1%), adoption (40.3%), implementation (45.6%), and maintenance (26.5%). This review suggests that while the current HPV vaccination social media-driven interventions provide sufficient information on internal validity (reach and effectiveness), few have aimed to gather data on external validity needed to translate the interventions into real world implementation. Our data suggest that implementation research is needed to move HPV vaccination-related interventions into practice. Included in this review are recommendations for enhancing the design and reporting of these HPV vaccination social media-related interventions.

Current Issues: Evidence-Based Practice for the Real World: Thoughts on Clinical Education in Communication Disorders

2010 ◽  
Vol 20 (3) ◽  
pp. 100-105 ◽  
Author(s):  
Anne K. Bothe
Keyword(s):  
Clinical Education ◽  
Real World ◽  
Evidence Based Practice ◽  
Communication Disorders ◽  
University Faculty ◽  
Evidence Based ◽  
Speech Language Pathology ◽  
Basic Principles ◽  
The Real ◽  
Language Pathology

This article presents some streamlined and intentionally oversimplified ideas about educating future communication disorders professionals to use some of the most basic principles of evidence-based practice. Working from a popular five-step approach, modifications are suggested that may make the ideas more accessible, and therefore more useful, for university faculty, other supervisors, and future professionals in speech-language pathology, audiology, and related fields.

The Real-World Fellowship

2006 ◽  
Vol 40 (7) ◽  
pp. 47
Author(s):  
LEE SAVIO BEERS
Keyword(s):  
Real World ◽  
The Real
Sign in / Sign up

Export Citation Format

Share Document