scholarly journals Conceptualizing and Measuring the Promotion of Nonprofit Organizations’ Evidence Use by U.S. Social Service Funding Programs

2021 ◽  
Vol 7 (2) ◽  
pp. 240-263
Author(s):  
Christopher S. Horne ◽  
John K. Brock ◽  
J. Kenzie Freeman ◽  
Holly S. Odell
Keyword(s):  
Nonprofit Organizations ◽  
Conceptual Framework ◽  
Social Service ◽  
External Validity ◽  
Internal Validity ◽  
Federal Funding ◽  
Fiscal Year ◽  
Evidence Based ◽  
Evidence Use ◽  
Funding Programs

Previous research on U.S. federal promotion of evidence-based programming has focused on evidence-based program registries and concludes their usefulness is undermined by prioritizing internal validity over external validity. This research explores how federal funding programs are actually promoting funded nonprofit organizations’ evidence use instead of what we might infer from registries alone. An inductively developed conceptual framework is applied to describe all 53 fiscal year (FY) 2019 social service funding programs that include nonprofit organizations among the eligible applicants, finding they promote multiple types of evidence use, with generally low coerciveness, and with applicants frequently co-determining what counts as evidence. These findings point to promotion of evidence use that balances evidence-driven prescriptiveness and enabling nonprofits’ innovation.

A Focus on Evaluating Major Study Limitations in Order to Apply Clinical Trials to Patient Care: Implications for the Healthcare Team

2020 ◽  
pp. 001857872093175
Author(s):  
Mary J. Ferrill ◽  
Alireza FakhriRavari ◽  
Lisa Hong ◽  
Jody Jacobson Wedret
Keyword(s):  
Clinical Trials ◽  
Clinical Practice ◽  
Patient Care ◽  
External Validity ◽  
Healthcare Providers ◽  
Internal Validity ◽  
Healthcare Team ◽  
Evidence Based ◽  
Major Study ◽  
Clinically Significant

Background: With more than a million new biomedical articles published annually, healthcare providers must stay up to date in order to provide optimal evidence-based patient care. The concise ROOTs (relevance, observe validity, obtain clinically significant results, and translate results to clinical practice) format is a valuable tool to assist with literature evaluation. Purpose: To illustrate how major study limitations found in clinical trials might inhibit the ability to adopt the findings of such studies to patient care. Methods: Examples from published clinical trials that contain major study flaws were used to illustrate, if taken at face value, would lead to erroneous assumptions, and if adopted, could potentiallly harm patients. Conclusion: When evaluating the literature, it is crucial to identify limitations in the published literature that might reduce the internal validity, affect the results, or limit the external validity of clinical trials, hence affecting the usability of literature for patient care. This article provides examples of clinical trials that contain major study limitations with potentially erroneous assumptions. These illustrations are meant to show how important it is to delve deeper into an article before conclusions are drawn.

Furthering Dissemination and Implementation Research

2017 ◽  
Author(s):  
Lawrence W. Green ◽  
Mona Nasser
Keyword(s):  
Systematic Reviews ◽  
Implementation Research ◽  
External Validity ◽  
Evidence Based Practice ◽  
Internal Validity ◽  
Implementation Process ◽  
Dissemination And Implementation ◽  
Evidence Based ◽  
Evidence Based Practices ◽  
Dissemination And Implementation Research

This chapter raises questions about the reliability of much “evidence-based practice” disseminated from the original studies and systematic reviews of those studies, insofar as they were often conducted and reviewed with inadequate attention to external validity. Important issues are raised for dissemination and implementation researchers. Indeed, the pressure on investigators to provide for increasingly rigorous controls on threats to internal validity, and to exclude studies that fall below standards for internal validity, has made many such sources of evidence more suspect in their external validity and less credible to the practitioners or policymakers who would adopt them. Greater attention is needed to ways to incorporate considerations of external validity into studies and in systematic reviews of studies to produce more generalizable evidence, and greater attention to practice-based evidence that can complement the more formal evidence-based practices in the process of implementing and evaluating the dissemination and implementation process

Training Nonprofit Management Students to Critically Consume and Apply Knowledge Curated by Evidence-Based Program Registries

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Christopher Scott Horne
Keyword(s):  
Nonprofit Organizations ◽  
Real Life ◽  
Internal Validity ◽  
Nonprofit Management ◽  
Evidence Based ◽  
Complex Environments ◽  
Pedagogical Model ◽  
Program Evaluations ◽  
Wide Range ◽  
Management Students

Nonprofit management students can learn a great deal about program design and evaluation from evidence-based program registries (EBPRs), such as the U.S. Department of Education’s What Works Clearinghouse and the Department of Justice’s CrimeSolutions.gov, which synthesize program evaluations and research about a wide range of types of programs that nonprofit organizations commonly operate. Nonprofit management educators and their students should use EBPRs with caution, though, as their research syntheses tend to prioritize internal validity over external validity. This article offers a pedagogical model useful for training students to critically consume and apply EBPR resources. Classroom-tested examples are provided to demonstrate the usefulness of this pedagogical model for preparing students to use EBPR resources independently in real-life settings. This pedagogical model is recommended for use in nonprofit management education more generally, as future nonprofit managers should be prepared to work effectively and independently in complex environments.

Evidence-Based Social Sciences: A New Emerging Field

2018 ◽  
Vol 5 (2) ◽  
pp. 207-211
Author(s):  
Nazila Zarghi ◽  
Soheil Dastmalchian Khorasani
Keyword(s):  
Social Sciences ◽  
Critical Appraisal ◽  
State Of The Art ◽  
Internal Validity ◽  
Evidence Based ◽  
Multiple Sources ◽  
Social Settings ◽  
Research Findings ◽  
Academic Area ◽  
Different Levels

Abstract Evidence based social sciences, is one of the state-of- the-art area in this field. It is making decisions on the basis of conscientious, explicit and judicious use of the best available evidence from multiple sources. It also could be conducive to evidence based social work, i.e a kind of evidence based practice in some extent. In this new emerging field, the research findings help social workers in different levels of social sciences such as policy making, management, academic area, education, and social settings, etc.When using research in real setting, it is necessary to do critical appraisal, not only for trustingon internal validity or rigor methodology of the paper, but also for knowing in what extent research findings could be applied in real setting. Undoubtedly, the latter it is a kind of subjective judgment. As social sciences findings are highly context bound, it is necessary to pay more attention to this area. The present paper tries to introduce firstly evidence based social sciences and its importance and then propose criteria for critical appraisal of research findings for application in society.

scholarly journals Development of a measure of kindness

2021 ◽  
Author(s):  
Donna E. Youngs ◽  
Miroslava A. Yaneva ◽  
David V. Canter
Keyword(s):  
Factor Analysis ◽  
Multivariate Analysis ◽  
Positive Psychology ◽  
Conceptual Framework ◽  
External Validity ◽  
Exploratory Study ◽  
Male And Female ◽  
Psychometric Instrument ◽  
Smallest Space Analysis ◽  
Item Questionnaire

AbstractIn the spirit of the growing developments in positive psychology, there is an increasing interest in how kind people are to each other. Yet, this area lacks any strong psychometric instrument. An initial exploratory study demonstrated that a 40-item questionnaire, completed by 165 people, revealed distinct aspects of kindness when subjected to multivariate analysis. A subsequent study is reported, using the structure of the exploratory results to further clarify the conceptual framework (Study 1). The revised 45-item questionnaire was administered to 1039 individuals from the general British population. Smallest Space Analysis of the variables, supported by Factor analysis, confirmed the hypothesis of two facets to kindness, the psychological source of the action (from principles or empathy), and the form of expression (through psychological involvement or following social prescription. It also revealed an additional general, core kindness, labelled Anthropophilia. Reliable scales derived from the combinations of the two elements from each facet were identified: Affective-Socially Prescribed; Affective-Proactive; Principle-Socially Prescribed and Principle-Proactive. Intercorrelations between the scales revealed that they measure different modes of kindness. Comparisons between male and female respondents provided external validity for the questionnaire. Study 2 (N = 251) reported that the scales measure independent dimensions when correlated with similar and dissimilar concepts.

scholarly journals A novel preference-informed complementary trial (PICT) design for clinical trial research influenced by strong patient preferences

Trials ◽  
2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Samina Ali ◽  
◽  
Gareth Hopkin ◽  
Naveen Poonai ◽  
Lawrence Richer ◽  
...  
Keyword(s):  
External Validity ◽  
Treatment Options ◽  
Internal Validity ◽  
Clinical Effectiveness ◽  
Drop Out ◽  
Innovative Design ◽  
Clinical Trial Research ◽  
Research Questions ◽  
Internal And External Validity ◽  
Treatment Population

Abstract Background Patients and their families often have preferences for medical care that relate to wider considerations beyond the clinical effectiveness of the proposed interventions. Traditionally, these preferences have not been adequately considered in research. Research questions where patients and families have strong preferences may not be appropriate for traditional randomized controlled trials (RCTs) due to threats to internal and external validity, as there may be high levels of drop-out and non-adherence or recruitment of a sample that is not representative of the treatment population. Several preference-informed designs have been developed to address problems with traditional RCTs, but these designs have their own limitations and may not be suitable for many research questions where strong preferences and opinions are present. Methods In this paper, we propose a novel and innovative preference-informed complementary trial (PICT) design which addresses key weaknesses with both traditional RCTs and available preference-informed designs. In the PICT design, complementary trials would be operated within a single study, and patients and/or families would be given the opportunity to choose between a trial with all treatment options available and a trial with treatment options that exclude the option which is subject to strong preferences. This approach would allow those with strong preferences to take part in research and would improve external validity through recruiting more representative populations and internal validity. Here we discuss the strengths and limitations of the PICT design and considerations for analysis and present a motivating example for the design based on the use of opioids for pain management for children with musculoskeletal injuries. Conclusions PICTs provide a novel and innovative design for clinical trials with more than two arms, which can address problems with existing preference-informed trial designs and enhance the ability of researchers to reflect shared decision-making in research as well as improving the validity of trials of topics with strong preferences.

Investigating the Youth Physical Activity Promotion Model: Internal Structure and External Validity Evidence for a Potential Measurement Model

2007 ◽  
Vol 19 (4) ◽  
pp. 420-435 ◽  
Author(s):  
David A. Rowe ◽  
Thomas D. Raedeke ◽  
Lenny D. Wiersma ◽  
Matthew T. Mahar
Keyword(s):  
Physical Activity ◽  
External Validity ◽  
Internal Validity ◽  
Physical Activity Promotion ◽  
Measurement Model ◽  
Measurement Properties ◽  
Validity Evidence ◽  
Potential Measurement ◽  
Activity Promotion ◽  
Youth Physical Activity

The purpose of the study was to investigate the measurement properties of questionnaires associated with the Youth Physical Activity Promotion (YPAP) model. Data were collected from 296 children in Grades 5–8 using several existing questionnaires corresponding to YPAP model components, a physical activity questionnaire, and 6 consecutive days of pedometer data. Internal validity of the questionnaires was tested using confirmatory factor analyses, and external validity was investigated via correlations with physical activity and body composition. Initial model fit of the questionnaires ranged from poor to very good. After item removal, all scales demonstrated good fit. Correlations with percentage body fat and objectively measured physical activity were low but in the theoretically predicted direction. The current study provides good internal validity evidence and acceptable external validity evidence for a brief set of questionnaire items to investigate the theoretical basis for the YPAP model.

scholarly journals Toward Experimental International Business

2015 ◽  
Vol 22 (4) ◽  
pp. 530-544 ◽  
Author(s):  
Arjen van Witteloostuijn
Keyword(s):  
International Business ◽  
External Validity ◽  
Design Methodology ◽  
Internal Validity ◽  
Cross Cultural ◽  
Web Based ◽  
Content Type ◽  
Experimental Laboratory ◽  
Experimental Management ◽  
Missed Opportunity

Purpose – The purpose of this paper is to argue that the time is ripe to establish a powerful tradition in Experimental International Business (IB). Probably due to what the Arjen van Witteloostuijn refers to as the external validity myth, experimental laboratory designs are underutilized in IB, which implies that the internal validity miracle of randomized experimentation goes largely unnoticed in this domain of the broader management discipline. Design/methodology/approach – In the following pages, the author explains why the author believes this implies a missed opportunity, providing arguments and examples along the way. Findings – Although an Experimental Management tradition has never really gained momentum, to the author, the lab experimental design has a very bright future in IB (and management at large). To facilitate the development of an Experimental IB tradition, initiating web-based tools would be highly instrumental. This will not only boost further progress in IB research, but will also increase the effectiveness and playfulness of IB teaching. Originality/value – Given the high potential of an Experimental IB, the Cross-Cultural and Strategic Management journal will offer a platform for such exciting and intriguing laboratory work, cumulatively contributing to the establishment of an Experimental IB tradition.

scholarly journals Designing Research With Qualitative Comparative Analysis (QCA): Approaches, Challenges, and Tools

2017 ◽  
Vol 49 (2) ◽  
pp. 356-386 ◽  
Author(s):  
Eva Thomann ◽  
Martino Maggetti
Keyword(s):  
Comparative Analysis ◽  
External Validity ◽  
Qualitative Comparative Analysis ◽  
Internal Validity ◽  
Specific Research ◽  
Extant Literature ◽  
Specific Mode ◽  
Research Context ◽  
Trade Offs ◽  
Conducting Research

Recent years have witnessed a host of innovations for conducting research with qualitative comparative analysis (QCA). Concurrently, important issues surrounding its uses have been highlighted. In this article, we seek to help users design QCA studies. We argue that establishing inference with QCA involves three intertwined design components: first, clarifying the question of external validity; second, ensuring internal validity; and third, explicitly adopting a specific mode of reasoning. We identify several emerging approaches to QCA rather than just one. Some approaches emphasize case knowledge, while others are condition oriented. Approaches emphasize either substantively interpretable or redundancy-free explanations, and some designs apply an inductive/explorative mode of reasoning, while others integrate deductive elements. Based on extant literature, we discuss issues surrounding inference with QCA and the tools available under different approaches to address these issues. We specify trade-offs and the importance of doing justice to the nature and goals of QCA in a specific research context.

Sign in / Sign up

Export Citation Format

Share Document