A Comparative Analysis: Heuristic Self-Search Inquiry as Self-Knowledge and Knowledge of Society

2015 ◽  
Vol 57 (3) ◽  
pp. 237-251 ◽  
Author(s):  
Engin Ozertugrul
Keyword(s):  
Comparative Analysis ◽  
Knowledge Production ◽  
External Validity ◽  
Internal Validity ◽  
Standard View ◽  
Validity Measures ◽  
Self Knowledge ◽  
Conventional Validity ◽  
Knowledge Exploration

In research, the standard view of credibility seeks to illuminate what the researcher did with the data vis-à-vis collection, analysis, and interpretation. This works well in standard research where data can be checked through conventional validity measures (internal validity, external validity, reliability, replicability, and objectivity). It does not work well in heuristic self-search inquiry (HSSI) method where the data are in the researcher. In previous HSSI works, there is a level of uncertainty regarding the use of the method in knowledge exploration. It seems that there is still a need for the development of methodological understanding, particularly in terms of those who favor the use of multiple participants in HSSI, as opposed to those who do not. In this article, I compared four studies to clarify HSSI’s utility in knowledge production for future use.

scholarly journals Designing Research With Qualitative Comparative Analysis (QCA): Approaches, Challenges, and Tools

2017 ◽  
Vol 49 (2) ◽  
pp. 356-386 ◽  
Author(s):  
Eva Thomann ◽  
Martino Maggetti
Keyword(s):  
Comparative Analysis ◽  
External Validity ◽  
Qualitative Comparative Analysis ◽  
Internal Validity ◽  
Specific Research ◽  
Extant Literature ◽  
Specific Mode ◽  
Research Context ◽  
Trade Offs ◽  
Conducting Research

Recent years have witnessed a host of innovations for conducting research with qualitative comparative analysis (QCA). Concurrently, important issues surrounding its uses have been highlighted. In this article, we seek to help users design QCA studies. We argue that establishing inference with QCA involves three intertwined design components: first, clarifying the question of external validity; second, ensuring internal validity; and third, explicitly adopting a specific mode of reasoning. We identify several emerging approaches to QCA rather than just one. Some approaches emphasize case knowledge, while others are condition oriented. Approaches emphasize either substantively interpretable or redundancy-free explanations, and some designs apply an inductive/explorative mode of reasoning, while others integrate deductive elements. Based on extant literature, we discuss issues surrounding inference with QCA and the tools available under different approaches to address these issues. We specify trade-offs and the importance of doing justice to the nature and goals of QCA in a specific research context.

scholarly journals High Fidelity Microsurgical Simulation: The Thiel Model and Evaluation Instrument

2018 ◽  
Vol 27 (2) ◽  
pp. 118-124 ◽  
Author(s):  
Andrei Odobescu ◽  
Isak Goodwin ◽  
Djamal Berbiche ◽  
Joseph BouMerhi ◽  
Patrick G. Harris ◽  
...  
Keyword(s):  
External Validity ◽  
Internal Validity ◽  
High Fidelity ◽  
Evaluation Instrument ◽  
Intra Class Correlation ◽  
Validity Measures ◽  
The Difference ◽  
Fidelity Model ◽  
Junior Residents ◽  
Internal And External Validity

Background: The Thiel embalmment method has recently been used in a number of medical simulation fields. The authors investigate the use of Thiel vessels as a high fidelity model for microvascular simulation and propose a new checklist-based evaluation instrument for microsurgical training. Methods: Thirteen residents and 2 attending microsurgeons performed video recorded microvascular anastomoses on Thiel embalmed arteries that were evaluated using a new evaluation instrument (Microvascular Evaluation Scale) by 4 fellowship trained microsurgeons. The internal validity was assessed using the Cronbach coefficient. The external validity was verified using regression models. Results: The reliability assessment revealed an excellent intra-class correlation of 0.89. When comparing scores obtained by participants from different levels of training, attending surgeons and senior residents (Post Graduate Year [PGY] 4-5) scored significantly better than junior residents (PGY 1-3). The difference between senior residents and attending surgeons was not significant. When considering microsurgical experience, the differences were significant between the advanced group and the minimal and moderate experience groups. The differences between minimal and moderate experience groups were not significant. Based on the data obtained, a score of 8 would translate into a level of microsurgical competence appropriate for clinical microsurgery. Conclusions: Thiel cadaveric vessels are a high fidelity model for microsurgical simulation. Excellent internal and external validity measures were obtained using the Microvascular Evaluation Scale (MVES).

scholarly journals A novel preference-informed complementary trial (PICT) design for clinical trial research influenced by strong patient preferences

Trials ◽  
2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Samina Ali ◽  
◽  
Gareth Hopkin ◽  
Naveen Poonai ◽  
Lawrence Richer ◽  
...  
Keyword(s):  
External Validity ◽  
Treatment Options ◽  
Internal Validity ◽  
Clinical Effectiveness ◽  
Drop Out ◽  
Innovative Design ◽  
Clinical Trial Research ◽  
Research Questions ◽  
Internal And External Validity ◽  
Treatment Population

Abstract Background Patients and their families often have preferences for medical care that relate to wider considerations beyond the clinical effectiveness of the proposed interventions. Traditionally, these preferences have not been adequately considered in research. Research questions where patients and families have strong preferences may not be appropriate for traditional randomized controlled trials (RCTs) due to threats to internal and external validity, as there may be high levels of drop-out and non-adherence or recruitment of a sample that is not representative of the treatment population. Several preference-informed designs have been developed to address problems with traditional RCTs, but these designs have their own limitations and may not be suitable for many research questions where strong preferences and opinions are present. Methods In this paper, we propose a novel and innovative preference-informed complementary trial (PICT) design which addresses key weaknesses with both traditional RCTs and available preference-informed designs. In the PICT design, complementary trials would be operated within a single study, and patients and/or families would be given the opportunity to choose between a trial with all treatment options available and a trial with treatment options that exclude the option which is subject to strong preferences. This approach would allow those with strong preferences to take part in research and would improve external validity through recruiting more representative populations and internal validity. Here we discuss the strengths and limitations of the PICT design and considerations for analysis and present a motivating example for the design based on the use of opioids for pain management for children with musculoskeletal injuries. Conclusions PICTs provide a novel and innovative design for clinical trials with more than two arms, which can address problems with existing preference-informed trial designs and enhance the ability of researchers to reflect shared decision-making in research as well as improving the validity of trials of topics with strong preferences.

Investigating the Youth Physical Activity Promotion Model: Internal Structure and External Validity Evidence for a Potential Measurement Model

2007 ◽  
Vol 19 (4) ◽  
pp. 420-435 ◽  
Author(s):  
David A. Rowe ◽  
Thomas D. Raedeke ◽  
Lenny D. Wiersma ◽  
Matthew T. Mahar
Keyword(s):  
Physical Activity ◽  
External Validity ◽  
Internal Validity ◽  
Physical Activity Promotion ◽  
Measurement Model ◽  
Measurement Properties ◽  
Validity Evidence ◽  
Potential Measurement ◽  
Activity Promotion ◽  
Youth Physical Activity

The purpose of the study was to investigate the measurement properties of questionnaires associated with the Youth Physical Activity Promotion (YPAP) model. Data were collected from 296 children in Grades 5–8 using several existing questionnaires corresponding to YPAP model components, a physical activity questionnaire, and 6 consecutive days of pedometer data. Internal validity of the questionnaires was tested using confirmatory factor analyses, and external validity was investigated via correlations with physical activity and body composition. Initial model fit of the questionnaires ranged from poor to very good. After item removal, all scales demonstrated good fit. Correlations with percentage body fat and objectively measured physical activity were low but in the theoretically predicted direction. The current study provides good internal validity evidence and acceptable external validity evidence for a brief set of questionnaire items to investigate the theoretical basis for the YPAP model.

scholarly journals Toward Experimental International Business

2015 ◽  
Vol 22 (4) ◽  
pp. 530-544 ◽  
Author(s):  
Arjen van Witteloostuijn
Keyword(s):  
International Business ◽  
External Validity ◽  
Design Methodology ◽  
Internal Validity ◽  
Cross Cultural ◽  
Web Based ◽  
Content Type ◽  
Experimental Laboratory ◽  
Experimental Management ◽  
Missed Opportunity

Purpose – The purpose of this paper is to argue that the time is ripe to establish a powerful tradition in Experimental International Business (IB). Probably due to what the Arjen van Witteloostuijn refers to as the external validity myth, experimental laboratory designs are underutilized in IB, which implies that the internal validity miracle of randomized experimentation goes largely unnoticed in this domain of the broader management discipline. Design/methodology/approach – In the following pages, the author explains why the author believes this implies a missed opportunity, providing arguments and examples along the way. Findings – Although an Experimental Management tradition has never really gained momentum, to the author, the lab experimental design has a very bright future in IB (and management at large). To facilitate the development of an Experimental IB tradition, initiating web-based tools would be highly instrumental. This will not only boost further progress in IB research, but will also increase the effectiveness and playfulness of IB teaching. Originality/value – Given the high potential of an Experimental IB, the Cross-Cultural and Strategic Management journal will offer a platform for such exciting and intriguing laboratory work, cumulatively contributing to the establishment of an Experimental IB tradition.

External Validity: The Next Step for Systematic Reviews?

2016 ◽  
Vol 41 (4) ◽  
pp. 283-325 ◽  
Author(s):  
Sarah A. Avellar ◽  
Jaime Thomas ◽  
Rebecca Kleinman ◽  
Emily Sama-Miller ◽  
Sara E. Woodruff ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
External Validity ◽  
Home Visiting ◽  
Evaluation Research ◽  
Internal Validity ◽  
Reporting Guidelines ◽  
Sample Characteristics ◽  
Statistical Representativeness

Background: Systematic reviews—which identify, assess, and summarize existing research—are usually designed to determine whether research shows that an intervention has evidence of effectiveness, rather than whether an intervention will work under different circumstances. The reviews typically focus on the internal validity of the research and do not consistently incorporate information on external validity into their conclusions. Objectives: In this article, we focus on how systematic reviews address external validity. Methods: We conducted a brief scan of 19 systematic reviews and a more in-depth examination of information presented in a systematic review of home visiting research. Results: We found that many reviews do not provide information on generalizability, such as statistical representativeness, but focus on factors likely to increase heterogeneity (e.g., numbers of studies or settings) and report on context. The latter may help users decide whether the research characteristics—such as sample demographics or settings—are similar to their own. However, we found that differences in reporting, such as which variables are included and how they are measured, make it difficult to summarize across studies or make basic determinations of sample characteristics, such as whether the majority of a sample was unemployed or married. Conclusion: Evaluation research and systematic reviews would benefit from reporting guidelines for external validity to ensure that key information is reported across studies.

External Validity Reconsidered

2011 ◽  
Author(s):  
Diana C. Mutz
Keyword(s):  
Political Science ◽  
Real World ◽  
External Validity ◽  
Observational Studies ◽  
Internal Validity ◽  
Conventional Wisdom ◽  
Observational Research ◽  
Convincing Case

This chapter talks about the significance of generalizability. Experimentalists often go to great lengths to argue that student or other convenience samples are not problematic in terms of external validity. Likewise, a convincing case for causality is often elusive with observational research, no matter how stridently one might argue to the contrary. The conventional wisdom is that experiments are widely valued for their internal validity, and experiments lack external validity. These assumptions are so widespread as to go without question in most disciplines, particularly those emphasizing external validity, such as political science and sociology. But observational studies, such as surveys, are still supposed to be better for purposes of maximizing external validity because this method allows studying people in real world settings.

Evaluation as a Systemic Component of Organizational Change Efforts

1978 ◽  
Vol 22 (1) ◽  
pp. 404-408 ◽  
Author(s):  
Kerry A. Bunker
Keyword(s):  
Organizational Change ◽  
External Validity ◽  
Internal Validity ◽  
Organizational Training ◽  
Feedback Systems ◽  
Systems Model ◽  
Potential Benefits ◽  
Validity Research ◽  
Systemic Component ◽  
Change Efforts

A cyclical-feedback systems model is proposed as a format for structuring and utilizing the results of evaluative investigations of organizational change efforts. The issues of internal validity, external validity, research design and results utilization are discussed; and an organizational training evaluation example is presented to demonstrate the feasibility, practicality and potential benefits of the approach.

What conclusions should we draw from the data?

1995 ◽  
Vol 84 (02) ◽  
pp. 95-101 ◽  
Author(s):  
Andrew Vickers
Keyword(s):  
Cohort Studies ◽  
External Validity ◽  
Large Scale ◽  
Critical Appraisal ◽  
Internal Validity ◽  
Research Process ◽  
Clinical Value ◽  
Blind Trials

AbstractCritical appraisal of a scientific trial involves deciding on its internal validity— whether the hypothesis has been correctly accepted or rejected—and its external validity—the extent to which the trial's findings can be generalized. Discourse on homoeopathic research has focused on the former at the expense of the latter and an analysis of homoeopathic research demonstrates that it has low external validity. One solution would be to split the research process in two. Large scale, triple-blind trials could be used to determine the extent to which the action of homoeopathy may be explained by placebo. Importantly, no assessment of external validity would be made. Audit and cohort studies could then be used to examine questions usually associated with external validity, such as the conditions most suitable for treatment and the long-term clinical value of homoeopathy.

scholarly journals Using the RE-AIM framework to evaluate internal and external validity of mobile phone–based interventions in diabetes self-management education and support

2020 ◽  
Vol 27 (6) ◽  
pp. 946-956 ◽  
Author(s):  
Yilin Yoshida ◽  
Sonal J Patil ◽  
Ross C Brownson ◽  
Suzanne A Boren ◽  
Min Kim ◽  
...  
Keyword(s):  
External Validity ◽  
Management Education ◽  
Data Extraction ◽  
Internal Validity ◽  
Self Management ◽  
Future Research ◽  
Limited Information ◽  
Quality Of Reporting ◽  
Screening Process ◽  
Internal And External Validity

Abstract Objective We evaluated the extent to which studies that tested short message service (SMS)– and application (app)-based interventions for diabetes self-management education and support (DSMES) report on factors that inform both internal and external validity as measured by the RE-AIM (Reach, Efficacy/Effectiveness, Adoption, Implementation, and Maintenance) framework. Materials and Methods We systematically searched PubMed, Embase, Web of Science, CINAHL (Cumulative Index of Nursing and Allied Health Literature), and IEEE Xplore Digital Library for articles from January 1, 2009, to February 28, 2019. We carried out a multistage screening process followed by email communications with study authors for missing or discrepant information. Two independent coders coded eligible articles using a 23-item validated data extraction tool based on the RE-AIM framework. Results Twenty studies (21 articles) were included in the analysis. The comprehensiveness of reporting on the RE-AIM criteria across the SMS- and app-based DSMES studies was low. With respect to internal validity, most interventions were well described and primary clinical or behavioral outcomes were measured and reported. However, gaps exist in areas of attrition, measures of potential negative outcomes, the extent to which the protocol was delivered as intended, and description on delivery agents. Likewise, we found limited information on external validity indicators across adoption, implementation, and maintenance domains. Conclusions Reporting gaps were found in internal validity but more so in external validity in the current SMS- and app-based DSMES literature. Because most studies in this review were efficacy studies, the generalizability of these interventions cannot be determined. Future research should adopt the RE-AIM dimensions to improve the quality of reporting and enhance the likelihood of translating research to practice.

Sign in / Sign up

Export Citation Format

Share Document