scholarly journals Crossing the Red Line? Empirical Evidence and Useful Recommendations on Questionable Research Practices among Business Scholars

2021 ◽  
Author(s):  
Hengky Latan ◽  
Charbel Jose Chiappetta Jabbour ◽  
Ana Beatriz Lopes de Sousa Jabbour ◽  
Murad Ali
Keyword(s):  
Empirical Evidence ◽  
Research Practices ◽  
Academic Leaders ◽  
Questionable Research Practices ◽  
Main Research ◽  
Admission Rates ◽  
Level Of Use ◽  
Research Findings ◽  
Fear Of Rejection

AbstractAcademic leaders in management from all over the world—including recent calls by the Academy of Management Shaw (Academy of Management Journal 60(3): 819–822, 2017)—have urged further research into the extent and use of questionable research practices (QRPs). In order to provide empirical evidence on the topic of QRPs, this work presents two linked studies. Study 1 determines the level of use of QRPs based on self-admission rates and estimated prevalence among business scholars in Indonesia. It was determined that if the level of QRP use identified in Study 1 was quite high, Study 2 would be conducted to follow-up on this result, and this was indeed the case. Study 2 examines the factors that encourage and discourage the use of QRPs in the sample analyzed. The main research findings are as follows: (a) in Study 1, we found the self-admission rates and estimated prevalence of business scholars’ involvement in QRPs to be quite high when compared with studies conducted in other countries and (b) in Study 2, we found pressure for publication from universities, fear of rejection of manuscripts, meeting the expectations of reviewers, and available rewards to be the main reasons for the use of QRPs in Indonesia, whereas (c) formal sanctions and prevention efforts are factors that discourage QRPs. Recommendations for stakeholders (in this case, reviewers, editors, funders, supervisors, chancellors and others) are also provided in order to reduce the use of QRPs.

scholarly journals The effect of preregistration on trust in empirical research findings: results of a registered report

2020 ◽  
Vol 7 (4) ◽  
pp. 181351 ◽  
Author(s):  
Sarahanne M. Field ◽  
E.-J. Wagenmakers ◽  
Henk A. L. Kiers ◽  
Rink Hoekstra ◽  
Anja F. Ernst ◽  
...  
Keyword(s):  
Pilot Study ◽  
Empirical Evidence ◽  
Empirical Research ◽  
Strong Evidence ◽  
Bayes Factors ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Research Findings ◽  
Research Questions ◽  
Research Initiatives

The crisis of confidence has undermined the trust that researchers place in the findings of their peers. In order to increase trust in research, initiatives such as preregistration have been suggested, which aim to prevent various questionable research practices. As it stands, however, no empirical evidence exists that preregistration does increase perceptions of trust. The picture may be complicated by a researcher's familiarity with the author of the study, regardless of the preregistration status of the research. This registered report presents an empirical assessment of the extent to which preregistration increases the trust of 209 active academics in the reported outcomes, and how familiarity with another researcher influences that trust. Contrary to our expectations, we report ambiguous Bayes factors and conclude that we do not have strong evidence towards answering our research questions. Our findings are presented along with evidence that our manipulations were ineffective for many participants, leading to the exclusion of 68% of complete datasets, and an underpowered design as a consequence. We discuss other limitations and confounds which may explain why the findings of the study deviate from a previously conducted pilot study. We reflect on the benefits of using the registered report submission format in light of our results. The OSF page for this registered report and its pilot can be found here: http://dx.doi.org/10.17605/OSF.IO/B3K75 .

scholarly journals The Effect of Preregistration on Trust in Empirical Research Findings

2018 ◽  
Author(s):  
Sarahanne Miranda Field ◽  
Eric-Jan Wagenmakers ◽  
Henk Kiers ◽  
Rink Hoekstra ◽  
Anja Ernst ◽  
...  
Keyword(s):  
Empirical Evidence ◽  
Empirical Research ◽  
Scientific Literature ◽  
Subjective Assessment ◽  
Primary Role ◽  
Research Practices ◽  
Research Results ◽  
Questionable Research Practices ◽  
Research Findings

The crisis of confidence has played a primary role in undermining the trust researchers place in the findings of their peers, and our beliefs about the credibility of research results. Thus, the importance of increasing trust in credible reported research is paramount. Incentives such as preregistration are aimed at establishing a more trustworthy scientific literature, in that they help prevent various questionable research practices. As it stands, however, no empirical evidence exists demonstrating that preregistration does increase trust. Indeed, the objective merits of preregistration greatly lose in value if a researcher's subjective assessment of the value of preregistration does not align. Additionally, the picture may be complicated by a researcher's familiarity with the author of the study, regardless of the preregistration status of the research. The following proposal describes how we aim to test the extent to which preregistration increases the trust of participants in the reported outcomes. We also aim to assess how familiarity with another researcher might influence trust. We expect that preregistration increases researchers' trust in findings, relative to no preregistration, and that registered reporting increases trust more than preregistration alone. We also expect that familiarity enhances trust judgments to some extent, however we do not have specific expectations regarding the nature of this effect. We therefore include familiarity as an exploratory effect in our analyses. The OSF page for this registered report proposal and its pilot can be found here: http://dx.doi.org/10.17605/OSF.IO/B3K75

scholarly journals Research practices and statistical reporting quality in 250 economic psychology master's theses: a meta-research investigation

2019 ◽  
Vol 6 (12) ◽  
pp. 190738 ◽  
Author(s):  
Jerome Olsen ◽  
Johanna Mosen ◽  
Martin Voracek ◽  
Erich Kirchler
Keyword(s):  
Statistical Power ◽  
Testing Effect ◽  
Reporting Quality ◽  
Open Science ◽  
Sample Size Determination ◽  
Effect Sizes ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Standardized Effect Sizes ◽  
Research Findings

The replicability of research findings has recently been disputed across multiple scientific disciplines. In constructive reaction, the research culture in psychology is facing fundamental changes, but investigations of research practices that led to these improvements have almost exclusively focused on academic researchers. By contrast, we investigated the statistical reporting quality and selected indicators of questionable research practices (QRPs) in psychology students' master's theses. In a total of 250 theses, we investigated utilization and magnitude of standardized effect sizes, along with statistical power, the consistency and completeness of reported results, and possible indications of p -hacking and further testing. Effect sizes were reported for 36% of focal tests (median r = 0.19), and only a single formal power analysis was reported for sample size determination (median observed power 1 − β = 0.67). Statcheck revealed inconsistent p -values in 18% of cases, while 2% led to decision errors. There were no clear indications of p -hacking or further testing. We discuss our findings in the light of promoting open science standards in teaching and student supervision.

An evaluation of knowledge management tools: Part 2 – managing knowledge flows and enablers

2014 ◽  
Vol 18 (6) ◽  
pp. 1101-1126 ◽  
Author(s):  
Peter Massingham
Keyword(s):  
Action Research ◽  
Empirical Evidence ◽  
Organisational Change ◽  
Knowledge Flows ◽  
Organisational Performance ◽  
Learning Organisation ◽  
Content Type ◽  
Research Findings ◽  
And Performance

Purpose – This paper aims to evaluate a range of best practice knowledge management (KM) ideas used to manage knowledge flows and enablers. In total, four KM toolkits and 23 KM tools were tested over a five-year period (2008-2013), as part of a large-scale longitudinal change project. Each tool was assessed against an evaluative framework designed to test criticisms of KM: strategy, implementation and performance. The results provide empirical evidence about what KM tools work and which do not and why, and outcomes for practitioners, researchers and consultants. Design/methodology/approach – This paper presents a summary of the findings of a large Australian Research Council (ARC) Linkage Project grant in the period of 2008-2013. The case study organisation (CSO) was a large public sector department, which faced the threat of lost capability caused by its ageing workforce and knowledge loss. The project aimed to solve this problem by minimising its impact via achieving learning organisation capacity. The CSO participating in the study was selected because it was a knowledge-intensive organisation, with an ageing workforce. All 150 engineering and technical staff at the CSO were invited to participate, including management and staff. An action research methodology was used. Findings – The results provide empirical evidence that KM can be used to manage knowledge flows and enablers. The highest rating toolkit was knowledge preservation, followed by knowledge usage. The most value was created by using KM to provide “why context” to structural capital (e.g. reports, databases, policies) (meta-data) and to create opportunities to reflect on experience and share the learning outcomes (peer assists and after action reviews). The results tended to support criticism that KM is difficult to implement and identified the main barriers as participation located at the tactical action research level, i.e. why is this useful? Evidence that KM works was found in progress towards learning organisation capacity and in practical outcomes. Research limitations/implications – The action research cycle and learning flows provide opportunities to examine barriers to KM implementation. The research also presents opportunities for further research to examine the findings in other organisational and industry settings, for example, the relationship between the KM toolkits and organisational change and performance, presents an important area for further research. Researchers might also consider some of the toolkits which rated poorly, e.g. knowledge sharing, and challenge these findings, perhaps selecting different KS tools for testing. The paper has limitations. It is based on a single case study organisation, offset, to some degree, by the longitudinal nature of the empirical evidence. It is ambitious and the findings may be controversial. However, the depth of the study and its findings provide rare longitudinal empirical evidence about KM and the results should be useful for practitioners, researchers and consultants. Practical implications – For practitioners, the research findings provide management with an evaluative framework to use when making decisions regarding KM. The findings provide discussion of KM toolkits and tools that may be used to manage knowledge flows and enablers. In addition to the discussion of each tool, there is analysis of what works and what does not and why, barriers to implementation as well as explanation of their impact on organisational change and performance, and a scorecard to guide toolkit choices. This method should allow managers to make sensible decisions about KM. Originality/value – The paper addresses criticisms of KM by examining the KM toolkits within the context of whether knowledge can be managed, implementation barriers may be addressed and improved organisational performance can be demonstrated. This approach allows generalisability of the findings to enable others to apply the research findings in their organisational contexts. The outcome is three sets of guidelines: strategy: which KM tools work; implementation: addressing barriers; and organisational performance: how to measure value. In doing so, the paper provides a systematic framework for evaluating KM tools. It also provides a rare opportunity to present empirical evidence gathered over a five-year longitudinal study.

scholarly journals (De)Constructing the Self and Other in Veer Zaara

2019 ◽  
Vol 3 (1) ◽  
pp. 44
Author(s):  
Shuri Mariasih Gietty Tambunan
Keyword(s):  
Research Question ◽  
The Self ◽  
Main Research ◽  
Self And Other ◽  
Research Findings ◽  
Cultural Texts ◽  
Similarities And Differences ◽  
Multiple Challenges ◽  
And Gender

<p class="AbstractText">Films as cultural texts articulate the politics of everyday lives and one of the issues often depicted is on nationalism. The chosen case study for this article is <em>Veer Zaara </em>(2004), a romantic Bollywood movie telling us the story of two lovers from India and Pakistan who have to undergone multiple challenges to be together. The conflicts represent the ongoing and completely unresolved sibling rivalry between the two countries particularly after the partition in 1947. The main research question is how the film depicts the process of self-identification from the Indian characters by looking at the “cultural similarities and differences” compared to the Pakistani character while representing the effort of drawing a boundary between India and Pakistan? Research findings who that there are three dominant representational elements (space, religion and gender) in which the film with its authority select what forms of representation it would present concerning each country. In doing so, the film is making sure that India is identifying itself as a nation which is different from Pakistan or by drawing the boundary of India as the self and Pakistan as the other.</p>

Finnish Students’ Dedication to and Interaction in Communicative Oral Practice in Foreign Language Classroom

2016 ◽  
Vol 7 (6) ◽  
pp. 1057
Author(s):  
Pirjo Harjanne
Keyword(s):  
Foreign Language ◽  
Qualitative Content Analysis ◽  
Good Deal ◽  
Ethnographic Case Study ◽  
Main Research ◽  
Cooperative Scheme ◽  
Language Classroom ◽  
Research Findings ◽  
Oral Practice

The aim of this study is to gain a deeper understanding of how Finnish-speaking students’ communicative oral practice in a foreign language, Swedish , is carried out through cooperative scheme-based and elaboration tasks in the language classroom. The specific focus is on the students’ dedication and participatory interaction. The study is carried out as a didactically oriented micro-ethnographic case study, in which the teacher acts as a researcher of her own teaching. The data, gathered through tape recordings of the students’ oral practice, are analysed through qualitative content analysis methods supplemented with some quantifications. The main research findings are that a good deal of dedication to the oral practice, as well as cooperation, and interactive and self-generated communication in Swedish are realised. Many students’ use of L1, Finnish, especially when creating intersubjectivity and in scaffolding, is also evident.

Publication bias and statistical power in gerontological psychology

2018 ◽  
Author(s):  
Christopher Brydges
Keyword(s):  
Publication Bias ◽  
Statistical Power ◽  
Research Practices ◽  
P Values ◽  
Questionable Research Practices ◽  
Gerontological Research ◽  
Article Type ◽  
Research Findings ◽  
Psychological Journal ◽  
Published Research

Objectives: Research has found evidence of publication bias, questionable research practices (QRPs), and low statistical power in published psychological journal articles. Isaacowitz’s (2018) editorial in the Journals of Gerontology Series B, Psychological Sciences called for investigation of these issues in gerontological research. The current study presents meta-research findings based on published research to explore if there is evidence of these practices in gerontological research. Method: 14,481 test statistics and p values were extracted from articles published in eight top gerontological psychology journals since 2000. Frequentist and Bayesian caliper tests were used to test for publication bias and QRPs (specifically, p-hacking and incorrect rounding of p values). A z-curve analysis was used to estimate average statistical power across studies.Results: Strong evidence of publication bias was observed, and average statistical power was approximately .70 – below the recommended .80 level. Evidence of p-hacking was mixed. Evidence of incorrect rounding of p values was inconclusive.Discussion: Gerontological research is not immune to publication bias, QRPs, and low statistical power. Researchers, journals, institutions, and funding bodies are encouraged to adopt open and transparent research practices, and using Registered Reports as an alternative article type to minimize publication bias and QRPs, and increase statistical power.

Questionable Research Practices: How to Recognize and Avoid Them

2020 ◽  
Vol 32 (4) ◽  
pp. 183-190
Author(s):  
William Newton Suter
Keyword(s):  
Scientific Progress ◽  
Research Integrity ◽  
Home Health ◽  
Research Practices ◽  
Scientific Rigor ◽  
Questionable Research Practices ◽  
Nursing Science ◽  
Health Nurse ◽  
Cast Doubt ◽  
Research Findings

This article focuses on questionable research practices (QRPs) that bias findings and conclusions. QRPs cast doubt on the credibility of research findings in home health and nursing science in general. They assault the research integrity of all researchers to the extent they are permitted to exist at all. Each QRP is defined via bundles of specific research behaviors with unifying labels that include deceptive mirages and phantom sharpshooters among others. These questionable behaviors are described in ways that enhance research understanding and enable QRP avoidance by careful home health nurse researchers using higher standards of scientific rigor. QRPs impede scientific progress by generating false conclusions. They threaten the validity and dependability of scientific research and confuse other researchers who practice rigorous science and maintain integrity. QRPs also clog the literature with studies that cannot be replicated. When researchers engage in QRPs at the expense of rigor, overall trust in the scientific knowledge base erodes.

scholarly journals The Use of Questionable Research Practices to Survive in Academia Examined With Expert Elicitation, Prior-Data Conflicts, Bayes Factors for Replication Effects, and the Bayes Truth Serum

2021 ◽  
Vol 12 ◽  
Author(s):  
Rens van de Schoot ◽  
Sonja D. Winter ◽  
Elian Griffioen ◽  
Stephan Grimmelikhuijsen ◽  
Ingrid Arts ◽  
...  
Keyword(s):  
Bayesian Methods ◽  
Open Science ◽  
Expert Elicitation ◽  
Bayes Factors ◽  
Published Data ◽  
Research Practices ◽  
Academic Leaders ◽  
Questionable Research Practices ◽  
Unethical Research ◽  
Science Framework

The popularity and use of Bayesian methods have increased across many research domains. The current article demonstrates how some less familiar Bayesian methods can be used. Specifically, we applied expert elicitation, testing for prior-data conflicts, the Bayesian Truth Serum, and testing for replication effects via Bayes Factors in a series of four studies investigating the use of questionable research practices (QRPs). Scientifically fraudulent or unethical research practices have caused quite a stir in academia and beyond. Improving science starts with educating Ph.D. candidates: the scholars of tomorrow. In four studies concerning 765 Ph.D. candidates, we investigate whether Ph.D. candidates can differentiate between ethical and unethical or even fraudulent research practices. We probed the Ph.D.s’ willingness to publish research from such practices and tested whether this is influenced by (un)ethical behavior pressure from supervisors or peers. Furthermore, 36 academic leaders (deans, vice-deans, and heads of research) were interviewed and asked to predict what Ph.D.s would answer for different vignettes. Our study shows, and replicates, that some Ph.D. candidates are willing to publish results deriving from even blatant fraudulent behavior–data fabrication. Additionally, some academic leaders underestimated this behavior, which is alarming. Academic leaders have to keep in mind that Ph.D. candidates can be under more pressure than they realize and might be susceptible to using QRPs. As an inspiring example and to encourage others to make their Bayesian work reproducible, we published data, annotated scripts, and detailed output on the Open Science Framework (OSF).

scholarly journals MODEL PENDIDIKAN MULTIKULTURAL DI ‘SEKOLAH PEMBAURAN’ MEDAN

2014 ◽  
Vol 3 (3) ◽  
Author(s):  
Saliman Saliman ◽  
Taat Wulandari ◽  
Mukminan Mukminan
Keyword(s):  
Multicultural Education ◽  
School Leadership ◽  
Main Research ◽  
Study Approach ◽  
Interactive Analysis ◽  
Whole School ◽  
Research Findings ◽  
Whole School Approach ◽  
North Sumatra

Abstrak: Penelitian ini bertujuan untuk mendeskripsikan model pendidikan multikultural di Sekolah Pembauran Medan. Penelitian menggunakan metode kualitatif dengan pendekatan studi kasus. Penelitian dilaksanakan di Sekolah Pembauran Medan. Pengumpulan data dilakukan dengan teknik observasi, wawancara, dan dokumentasi, sedang kredibilitas data diperoleh lewat triangulasi metode. Instrumen penelitian adalah peneliti sendiri dengan menggunakan bantuan pedoman observasi dan wawancara. Analisis data dilakukan dengan menggunakan teknik analisis interaktif versi Miles dan Huberman. Hasil penelitian menunjukkan di Sekolah Pembauran Medan merupakan nama yang digunakan untuk menyebut sekolah di bawah Yayasan Perguruan Sultan Iskandar Muda menggunakan Whole School Approach yang meliputi visi dan kebijakan sekolah, kepemimpinan dan manajemen, kapasitas dan kultur, aktivitas peserta didik, kolaborasi dengan masyarakat luas, serta kurikulum dan pengajaran. Kata Kunci: Sekolah Pembauran, pendidikan multikultural, whole school approach AMULTICULTURAL EDUCATION MODEL IN SEKOLAH PEMBAURAN MEDAN Abstract: This study was aimed to describe a model of the multicultural education implemented in Sekolah Pembauran Medan, North Sumatra. This study employed the qualitative approach using the case study approach. The study was conducted in Sekolah Pembauran Medan. The data collection techniques consisted of observations, interviews and documentation. The main research instrument was the researchers themselves supported by observation sheets and interview guidelines. The data analysis was performed using the interactive analysis developed by Miles and Huberman. The research findings showed that Sekolah Pembauran Medan is the name used to refer to the school under the Education Foundation of Sultan Iskandar Muda using the Whole School Approach which includes: the vision and policies of the school; leadership and management; capacity and culture; activities of learners; collaboration with wider communities; and curriculum and instruction. Keywords: Sekolah Pembauran, multicultural education, Whole School Approach

Sign in / Sign up

Export Citation Format

Share Document