scholarly journals Preregistration in diverse contexts: a preregistration template for the application of cognitive models

2021 ◽  
Vol 8 (10) ◽  
Author(s):  
Sophia Crüwell ◽  
Nathan J. Evans
Keyword(s):  
Degrees Of Freedom ◽  
Cognitive Model ◽  
Open Science ◽  
Experimental Manipulation ◽  
Mathematical Psychology ◽  
Measurement Tool ◽  
Current Debate ◽  
Cognitive Modelling ◽  
Model Application ◽  
Diverse Field

In recent years, open science practices have become increasingly popular in psychology and related sciences. These practices aim to increase rigour and transparency in science as a potential response to the challenges posed by the replication crisis. Many of these reforms—including the increasingly used preregistration —have been designed for purely experimental work that tests straightforward hypotheses with standard inferential statistical analyses, such as assessing whether an experimental manipulation has an effect on a variable of interest. But psychology is a diverse field of research. The somewhat narrow focus of the prevalent discussions surrounding and templates for preregistration has led to debates on how appropriate these reforms are for areas of research with more diverse hypotheses and more intricate methods of analysis, such as cognitive modelling research within mathematical psychology. Our article attempts to bridge the gap between open science and mathematical psychology, focusing on the type of cognitive modelling that Crüwell et al. (Crüwell S, Stefan AM, Evans NJ. 2019 Robust standards in cognitive science. Comput. Brain Behav. 2 , 255–265) labelled model application , where researchers apply a cognitive model as a measurement tool to test hypotheses about parameters of the cognitive model. Specifically, we (i) discuss several potential researcher degrees of freedom within model application, (ii) provide the first preregistration template for model application and (iii) provide an example of a preregistered model application using our preregistration template. More broadly, we hope that our discussions and concrete proposals constructively advance the mostly abstract current debate surrounding preregistration in cognitive modelling, and provide a guide for how preregistration templates may be developed in other diverse or intricate research contexts.

scholarly journals Preregistration in Complex Contexts: A Preregistration Template for the Application of Cognitive Models

2019 ◽  
Author(s):  
Sophia Crüwell ◽  
Nathan J. Evans
Keyword(s):  
Degrees Of Freedom ◽  
Cognitive Model ◽  
Open Science ◽  
Experimental Manipulation ◽  
Mathematical Psychology ◽  
Measurement Tool ◽  
Cognitive Modelling ◽  
Model Application ◽  
Diverse Field ◽  
Replication Crisis

In recent years, open science practices have become increasingly popular in psychology and related sciences. These practices aim to increase rigour and transparency in science as a potential response to the challenges posed by the replication crisis. Many of these reforms -- including the highly influential preregistration -- have been designed for experimental work that tests simple hypotheses with standard statistical analyses, such as assessing whether an experimental manipulation has an effect on a variable of interest. However, psychology is a diverse field of research, and the somewhat narrow focus of the prevalent discussions surrounding and templates for preregistration has led to debates on how appropriate these reforms are for areas of research with more diverse hypotheses and more complex methods of analysis, such as cognitive modelling research within mathematical psychology. Our article attempts to bridge the gap between open science and mathematical psychology, focusing on the type of cognitive modelling that Crüwell, Stefan, & Evans (2019) labelled model application, where researchers apply a cognitive model as a measurement tool to test hypotheses about parameters of the cognitive model. Specifically, we (1) discuss several potential researcher degrees of freedom within model application, (2) provide the first preregistration template for model application, and (3) provide an example of a preregistered model application using our preregistration template. More broadly, we hope that our discussions and proposals constructively advance the debate surrounding preregistration in cognitive modelling, and provide a guide for how preregistration templates may be developed in other diverse or complex research contexts.

scholarly journals Robust Standards in Cognitive Science

2019 ◽  
Author(s):  
Sophia Crüwell ◽  
Angelika Stefan ◽  
Nathan J. Evans
Keyword(s):  
Open Science ◽  
Mathematical Psychology ◽  
General Level ◽  
Cognitive Modelling ◽  
Local Consistency ◽  
Science Practices ◽  
Fine Grained ◽  
Different Types ◽  
Psychology Community ◽  
Modelling Process

Recent discussions within the mathematical psychology community have focused on how Open Science practices may apply to cognitive modelling. Lee et al. (2019) sketched an initial approach for adapting Open Science practices that have been developed for experimental psychology research to the unique needs of cognitive modelling. While we welcome the general proposal of Lee et al. (2019), we believe a more fine-grained view is necessary to accommodate the adoption of Open Science practices in the diverse areas of cognitive modelling. Firstly, we suggest a categorisation for the diverse types of cognitive modelling, which we argue will allow researchers to more clearly adapt Open Science practices to different types of cognitive modelling. Secondly, we consider the feasibility and usefulness of preregistration and lab notebooks for each of these categories, and address potential objections to preregistration in cognitive modelling. Finally, we separate several cognitive modelling concepts that we believe Lee et al. (2019) conflated, which should allow for greater consistency and transparency in the modelling process. At a general level, we propose a framework that emphasises local consistency in approaches while allowing for global diversity in modelling practices.

scholarly journals Published patterns of spin in biomedical literature: a protocol for a meta-research study

BMJ Open ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. e043784
Author(s):  
Naichuan Su ◽  
Michiel van der Linden ◽  
Geert JMG van der Heijden ◽  
Stefan Listl ◽  
Stefan Schandelmaier ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Scientific Journal ◽  
Open Science ◽  
Biomedical Literature ◽  
Measurement Tool ◽  
Delphi Consensus ◽  
Reporting Practices ◽  
Literature Reviews ◽  
Science Framework ◽  
Meta Analyses

IntroductionSpin is defined as reporting practices that distort the interpretation of results and create misleading conclusions by suggesting more favourable results. Such unjustifiable and misleading misrepresentation may negatively influence the development of further studies, clinical practice and healthcare policies. Spin manifests in various patterns in different sections of publications (titles, abstracts and main texts). The primary aim of this study is to identify reported spin patterns and assess the prevalence of spin in general, and the prevalence of spin patterns reported in biomedical literature based on previously published systematic reviews and literature reviews on spin.Methods and analysisPubMed, EMBASE and SCOPUS will be searched to identify systematic or literature reviews on spin in biomedicine. To improve the comprehensiveness of the search, the snowballing method will be used to broaden the search. The data on spin-related outcomes and characteristics of the included studies will be extracted. The methodological quality of the included studies will be assessed with selective items of the A MeaSurement Tool to Assess systematic Reviews-2 checklist. A new classification scheme for spin patterns will be developed if the classifications of spin patterns identified in the included studies vary. The prevalence of spin and spin patterns will be pooled based on meta-analyses if the classification schemes for spin are comparable across included studies. Otherwise, the prevalence will be described qualitatively. The seriousness of spin patterns will be assessed based on a Delphi consensus study.Ethics and disseminationThis study has been approved by the Academic Centre for Dentistry Amsterdam Ethics Review Committee (2020250). The study will be submitted to a peer-reviewed scientific journal.RegistrationOpen Science Framework: osf.io/hzv6e

scholarly journals Making the black box transparent: A template and tutorial for (pre-)registration of studies using Experience Sampling Methods (ESM)

2019 ◽  
Author(s):  
Olivia J Kirtley ◽  
Ginette Lafit ◽  
Robin Achterhof ◽  
Anu Pauliina Hiekkaranta ◽  
Inez Myin-Germeys
Keyword(s):  
Degrees Of Freedom ◽  
Sampling Method ◽  
Experience Sampling ◽  
Open Science ◽  
Science Practices ◽  
Methodological Rigor ◽  
Assessment Techniques ◽  
Ecological Momentary ◽  
Momentary Assessment ◽  
Experience Sampling Methods

A growing interest in understanding complex and dynamic psychological processes as they occur in everyday life has led to an increase in studies using Ambulatory Assessment techniques, including the Experience Sampling Method (ESM) and Ecological Momentary Assessment (EMA). There are, however, numerous “forking paths” and researcher degrees of freedom, even beyond those typically encountered with other research methodologies. Whilst a number of researchers working with ESM techniques are actively engaged in efforts to increase the methodological rigor and transparency of such research, currently, there is little routine implementation of open science practices in ESM research. In the current paper, we discuss the ways in which ESM research is especially vulnerable to threats to transparency, reproducibility and replicability. We propose that greater use of (pre-)registration, a cornerstone of open science, may address some of these threats to the transparency of ESM research. (Pre-)registration of ESM research is not without challenges, including model selection, accounting for potential model convergence issues and the use of pre-existing datasets. As these may prove to be significant barriers to (pre-)registration for ESM researchers, we also discuss ways of overcoming these challenges and of documenting them in a (pre-)registration. A further challenge is that current general templates do not adequately capture the unique features of ESM. Here we present a (pre-)registration template for ESM research, adapted from the original Pre-Registration Challenge (Mellor et al., 2019) and pre-registration of pre-existing data (van den Akker et al., 2020) templates, and provide examples of how to complete this.

scholarly journals Researcher Degrees of Freedom in the Psychology of Religion

2018 ◽  
Author(s):  
Sarah Jane Charles ◽  
James Edward Bartlett ◽  
Kyle J. Messick ◽  
Thomas Joseph Coleman ◽  
Alex Uzdavines
Keyword(s):  
Systematic Review ◽  
Degrees Of Freedom ◽  
Psychology Of Religion ◽  
Open Science ◽  
Science Practices ◽  
Replication Crisis ◽  
The Way

There is a push in psychology toward more transparent practices, stemming partially as a response to the replication crisis. We argue that the psychology of religion should help lead the way toward these new, more transparent practices to ensure a robust and dynamic subfield. One of the major issues that proponents of Open Science practices hope to address is researcher degrees of freedom (RDF). We pre-registered and conducted a systematic review of the 2017 issues from three psychology of religion journals. We aimed to identify the extent to which the psychology of religion has embraced Open Science practices and the role of RDF within the subfield. We found that many of the methodologies that help to increase transparency, such as pre-registration, have yet to be adopted by those in the subfield. In light of these findings, we present recommendations for addressing the issue of transparency in the psychology of religion and outline how to move toward these new Open Science practices.

Active Man Machine Interface for Advanced Rotorcraft

1993 ◽  
Vol 37 (15) ◽  
pp. 1032-1032
Author(s):  
Monica Z. Weiland ◽  
Brian A. Convery ◽  
Allen L. Zaklad ◽  
Wayne W. Zachary ◽  
Clarence A. Fry ◽  
...  
Keyword(s):  
Information Management ◽  
Cognitive Model ◽  
Software Tools ◽  
Cognitive Modelling ◽  
Context Sensitive ◽  
The Future ◽  
Modelling Methodology ◽  
Machine Interface ◽  
Man Machine Interface

The proliferation of digital avionic information presented to pilots has produced a critical need for intelligent avionic information management, particularly in the area of Caution, Warning, and Advisory (CWA) systems. This demonstration illustrates the role of an Active Man Machine Interface (AMMI) in the context of CWA systems in rotorcraft of the future. The basis of the AMMI's intelligence demonstrated here is provided by a cognitive model that 1) prunes the alert stream to only those messages that have meaning to the pilot depending on the tactical context, and 2) provide context-sensitive advice on the basis of the tactical context. The CWA AMMI is currently being designed using COGNET, an cognitive modelling methodology (Zachary, 1989), and implemented using BATON, a set of software tools used to implement and embed COGNET models into existing systems (Zachary et al, 1991).

Listening to the voice of the customer in the hospitality industry: Kano model application

2018 ◽  
Vol 10 (4) ◽  
pp. 436-448 ◽  
Author(s):  
Margarita Zobnina ◽  
Aleksandr Rozhkov
Keyword(s):  
Customer Satisfaction ◽  
Tourism Industry ◽  
Measurement Tool ◽  
Hotel Management ◽  
Kano Model ◽  
Model Application ◽  
Content Type ◽  
Depth Analysis ◽  
Hospitality And Tourism ◽  
Hospitality And Tourism Industry

Purpose This paper aims to discuss the customer satisfaction drivers of Russian tourists choosing hotels in Europe. Design/methodology/approach The study is focused on tangible aspects of the hotel service product adopted from the European Hotel Guest Satisfaction Index by J.D. Power. Research methodology is based on the Kano model that enables satisfaction driver classification based on the level of their impact. Data collection was conducted via online panel representative for 1 million+ Russian cities, totaling 1,238 respondents. Findings This paper reveals groups of customer satisfaction drivers by their impact from attractive to expected and indifferent, as well as customer preference profile by age, overall travel experience and trip purpose (recreational, sightseeing or active tourism). Research limitations/implications This paper focuses on the tangible attributes of hotel experience; the survey sample composed of Russian tourists that evaluated their satisfaction with hotels in Europe. Practical implications As a result of the study, the authors test Kano model application in the hospitality and tourism industry, providing hotel managers with an advanced yet easy to use customer satisfaction measurement tool. Also, the authors demonstrate substantial differences in customer satisfaction drivers by groups that can be used to plan product development. “Expected” and “one-dimensional” groups of product features would cause customer dissatisfaction if missing and should be the first priority of management. Interestingly after a certain level, “expected” factors have no marginal value, so their improvement by the hotel management should be limited. On the other hand, “attractive” factors boost customer satisfaction while present but with no negative impact if they are absent; that makes this group the second priority for hotel management. Factor distribution by group is different for various customer segments that can also be taken into consideration when designing the marketing communications of a hotel. Originality/value In this research, the authors use the Kano model to identify customer satisfaction drivers in the hospitality and tourism industry. This methodology enables in-depth analysis of the factors’ impact and provides understanding of the accommodation product improvement potential based on customer perception. The authors suggest several groups of factors and demonstrate that certain high-impact satisfaction drivers have zero incremental value for customers.

scholarly journals Research Study of the Problems of Human Resourcing of the Scientific and Innovation Entrepreneurship

2019 ◽  
Vol 8 (3C) ◽  
pp. 213-218
Keyword(s):  
Economic Model ◽  
Research Study ◽  
Cognitive Model ◽  
Cognitive Modelling ◽  
Innovation Development ◽  
Methodical Approach ◽  
Innovative Products ◽  
Innovation Processes

Problems of human resourcing of development of the scientific and innovation entrepreneurship are very urgent especially for the countries, which require a transition from the resource-based economic model to the model of innovation development. That is why, studies of the problems of human resourcing with respect to creation of own scientific developments, science-intensive and innovative products and also a possibility of their commercialization in the domestic entrepreneurial sector is very urgent not only for Ukraine but also for other countries, which are interested in ensuring their competitiveness. The goal of the article is a study of the problems of human resourcing of development of the scientific and innovation entrepreneurship with the help of the cognitive modelling. The paper offers a methodical approach to the study of the human resourcing of development of the scientific and innovation entrepreneurship based on the use of the cognitive modelling. Factors of development of the human resourcing of scientific and innovation processes in the sphere of entrepreneurship of Ukraine have been studied and directions of their reformation have been justified on the basis of the developed cognitive model.

scholarly journals Preregistration specificity & adherence: A review of preregistered gambling studies & cross-disciplinary comparison

2021 ◽  
Author(s):  
Robert Heirene ◽  
Debi LaPlante ◽  
Eric R. Louderback ◽  
Brittany Keen ◽  
Marjan Bakker ◽  
...  
Keyword(s):  
Degrees Of Freedom ◽  
Open Data ◽  
Open Science ◽  
Eligibility Criteria ◽  
Science Practices ◽  
Methodological Choices ◽  
The Mean ◽  
The Many ◽  
Practical Recommendations ◽  
Current Practices

Study preregistration is one of several “open science” practices (e.g., open data, preprints) that researchers use to improve the transparency and rigour of their research. As more researchers adopt preregistration as a regular research practice, examining the nature and content of preregistrations can help identify strengths and weaknesses of current practices. The value of preregistration, in part, relates to the specificity of the study plan and the extent to which investigators adhere to this plan. We identified 53 preregistrations from the gambling studies field meeting our predefined eligibility criteria and scored their level of specificity using a 23-item protocol developed to measure the extent to which a clear and exhaustive preregistration plan restricts various researcher degrees of freedom (RDoF; i.e., the many methodological choices available to researchers when collecting and analysing data, and when reporting their findings). We also scored studies on a 32-item protocol that measured adherence to the preregistered plan in the study manuscript. We found that gambling preregistrations had low specificity levels on most RDoF. However, a comparison with a sample of cross-disciplinary preregistrations (N = 52; Bakker et al., 2020) indicated that gambling preregistrations scored higher on 12 (of 29) items. Thirteen (65%) of the 20 associated published articles or preprints deviated from the protocol without declaring as much (the mean number of undeclared deviations per article was 2.25, SD = 2.34). Overall, while we found improvements in specificity and adherence over time (2017-2020), our findings suggest the purported benefits of preregistration—including increasing transparency and reducing RDoF—are not fully achieved by current practices. Using our findings, we provide 10 practical recommendations that can be used to support and refine preregistration practices.

Sign in / Sign up

Export Citation Format

Share Document