Endogenous Moderator Models: What They are, What They Aren’t, and Why it Matters

Models that combine moderation and mediation are increasingly common. One such model is that in which one variable causes another variable that, in turn, moderates the relationship between two other variables. There are many recent examples of these Endogenous Moderator Models (EMMs). They bear little superficial resemblance to second-stage moderation models, and they are almost never conceptualized and tested as such. We use path analytic equations to show that this is precisely what EMMs are. Specifically, we use these path analytic equations and a review of recent EMMs in order to show that these models are seldom conceptualized or tested properly and to understand the best ways to handle such models. We then use Monte Carlo simulation to show the consequences of testing these models as they are typically tested rather than as second-stage moderation models. We end with recommendations and provide example datasets and code for SPSS and R.

Conducting and Evaluating Multilevel Studies: Recommendations, Resources, and a Checklist

Multilevel methods allow researchers to investigate relationships that expand across levels (e.g., individuals, teams, and organizations). The popularity of these methods for studying organizational phenomena has increased in recent decades. Methodologists have examined how these methods work under different conditions, providing an empirical base for making sound decisions when using these methods. In this article, we provide recommendations, tools, resources, and a checklist that can be useful for scholars involved in conducting or assessing multilevel studies. The focus of our article is on two-level designs, in which Level-1 entities are neatly nested within Level-2 entities, and top-down effects are estimated. However, some of our recommendations are also applicable to more complex multilevel designs.

A Review of Measurement Equivalence in Organizational Research: What's Old, What's New, What's Next?

The study of measurement equivalence has important implications for organizational research. Nonequivalence across groups or over time can affect the results of a study and the conclusions that are drawn from it. As a result, the review paper by Vandenberg & Lance (2000) has been highly cited and has played an important role in understanding the measurement of organizational constructs. However, that paper is now 20 years old, and a number of advances have been made in the application and interpretation of measurement equivalence (ME) since its publication. Therefore, the goal of the present paper is to provide an updated review of ME techniques that describes recent advances in testing for ME and proposes a taxonomy of potential sources of nonequivalence. Finally, we articulate recommendations for applying these newer methods and consider future directions for measurement equivalence research in the organizational literature.

Templates in Qualitative Research Methods: Origins, Limitations, and New Directions

In this feature topic, we explore the burgeoning trend to employ templates in qualitative research. To understand authors’ motivations to use templates and perceptions regarding template use in the scholarly community, we conducted an interview study with 21 interviewees who had published qualitative research in one of nine premier management journals between 2014 and 2018. Our analysis identified four key tensions in template use: structure versus restriction, convention versus innovativeness, legitimization versus obscuration, and instrumentality versus craft. We discuss these tensions and their implications for the practice and evaluation of qualitative research. We then introduce the eight empirical papers included in this feature topic. Each of them addresses and explores specific aspects of the origins and limitations of template use, whilst providing insights and guidance for future qualitative research. We envisage this feature topic to be a catalyst for discourse amongst scholars, engaging with contemporary trends and tensions to pave new pathways that embrace the diversity and plurality of qualitative approaches.

The Quick and the Careless: The Construct Validity of Page Time as a Measure of Insufficient Effort Responding to Surveys

Several recent studies have examined the prevention, causes, and consequences of insufficient effort responding (IER) to surveys. Scientific progress in this area, however, rests on the availability of construct-valid IER measures. In the current paper we describe the potential merits of the page time index, which is computed by counting the number of questionnaire pages to which a participant has responded more quickly than two seconds per item (see Huang et al., 2012 ). We conducted three studies (total N = 1,056) to examine the page time index's construct validity. Across these studies, we found that page time converged highly with other IER indices, that it was sensitive to an experimental manipulation warning participants to respond carefully, and that it predicted the extent to which participants were unable to recognize item content. We also found that page time's validity was superior to that of total completion time and that the two-seconds-per-item rule yielded a construct-valid page time score for items of various word lengths. Given its apparent validity, we provide practical recommendations for the use of the page time index.

Inflection Points, Kinks, and Jumps: A Statistical Approach to Detecting Nonlinearities

Inflection points, kinks, and jumps identify places where the relationship between dependent and independent variables switches in some important way. Although these switch points are often mentioned in management research, their presence in the data is either ignored, or postulated ad hoc by testing arbitrarily specified functional forms (e.g., U or inverted U-shaped relationships). This is problematic if we want accurate tests for our theories. To address this issue, we provide an integrative framework for the identification of nonlinearities. Our approach constitutes a precursor step that researchers will want to conduct before deciding which estimation model may be most appropriate. We also provide instructions on how our approach can be implemented, and a replicable illustration of the procedure. Our illustrative example shows how the identification of endogenous switch points may lead to significantly different conclusions compared to those obtained when switch points are ignored or their existence is conjectured arbitrarily. This supports our claim that capturing empirically the presence of nonlinearity is important and should be included in our empirical investigations.

The Use and Misuse of Organizational Research Methods ‘Best Practice’ Articles

This study explores how researchers in the organizational sciences use and/or cite methodological ‘best practice’ (BP) articles. Namely, are scholars adhering fully to the prescribed practices they cite, or are they cherry picking from recommended practices without disclosing? Or worse yet, are scholars inaccurately following the methodological best practices they cite? To answer these questions, we selected three seminal and highly cited best practice articles published in Organizational Research Methods (ORM) within the past ten years. These articles offer clear and specific methodological recommendations for researchers as they make decisions regarding the design, measurement, and interpretation of empirical studies. We then gathered all articles that have cited these best practice pieces. Using comprehensive coding forms, we evaluated how authors are using and citing best practice articles (e.g., if they are appropriately following the recommended practices). Our results revealed substantial variation in how authors cited best practice articles, with 17.4% appropriately citing, 47.7% citing with minor inaccuracies, and 34.5% inappropriately citing BP articles. These findings shed light on the use (and misuse) of methodological recommendations, offering insight into how we can better improve our digestion and implementation of best practices as we design and test research and theory. Key implications and recommendations for editors, reviewers, and authors are discussed.

Making the Invisible Visible: Guidelines for the Coding Process in Meta-Analyses

This article contributes to the practice of coding in meta-analyses by offering direction and advice for experienced and novice meta-analysts on the “how” of coding. The coding process, the invisible architecture of any meta-analysis, has received comparably little attention in methodological resources, leaving the research community with insufficient guidance on “how” it should be rigorously planned (i.e., cohere with the research objective), conducted (i.e., make reliable and valid coding decisions), and reported (i.e., in a sufficiently transparent manner for readers to comprehend the authors’ decision-making). A lack of rigor in these areas can lead to erroneous results, which is problematic for entire research communities who build their future knowledge upon meta-analyses. Along four steps, the guidelines presented here elucidate “how” the coding process can be performed in a coherent, efficient, and credible manner that enables connectivity with future research, thereby enhancing the reliability and validity of meta-analytic findings. Our recommendations also support editors and reviewers in advising authors on how to improve the rigor of their coding and ultimately establish higher quality standards in meta-analytic research.

Interaction Effects in Cross-Lagged Panel Models: SEM with Latent Interactions Applied to Work-Family Conflict, Job Satisfaction, and Gender

Researchers often combine longitudinal panel data analysis with tests of interactions (i.e., moderation). A popular example is the cross-lagged panel model (CLPM). However, interaction tests in CLPMs and related models require caution because stable (i.e., between-level, B) and dynamic (i.e., within-level, W) sources of variation are present in longitudinal data, which can conflate estimates of interaction effects. We address this by integrating literature on CLPMs, multilevel moderation, and latent interactions. Distinguishing stable B and dynamic W parts, we describe three types of interactions that are of interest to researchers: 1) purely dynamic or WxW; 2) cross-level or BxW; and 3) purely stable or BxB. We demonstrate estimating latent interaction effects in a CLPM using a Bayesian SEM in Mplus to apply relationships among work-family conflict and job satisfaction, using gender as a stable B variable. We support our approach via simulations, demonstrating that our proposed CLPM approach is superior to a traditional CLPMs that conflate B and W sources of variation. We describe higher-order nonlinearities as a possible extension, and we discuss limitations and future research directions.