Commentary on „Heterogeneity in direct replications in psychology and its association with effect size” (Olsson-Collentine, Wicherts & van Assen, 2020, Psychological Bulletin, online first publication, July 23, 2020.)

We argue that the observed correlation between the effect size and the heterogeneity of replicated studies is artificial. Specifically, that the observed heterogeneity for larger effects exceeds the observed heterogeneity for zero or small effects fails to provide a reason to invest comparatively less trust in parameter-estimates for large(r) effects.

Erfassung der Primärstudienqualität in psychologischen Meta-Analysen

Zusammenfassung. Meta-Analysen haben sich als Methodik zur Zusammenfassung von Studienergebnissen etabliert und sind ein wichtiges Instrument für Wissenschaftler, aber auch für politische und soziale Entscheidungsträger. Jedoch können die Schätzungen einer Meta-Analyse verzerrt sein, wenn nicht für die methodische Qualität der eingehenden Primärstudien kontrolliert wird. Die Erfassung der Primärstudienqualität und die Bereitstellung entsprechender Instrumente sollte dementsprechend essentieller Bestandteil jeder wissenschaftlichen Disziplin sein, die sich auf Meta-Analysen zur quantitativen Integration von Studienergebnissen stützt. Die vorliegende Übersichtsarbeit bietet daher einen Überblick ob und welche Qualitätskriterien in der Psychologie herangezogen werden. Insgesamt 225 Meta-Analysen, die in den letzten 10 Jahren im Psychological Bulletin veröffentlicht wurden, gingen in die Analyse ein. Nur etwa 18 % dieser Studien berücksichtigen explizit die Qualität der eingehenden Primärstudien. Zudem weisen die Strategie der Qualitätsberücksichtigung, wie auch die verwendeten Instrumente, bzw. die entwickelten Items eine ausgeprägte Heterogenität zwischen den Studien auf. Diese Unterschiede, die Vor- und Nachteile der spezifischen Vorgehensweisen als auch die praktischen Implikationen, die sich daraus ergeben werden anschließend diskutiert.

Implementing a General Framework for Assessing Interrater Agreement in Stata

Despite its well-known weaknesses, researchers continuously choose the kappa coefficient (Cohen, 1960, Educational and Psychological Measurement 20: 37–46; Fleiss, 1971, Psychological Bulletin 76: 378–382) to quantify agreement among raters. Part of kappa's persistent popularity seems to arise from a lack of available alternative agreement coefficients in statistical software packages such as Stata. In this article, I review Gwet’s (2014, Handbook of Inter-Rater Reliability) recently developed framework of interrater agreement coefficients. This framework extends several agreement coefficients to handle any number of raters, any number of rating categories, any level of measurement, and missing values. I introduce the kappaetc command, which implements this framework in Stata.

New perspectives for Social Psychological Bulletin (Psychologia Społeczna)

Editorial - no abstract

Communicative Types of Self-Disclosure in Public Speaking Setting

The present paper focuses on the study of the communicative nature of self-disclosure as a technique of verbal influence on people. The phenomenon of self-disclosure is interpreted as the process of divulging private information to others. The aim of the study is to determine the specific features of self-disclosure in a public speaking context. The paper discusses the communicative conventions and parameters of the public speaking scenario as the factors that determine a certain manner of self-disclosing. The key assumption of the study is that public self-disclosure should be viewed as a tactic of self-presentation that promotes the speaker’s image building. In order to indicate the verbal forms that speakers use while disclosing private information in public the discourse and lexical-grammatical analyses of eight celebrity interview texts are conducted. The results of the research demonstrate that the speakers use recurrently certain verbal patterns which are identified as statements of self-description (it includes statements of self-confirmation, self-assurance, personal preferences and personal aspirations), self-narrative and attitude statements. The study also shows which types of the statements are used more frequently than the others, particularly, the statements of self-confirmation, personal aspirations and self-narratives are apparently preferred by the speakers as tactics of public self-disclosure. The paper highlights the fact that the self-disclosure tactics are employed by public speakers selectively and that their choice is motivated by the speakers’ strategic goal of impression management. References Altman, I., Taylor, D. A. (1973). Social Penetration: The Development of InterpersonalRelationship. New York, NY: Holl, Rinehart & Winston. Baumeister, R. F. (1982). A self-presentational view of social phenomena. PsychologicalBulletin, 91, 3–26. Carver, C. S., Scheier, M. R. (1998). On the Self-Regulation of Behavior. CambridgeUniversity Press. Cozby, P.C. (1973). Self-disclosure: A literature review. Psychological Bulletin, 79(2),73–91. Derlega, V. J., Metts, S., Petronio, S., Margulis, S. T. (1993). Self-Disclosure. NewburyPart, CA: Sage. Hargie, O. (2011). Skilled Interpersonal Interaction: Research, Theory and Practice.London: Reutledge. Johnson, J.A. (1981). The ‘self-disclosure’ and ‘self-presentation’ views of item responsedynamics and personality scale validity. Journal of Personality and Social Psychology,40(4), 761–769. Jourard, S. (1971). The Transparent Self. (2nd ed.).New York: Van Nostrand Reinhold. Leary, M. R. (1995). Self-Presentation: Impression Management and InterpersonalBehavior. Madison, WI: Brown & Benchmark. Leary, M. R. (1996). Self-Presentation: Iimpression Management and InterpersonalBehavior. Boulder, CO: Westview Press. Leary, M. R. (2012). Introduction to Behavioral Research Methods. (6nd ed.). Boston:Pearson. Leary, M. R., Kowalski, R. M. (1990). Impression management: A literature review andtwo-component model. Psychological Bulletin, 107(1), 34–47. Luft, J., Ingham, H. (1969). Of Human Interaction. Palo Alto, CA: National Press Books. Rosenfeld, L. B. (2014). Overview of the ways privacy, secrecy, and disclosure arebalanced in today’s society. In: Balancing the Secrets of Private Disclosure, (pp. 3 – 18).S. Petronio, (ed.). New York and London: Psychology Press. Schlenker, B. R. (1980). Impression management: the self-concept, social identity, andinterpersonal relations. Monterey, CA: Brooks/Cole. Schlenker, B. R. (1985). Identity and self-identification. In: The self and social life, (pp.65–99). B.R.Schlenker, (ed.). New York: McGrow-Hill. Schlenker, B. R. (2003). Self-presentation. In: Handbook of Self and Identity, (pp. 492–518). M. R.Leary, J. P.Tangney, (eds.). New York: Guilford. Sources J.K. Rolling meets Lauren Laverne, 2015. Oprah talks to Barack Obama, 2004. O, The Oprah magazine. Oprah talks to Daniel Pink, 2008. O, The Oprah Magazine. Oprah talks to Ellen De Generes, 2009. O, The Oprah Magazine. Oprah talks to Jay-Z, 2009. O, The Oprah Magazine. Oprah talks to Tine Fey, 2009, O, The Oprah Magazine. Oprah talks to Thich Nhat Hanh, 2010.8.The ultimate O interview: Oprah answers all your questions, 2010. O, the Oprah magazine.

Estimates of Between-Study Heterogeneity for 705 Meta-Analyses Reported in Psychological Bulletin From 1990-2013

We present a data set containing 705 between-study heterogeneity estimates as reported in 61 articles published in Psychological Bulletin from 1990-2013. The data set also includes information about the number and type of effect sizes, the Q-statistic, and publication bias. The data set is stored in the Open Science Framework repository and can be used for several purposes: (1) to compare a specific heterogeneity estimate to the distribution of between-study heterogeneity estimates in psychology; (2) to construct an informed prior distribution for the between-study heterogeneity in psychology; (3) to obtain realistic population values for Monte Carlo simulations investigating the performance of meta-analytic methods.

Publication Bias in Meta-Analyses from Psychology and Medicine: A Meta-Meta-Analysis

Publication bias is a substantial problem for the credibility of research in general and of meta-analyses in particular, as it yields overestimated effects and may suggest the existence of non-existing effects. Although there is consensus that publication bias exists, how strongly it affects different scientific literatures is currently less well-known. We examined evidence of publication bias in a large-scale data set of primary studies that were included in 83 meta-analyses published in Psychological Bulletin (representing meta-analyses from psychology) and 499 systematic reviews from the Cochrane Database of Systematic Reviews (CDSR; representing meta-analyses from medicine). Publication bias was assessed on all homogeneous subsets (3.8% of all subsets of meta-analyses published in Psychological Bulletin) of primary studies included in meta-analyses, because publication bias methods do not have good statistical properties if the true effect size is heterogeneous. The Monte-Carlo simulation study revealed that the creation of homogeneous subsets resulted in challenging conditions for publication bias methods since the number of effect sizes in a subset was rather small (median number of effect sizes equaled 6). No evidence of bias was obtained using the publication bias tests. Overestimation was minimal but statistically significant, providing evidence of publication bias that appeared to be similar in both fields. These and other findings, in combination with the small percentages of statistically significant primary effect sizes (28.9% and 18.9% for subsets published in Psychological Bulletin and CDSR), led to the conclusion that evidence for publication bias in the studied homogeneous subsets is weak, but suggestive of mild publication bias in both psychology and medicine.