scholarly journals STATISTICAL REPORT REFORM IN SECOND LANGUAGE RESEARCH: A CASE OF EXPERIMENTAL DESIGNS

2021 ◽  
Vol 8 (2) ◽  
pp. 1-27
Author(s):  
Eka Fadilah
Keyword(s):  
Second Language ◽  
Confidence Intervals ◽  
Experimental Studies ◽  
Experimental Designs ◽  
Effect Sizes ◽  
Language Research ◽  
Statistical Report ◽  
Reported Effect ◽  
Power Analyses

This survey aims to review statisical report procedures in the experimental studies appearing in ten SLA and Applied Linguistic journals from 2011 to 2017. We specify our study on how the authors report and interprete their power analyses, effect sizes, and confidence intervals. Results reveal that of 217 articles, the authors reported effect sizes (70%), apriori power and posthoc power consecutively (1.8% and 6.9%), and confidence intervals (18.4%). Additionally, it shows that the authors interprete those statistical terms counted 5.5%, 27.2%, and 6%, respectively. The call for statistical report reform recommended and endorsed by scholars, researchers, and editors is inevitably echoed to shed more light on the trustworthiness and practicality of the data presented.

Effect Sizes Reported in Highly Cited Emotion Research Compared With Larger Studies and Meta-Analyses Addressing the Same Questions

2021 ◽  
pp. 216770262110493
Author(s):  
Ioana A. Cristea ◽  
Raluca Georgescu ◽  
John P. A. Ioannidis
Keyword(s):  
Confidence Interval ◽  
Observational Studies ◽  
Experimental Studies ◽  
Experimental Designs ◽  
Effect Sizes ◽  
Meta Analyses ◽  
Highly Cited

We assessed whether the most highly cited studies in emotion research reported larger effect sizes compared with meta-analyses and the largest studies on the same question. We screened all reports with at least 1,000 citations and identified matching meta-analyses for 40 highly cited observational studies and 25 highly cited experimental studies. Highly cited observational studies had effects greater on average by 1.42-fold (95% confidence interval [CI] = [1.09, 1.87]) compared with meta-analyses and 1.99-fold (95% CI = [1.33, 2.99]) compared with largest studies on the same questions. Highly cited experimental studies had increases of 1.29-fold (95% CI = [1.01, 1.63]) compared with meta-analyses and 2.02-fold (95% CI = [1.60, 2.57]) compared with the largest studies. There was substantial between-topics heterogeneity, more prominently for observational studies. Highly cited studies often did not have the largest weight in meta-analyses (12 of 65 topics, 18%) but were frequently the earliest ones published on the topic (31 of 65 topics, 48%). Highly cited studies may offer, on average, exaggerated estimates of effects in both observational and experimental designs.

STUDY QUALITY IN SLA

2013 ◽  
Vol 35 (4) ◽  
pp. 655-687 ◽  
Author(s):  
Luke Plonsky
Keyword(s):  
Second Language ◽  
Second Language Acquisition ◽  
Language Learning ◽  
Statistical Power ◽  
Applied Linguistics ◽  
Experimental Designs ◽  
Effect Sizes ◽  
Study Quality ◽  
Reporting Practices ◽  
Lack Of Control

This study assesses research and reporting practices in quantitative second language (L2) research. A sample of 606 primary studies, published from 1990 to 2010 inLanguage LearningandStudies in Second Language Acquisition, was collected and coded for designs, statistical analyses, reporting practices, and outcomes (i.e., effect sizes). The results point to several systematic strengths as well as many flaws, such as a lack of control in experimental designs, incomplete and inconsistent reporting practices, and low statistical power. I discuss these trends, strengths, and weaknesses in comparison with methodological reviews of L2 research (e.g., Plonsky & Gass, 2011) as well as reviews from other fields (e.g., education, Skidmore & Thompson, 2010). On the basis of the findings, I also make a number of suggestions for methodological reforms in applied linguistics.

scholarly journals Are effect sizes reported in highly cited emotion research overestimated relative to larger studies and meta-analyses addressing the same questions?

2021 ◽  
Author(s):  
Ioana Cristea ◽  
john Ioannidis ◽  
Raluca Georgescu
Keyword(s):  
Observational Studies ◽  
Experimental Studies ◽  
Experimental Designs ◽  
Effect Sizes ◽  
Research Report ◽  
Meta Analyses ◽  
Highly Cited

We assessed whether the most highly-cited studies in emotion research report larger effect sizes compared with meta-analyses and the largest studies on the same questions. We screened all reports with at least 1000 citations and identified matching meta-analyses for 40 highly-cited observational and 25 highly-cited experimental studies. Observational studies had on average 1.42-fold (95% CI 1.09 to 1.87) larger effects than meta-analyses and 1.99-fold (95% CI 1.33 to 2.99) larger effects than largest studies on the same questions. Experimental studies had fold-increases of 1.29 (95% CI 1.01 to 1.63) versus meta-analyses and 2.02 (95% CI 1.60 to 2.57) versus largest studies. There was substantial between-topic heterogeneity, more prominently for observational studies. Highly-cited studies were uncommonly (12/65 topics, 18%) the largest ones, but they were frequently (31/65 topics, 48%) the earliest published on the topic. Highly-cited studies may offer, on average, exaggerated estimates of effects in both observational and experimental designs.

Introspection in Second Language Research

1989 ◽  
Vol 73 (1) ◽  
pp. 77
Author(s):  
Mary-Ann Reiss ◽  
Claus Faerch ◽  
Gabriele Kasper
Keyword(s):  
Second Language ◽  
Language Research

Pragmatic transfer

1992 ◽  
Vol 8 (3) ◽  
pp. 203-231 ◽  
Author(s):  
Gabriele Kasper
Keyword(s):  
Second Language ◽  
Native Speakers ◽  
Interlanguage Pragmatics ◽  
Structural Factors ◽  
Pragmatic Transfer ◽  
Language Research ◽  
Linguistic Action ◽  
Core Areas ◽  
Basic Distinction ◽  
Pragmatic Knowledge

Throughout the short life of interlanguage pragmatics as a subdiscipline of second language research, it has been a virtually uncontested assumption that non-native speakers' comprehension and production of linguistic action is considerably influenced by their L1 pragmatic knowledge. The literature strongly supports this hypothesis. However, whereas there has been a lively controversy about the role of transfer in the traditional core areas of second language research (syntax, morphology, semantics), there has been little theoretical and methodological debate about transfer in interlanguage pragmatics. As a contribution to such a debate, this article seeks to clarify the concept of pragmatic transfer, proposing as a basic distinction Leech/Thomas' dichotomy of sociopragmatics versus pragmalinguistics and presenting evidence for transfer at both levels. Evidence for purported pragmatic universals in speech act realization and for positive and negative pragmatic transfer is discussed. Further issues to be addressed include the conditions for pragmatic transfer (transferability), the interaction of transfer with non-structural factors (proficiency, length of residence, context of acquisition), and the effect of transfer on communicative outcomes. The article concludes by briefly considering some problems of research method in studies of pragmatic transfer.

Second Language Acquisition Research and Language Pedagogy: A Critical View

2015 ◽  
Vol 7 (4) ◽  
pp. 1 ◽  
Author(s):  
Mohammad Reza Talebinejad ◽  
Aasa Moattarian
Keyword(s):  
Second Language ◽  
Second Language Acquisition ◽  
Language Acquisition ◽  
Teaching Practice ◽  
Structured Interview ◽  
Language Teachers ◽  
Language Pedagogy ◽  
Language Research ◽  
Class 1 ◽  
Research Findings

<p class="1"><span lang="X-NONE">Over the past several decades, a substantial body of research on second language acquisition has been provided. The current study was an attempt to investigate language teachers’ views on applying research findings in their every day practice of language teaching through a critical lens. Data for this qualitative study was collected by means of a semi structured interview with 10 language teachers teaching English at different language institutes in Iran. Analyses of data revealed that, although teachers find second language acquisition research a useful tool for their professional development; they do not usually consult bodies of research in their every day teaching practice. They report problems in applying second language research in their practice due to problems with practicality, particularity, and possibility. The findings suggest that language teachers need to be exposed to insight from SLA research and practice.</span></p>

Sign in / Sign up

Export Citation Format

Share Document