A Meta-Analysis of Reliability Coefficients in Second Language Research

2016 ◽  
Vol 100 (2) ◽  
pp. 538-553 ◽  
Author(s):  
LUKE PLONSKY ◽  
DEIRDRE J. DERRICK
Keyword(s):  
Second Language ◽  
Meta Analysis ◽  
Language Research ◽  
Reliability Coefficients

Practice and progression in Second Language Research methods

AILA Review ◽  
2014 ◽  
Vol 27 ◽  
pp. 80-97 ◽  
Author(s):  
Alison Mackey
Keyword(s):  
Second Language ◽  
Language Learning ◽  
Corpus Linguistics ◽  
Second Language Learning ◽  
Meta Analysis ◽  
Research Synthesis ◽  
Deep Dive ◽  
Psychology Education ◽  
Language Research ◽  
General Linguistics

Since its inception, the field of second language research has utilized methods from a number of areas, including general linguistics, psychology, education, sociology, anthropology and, recently, neuroscience and corpus linguistics. As the questions and objectives expand, researchers are increasingly pushing methodological boundaries to gain a clearer picture of second language learning. At one end for example, we see measures of cognition (e.g., brain imaging and eye tracking) and at the other end we see exploration of issues of culture and identity (e.g., ethnographies, deep dive case studies, introspective and narrative analyses). There is an emerging emphasis on research synthesis, meta-analysis, and replication. This article illustrates a few of the advancements in methods and research agendas in SLA. I will conclude by highlighting some of the ways that second language researchers can continue to incorporate, assimilate, and shape methodology, as well as pointing out some of the potential pitfalls, and overall, how these methodological innovations benefit the field.

Meta-analysis in Second Language Research: Choices and Challenges

2010 ◽  
Vol 30 ◽  
pp. 85-110 ◽  
Author(s):  
Frederick L. Oswald ◽  
Luke Plonsky
Keyword(s):  
Second Language ◽  
Second Language Acquisition ◽  
Statistical Power ◽  
Meta Analysis ◽  
Future Research ◽  
Language Research ◽  
Literature Reviews ◽  
Research Choices ◽  
Research Domain ◽  
Qualitative Literature

Applied linguists are increasingly conducting meta-analysis in their substantive domains, because as a quantitative approach for averaging effect sizes across studies, it is more systematic and replicable than traditional, qualitative literature reviews. Additional strengths, such as increased statistical power, moderator analyses, and model testing, have also contributed to its appeal. The current review describes typical stages of a meta-analysis in second language acquisition (SLA) research: (a) defining the research domain, (b) developing a reliable coding scheme, (c) analyzing data, and (d) interpreting results. Each stage has a host of equally reasonable decisions that can be made; each decision will influence the conduct of the meta-analysis, the nature of the results, and the substantive implications of findings for SLA. We highlight a number of benefits and challenges that inform these decisions. In general, when a meta-analysis in applied linguistics is well planned, employs sound statistical methods, and is based on a thorough understanding of relevant theory, it can provide critical information that informs theory as well as future research, practice, and policy.

scholarly journals The effects of dictionary use on second language vocabulary acquisition: A meta-analysis

2020 ◽  
Author(s):  
Songshan Zhang ◽  
Hai Xu ◽  
Xian Zhang
Keyword(s):  
Second Language ◽  
Vocabulary Acquisition ◽  
Meta Analysis ◽  
Vocabulary Knowledge ◽  
Learning Condition ◽  
Moderator Variable ◽  
Lexical Unit ◽  
Language Research ◽  
Dictionary Use ◽  
L2 Vocabulary

Abstract This study reports on the results of a meta-analysis which investigates the effects of dictionary use on second language (L2) vocabulary acquisition, as well as the magnitude of the moderating effect of a number of moderator variables on the effectiveness of dictionary use in improving L2 vocabulary knowledge. A total of 125 effect sizes were gleaned from 44 studies, which represented 87 independent samples and included 3,475 participants. A random-effects model of the meta-analysis shows that the overall effect of dictionary use on L2 vocabulary acquisition is g = 2.10 (p < .01) for within-group studies and g = 1.03 (p < .01) for between-group studies respectively, which are both large in the domain of second language research. Subsequent moderator variable analysis reveals how treatment-related variables (timepoint, dictionary form, dictionary type, target lexical unit, vocabulary knowledge type, learning condition, and research setting), methodological variables (type of experimental design, presence of pretest, and assessment type), as well as learner-related variables (age and proficiency) might contribute to the variation across studies. Implications of the findings are discussed.

Determinants of Interrater Reliability of Job Analysis: A Meta-analysis

2002 ◽  
Vol 18 (1) ◽  
pp. 52-62 ◽  
Author(s):  
Olga F. Voskuijl ◽  
Tjarda van Sliedregt
Keyword(s):  
Data Collection ◽  
Interrater Reliability ◽  
Meta Analysis ◽  
Job Analysis ◽  
Job Descriptions ◽  
Job Description ◽  
Research And Practice ◽  
Descriptive Information ◽  
Interrater Reliability Coefficients ◽  
Reliability Coefficients

Summary: This paper presents a meta-analysis of published job analysis interrater reliability data in order to predict the expected levels of interrater reliability within specific combinations of moderators, such as rater source, experience of the rater, and type of job descriptive information. The overall mean interrater reliability of 91 reliability coefficients reported in the literature was .59. The results of experienced professionals (job analysts) showed the highest reliability coefficients (.76). The method of data collection (job contact versus job description) only affected the results of experienced job analysts. For this group higher interrater reliability coefficients were obtained for analyses based on job contact (.87) than for those based on job descriptions (.71). For other rater categories (e.g., students, organization members) neither the method of data collection nor training had a significant effect on the interrater reliability. Analyses based on scales with defined levels resulted in significantly higher interrater reliability coefficients than analyses based on scales with undefined levels. Behavior and job worth dimensions were rated more reliable (.62 and .60, respectively) than attributes and tasks (.49 and .29, respectively). Furthermore, the results indicated that if nonprofessional raters are used (e.g., incumbents or students), at least two to four raters are required to obtain a reliability coefficient of .80. These findings have implications for research and practice.

The Cognition Hypothesis: A Synthesis and Meta-Analysis of Research on Second Language Task Complexity

2013 ◽  
Vol 63 (2) ◽  
pp. 330-367 ◽  
Author(s):  
Daniel O. Jackson ◽  
Sakol Suethanapornkul
Keyword(s):  
Second Language ◽  
Task Complexity ◽  
Meta Analysis

Introspection in Second Language Research

1989 ◽  
Vol 73 (1) ◽  
pp. 77
Author(s):  
Mary-Ann Reiss ◽  
Claus Faerch ◽  
Gabriele Kasper
Keyword(s):  
Second Language ◽  
Language Research

Pragmatic transfer

1992 ◽  
Vol 8 (3) ◽  
pp. 203-231 ◽  
Author(s):  
Gabriele Kasper
Keyword(s):  
Second Language ◽  
Native Speakers ◽  
Interlanguage Pragmatics ◽  
Structural Factors ◽  
Pragmatic Transfer ◽  
Language Research ◽  
Linguistic Action ◽  
Core Areas ◽  
Basic Distinction ◽  
Pragmatic Knowledge

Throughout the short life of interlanguage pragmatics as a subdiscipline of second language research, it has been a virtually uncontested assumption that non-native speakers' comprehension and production of linguistic action is considerably influenced by their L1 pragmatic knowledge. The literature strongly supports this hypothesis. However, whereas there has been a lively controversy about the role of transfer in the traditional core areas of second language research (syntax, morphology, semantics), there has been little theoretical and methodological debate about transfer in interlanguage pragmatics. As a contribution to such a debate, this article seeks to clarify the concept of pragmatic transfer, proposing as a basic distinction Leech/Thomas' dichotomy of sociopragmatics versus pragmalinguistics and presenting evidence for transfer at both levels. Evidence for purported pragmatic universals in speech act realization and for positive and negative pragmatic transfer is discussed. Further issues to be addressed include the conditions for pragmatic transfer (transferability), the interaction of transfer with non-structural factors (proficiency, length of residence, context of acquisition), and the effect of transfer on communicative outcomes. The article concludes by briefly considering some problems of research method in studies of pragmatic transfer.

Meta-analysis of the neural representation of first language and second language

2011 ◽  
Vol 32 (4) ◽  
pp. 799-819 ◽  
Author(s):  
RAJANI SEBASTIAN ◽  
ANGELA R. LAIRD ◽  
SWATHI KIRAN
Keyword(s):  
Second Language ◽  
Language Proficiency ◽  
Language Processing ◽  
Functional Neuroimaging ◽  
Meta Analysis ◽  
First Language ◽  
Likelihood Estimation ◽  
Neural Representation ◽  
Second Language Processing ◽  
Meta Analyses

ABSTRACTThis study reports an activation likelihood estimation meta-analysis of published functional neuroimaging studies of bilingualism. Four parallel meta-analyses were conducted by taking into account the proficiency of participants reported in the studies. The results of the meta-analyses suggest differences in the probabilities of activation patterns between high proficiency and moderate/low proficiency bilinguals. The Talairach coordinates of activation in first language processing were very similar to that of second language processing in the high proficient bilinguals. However, in the low proficient group, the activation clusters were generally smaller and distributed over wider areas in both the hemispheres than the clusters identified in the ALE maps from the high proficient group. These findings draw attention to the importance of language proficiency in bilingual neural representation.

Sign in / Sign up

Export Citation Format

Share Document