scholarly journals MALL language learning outcomes: A comprehensive meta-analysis 1994–2019

ReCALL ◽  
2021 ◽  
pp. 1-22
Author(s):  
Jack Burston ◽  
Konstantinos Giannakou

Abstract The aim of this study is to comprehensively evaluate quantitative experimental mobile-assisted language learning (MALL) studies published between 1994 and 2019 that meet minimal conditions of research design and statistical analysis. Starting with a bibliographical database of 1,144 references to experimental MALL implementations, of which there were 700 objectively substantiated by quantitative experimental language learning outcomes, only 84 experimental MALL studies met the inclusion requirements. Their analysis addresses two critical sets of research questions. First, what are the general characteristics of the selected studies and, second, what are their language learning outcomes in terms of measured effect size. Nine general characteristics are considered: publication source, chronological distribution, country of origin, institutional environment, sample size, intervention duration, targeted language, language learner competence level, and learning focus. Effect size was calculated separately for between-group (independent, experimental) and within-group (quasi-experimental) treatment studies. In both cases, the overall results were quite large: 0.72 for the former and 1.16 for the latter. An analysis of four critical moderator variables (language learner competence level, language area focus, institutional environment, and intervention duration) revealed similarly large effect sizes. Notwithstanding, analysis of the data also confirmed obvious publication bias and a very high level of heterogeneity that frequently approached 100%. The relevance of positive language learning outcome conclusions thus needs to be tempered by these shortcomings.

ReCALL ◽  
2014 ◽  
Vol 27 (1) ◽  
pp. 4-20 ◽  
Author(s):  
Jack Burston

AbstractDespite the hundreds of Mobile-Assisted Language Learning (MALL) publications over the past twenty years, statistically reliable measures of learning outcomes are few and far between. In part, this is due to the fact that well over half of all MALL-related studies report no objectively quantifiable learning outcomes, either because they did not involve MALL implementation projects, or if they did, learning gains were only based on subjective teacher assessments and/or student self-evaluations. Even more so, the paucity of statistically reliable learning outcome data stems from the short duration of projects and small numbers of students involved. Of the 291 distinct studies examined in this review only 35 meet minimal conditions of duration and sample size, i.e., ten experimental subjects over a period of at least a month. Sixteen of these suffer from serious design shortcomings, leaving only nineteen MALL studies that can reliably serve as a basis for determining the learning outcomes of mobile-based language applications. Of these studies, fifteen can be considered to report unequivocal positive results, with those focusing on reading, listening and speaking without exception evidencing a MALL application advantage. Four studies, all focusing on vocabulary, reported no significant differences.


2020 ◽  
Vol 8 (3) ◽  
Author(s):  
Irma Kumala Sari ◽  
Rokhmaniyah Rokhmaniyah ◽  
Tri Saptuti Susiani

<p><em>The Somatic Auditory Visualization Intellectually (SAVI) model is one of innovative learning models. It needs to accomplish learning purposes. The study aimed to examine the effectiveness of SAVI model on third grade origami project learning outcomes.</em><em> </em><em>The research was a quasi-experimental research in the form of Nonequivalent Control Group Design. The population was all third grade students of elementary schools in Rujakbeling cluster, Kebumen sub-district in academic year of 2019/2020. The sampling technique used cluster random sampling. Data collection techniques included t-test, n-gain test and effect size test.</em><em> </em><em>The result showed the value between the experimental class and the control class.  The t-test was 0.000&lt;0.05, N-Gain was 0.4550&gt;0.2747, and effect size was 2.123&gt;1.291. It concludes that the Somatic Auditory Visualization Intellectually (SAVI) model is more effective in improving third grade origami project learning outcomes of elementary schools in Rujakbeling cluster, Kebumen sub-district.</em></p>


2021 ◽  
Vol 4 (2) ◽  
pp. 94
Author(s):  
Rizkiana Nurazizah ◽  
Rinaningsih Rinaningsih

This review article aims to identify the effect size of the mastery learning model in chemistry learning. The writing of this review article uses a meta-analysis research design using the Glass method. An effect size test was performed with a comparison of the result of 5 relevant research literature. In this study using purposive sampling technique to obtain samples. Based on the effect size test, the effect size value obtained in chemistry learning with the mastery learning model includes 1) learning outcomes of 1.32. 2) learning activities shown by the creativity of students by 4 and activities of studens by 0,48. 3) the response of students as indicated by the interest and motivation of learning students, namely 21,98 and 0,75. 4) male students have a greater effect size than female students, namely 24,53 and 22,77. Student learning outcomes can be increased by using the mastery learning model. Students have a positive response to the mastery learning model indicated by students are very enthusiastic, critical and easier to understand the material being taught.


2021 ◽  
Author(s):  
◽  
Oliver Ballance

<p>Language learner use of concordances attracts considerable research interest. Concordances are now being conceptualized as presenting language learners with multiple affordances (Leńko-Szymańska & Boulton, 2015), affordances being the latent value of an object determined by the use made of it and not necessarily by its design. However, empirical research has typically operationalized concordances in simple, monolithic terms, and thus, from an experimental design perspective, they are implicitly seen as providing a single set of language learning affordances. That is, the majority of research conducted on concordancing treatments is discussed in terms of concordancing in fairly simple undifferentiated terms (see overviews of research in Boulton, 2010a; Cobb & Boulton, 2015; C. Yoon, 2011). Thus, previous research has contributed to an understanding of whether concordances can facilitate language learning, but it has rarely addressed the issue of how the operationalisation of concordances in pedagogical contexts interacts with the language learning affordances provided. Insufficient engagement with this issue is important because, alongside studies that have shown strong facilitative effects, there are many studies of learner use of concordances that show little to no facilitative effect of the concordancing condition. To address this issue, this thesis addresses the construct of learner use of concordances from an experiment design perspective, helping to define the construct of concordancing and examining potential variables in learner use of concordances.   The thesis reports a series of studies that examine the construct of concordancing from both the perspective of concordance users’ concordancing preferences and the perspective of the lexical qualities of concordances as texts. First, a quasi-experimental, quantitative survey of concordance users’ concordancing preferences showed the construct of concordancing to comprise four distinguishable operational parameters: citation format, type of corpus concordanced, citation order and reading style. It then found correlations between these parameters and three of four user groupings: grouping by frequency of concordance use, by linguistic relationship to concordance language, and by field of concordance use. It revealed that different types of concordance user can be characterized by preferences for different types of concordance use, and vice versa. In two further studies, quantitative analysis of concordances showed that manipulation of factors in concordance generation resulted in concordances with significant differences in measures of word frequency and type token ratio. These analyses showed the extent to which the affordances of concordances vary in relation to two key factors in concordance generation: which corpus is concordanced and level of corpus generality.  The findings of these four studies are discussed in relation to the definition of concordancing presented in Chapter 2 of the thesis and the affordances of concordances that have been discussed in the literature. Together the findings indicate that effective learner use of concordances is likely to be dependent on matching the operationalization of concordancing in pedagogical contexts to learner profiles. For this reason, research on learner use of concordances needs to adopt experimental designs that can account for variation in concordancing as a treatment condition. The findings are also discussed in relation to their practical implications for effectively operationalising concordancing in pedagogical practice and the development of pedagogical concordancers and concordance-based language learning materials.</p>


2021 ◽  
Author(s):  
◽  
Oliver Ballance

<p>Language learner use of concordances attracts considerable research interest. Concordances are now being conceptualized as presenting language learners with multiple affordances (Leńko-Szymańska & Boulton, 2015), affordances being the latent value of an object determined by the use made of it and not necessarily by its design. However, empirical research has typically operationalized concordances in simple, monolithic terms, and thus, from an experimental design perspective, they are implicitly seen as providing a single set of language learning affordances. That is, the majority of research conducted on concordancing treatments is discussed in terms of concordancing in fairly simple undifferentiated terms (see overviews of research in Boulton, 2010a; Cobb & Boulton, 2015; C. Yoon, 2011). Thus, previous research has contributed to an understanding of whether concordances can facilitate language learning, but it has rarely addressed the issue of how the operationalisation of concordances in pedagogical contexts interacts with the language learning affordances provided. Insufficient engagement with this issue is important because, alongside studies that have shown strong facilitative effects, there are many studies of learner use of concordances that show little to no facilitative effect of the concordancing condition. To address this issue, this thesis addresses the construct of learner use of concordances from an experiment design perspective, helping to define the construct of concordancing and examining potential variables in learner use of concordances.   The thesis reports a series of studies that examine the construct of concordancing from both the perspective of concordance users’ concordancing preferences and the perspective of the lexical qualities of concordances as texts. First, a quasi-experimental, quantitative survey of concordance users’ concordancing preferences showed the construct of concordancing to comprise four distinguishable operational parameters: citation format, type of corpus concordanced, citation order and reading style. It then found correlations between these parameters and three of four user groupings: grouping by frequency of concordance use, by linguistic relationship to concordance language, and by field of concordance use. It revealed that different types of concordance user can be characterized by preferences for different types of concordance use, and vice versa. In two further studies, quantitative analysis of concordances showed that manipulation of factors in concordance generation resulted in concordances with significant differences in measures of word frequency and type token ratio. These analyses showed the extent to which the affordances of concordances vary in relation to two key factors in concordance generation: which corpus is concordanced and level of corpus generality.  The findings of these four studies are discussed in relation to the definition of concordancing presented in Chapter 2 of the thesis and the affordances of concordances that have been discussed in the literature. Together the findings indicate that effective learner use of concordances is likely to be dependent on matching the operationalization of concordancing in pedagogical contexts to learner profiles. For this reason, research on learner use of concordances needs to adopt experimental designs that can account for variation in concordancing as a treatment condition. The findings are also discussed in relation to their practical implications for effectively operationalising concordancing in pedagogical practice and the development of pedagogical concordancers and concordance-based language learning materials.</p>


Author(s):  
Hansel Burley ◽  
Bolanle A. Olaniran

The researchers included 10 embedded assessments in the study for a mean gain effect size of .35. From the results of a pilot study, the authors learned that while the technique works, questions about the reliability and validity of the classroom assessments emerged. The authors speculate that institutional researchers can use these techniques to compare the assessment of standards within and among universities.


2017 ◽  
Vol 46 (3) ◽  
pp. 326-339 ◽  
Author(s):  
Christina L. Svec

The purpose of the meta-analysis was to address the varied results within the area of singing ability research by statistically summarizing the data of related studies. Across 34 studies, analyses yielded an overall mean effect size for instruction of g = 0.43. Studies were limited to pretest-posttest or posttest-only quasi-experimental designs. The largest overall study effect size across categorical variables included the effects of same and different discrimination techniques on mean singing score gains. Overall mean effects by primary moderator variable ranged from trivial to moderate. Feedback yielded the largest effect regarding teaching condition, 8-year-old children yielded the largest effect regarding age, the Boardman assessment measure yielded the largest effect regarding measurement instrument, and song accuracy yielded the largest effect regarding measured task. Regarding gender, boys and girls improved similarly from singing interventions across studies. Implications for singing instruction pertain to the importance of intervention, especially between the ages of 5 and 8. Results from the meta-analysis have highlighted a tendency for singing interventions to improve singing ability more than traditional song-singing and more than no music instruction at all. Results from the meta-analysis have also highlighted the importance of self, teacher, and computer feedback in the development of singing.


2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Jafar Rahmati ◽  
Siros Izadpanah ◽  
Ali Shahnavaz

AbstractAs more various types of computer-assisted language learning (CALL) programs have been incorporated into language classrooms over the recent decades, it has become more important to uncover whether, to what extent, and under which moderator variables CALL can be yield more effective outcomes than traditional language instruction. The issue of education is one of the most important materials addressed by technology. Instead, meta-analysis is a statistical and quantitative method that leads us to a general conclusion by integrating the results of different researches. In this study, researchers worked on the impact of educational technology in English language teaching by studying 67 articles and theses (from 1000 studies that were relevant in title and abstract). All articles and theses were included from 2009 to 2020 and 7 articles were excluded from this study due to insufficient information. Furthermore, two instruments, SPSS (mainly its sub-branch Kruskal-Wallis test) and CMA were used to calculate and evaluate data in this research. The total effect size calculated for studies under both fixed and random models was statistically significant and also the study of effects by year of publication, instruments used in research and research methods showed that their effect size was significant. Teaching English with the help of technology has an effective effect size and has shown the success of this technology in language learning.


2019 ◽  
Vol 5 (2) ◽  
pp. 152-168
Author(s):  
Heru Setiawan ◽  
Shane Phillipson

This study identifies the effectiveness of game-based science learning (GBSL) for improving students’ learning outcomes by conducting a literature review of the current research from 2010 to 2017. This study also explores the correlation between variation in school level and year of publication on GBSL effect size. Data were collected from peer-reviewed journal articles published in educational databases including ERIC (Educational Research Information Centre), Springer Link, ProQuest education journal, and A+ education. Seven inclusion criteria were used to select relevant studies. Comprehensive Meta-Analysis (CMA 2.0) was used to analyze the data. This study finds that (1) GBSL intervention has a statistically significant effect on students' learning outcomes with a higher average on the effect size of the experimental group (41.12) than the control group (37.07). The mean of the reviewed studies’ effect size is 0.667 in the medium category. (2) The implementation of GBSL in secondary school has a bigger average effect size than in elementary school. Year of publication and effect size has a low positive correlation with a coefficient of correlation 0.40. 


Author(s):  
Alistair M. Senior ◽  
Wolfgang Viechtbauer ◽  
Shinichi Nakagawa

AbstractMeta-analyses are frequently used to quantify the difference in the average values of two groups (e.g., control and experimental treatment groups), but examine the difference in the variability (variance) of two groups. For such comparisons, the two relatively new effect size statistics, namely the log-transformed ‘variability ratio’ (the ratio of two standard deviations; lnVR) and the log-transformed ‘CV ratio’ (the ratio of two coefficients of variation; lnCVR) are useful. In practice, lnCVR may be of most use because a treatment may affect the mean and the variance simultaneously. We review current, and propose new, estimators for lnCVR and lnVR. We also present methods for use when the two groups are dependent (e.g., for cross-over and pre-test-post-test designs). A simulation study evaluated the performance of these estimators and we make recommendations about which estimators one should use to minimise bias. We also present two worked examples that illustrate the importance of accounting for the dependence of the two groups. We found that the degree to which dependence is accounted for in the sampling variance estimates can impact heterogeneity parameters such as τ2 (i.e., the between-study variance) and I2 (i.e., the proportion of the total variability due to between-study variance), and even the overall effect, and in turn qualitative interpretations. Meta-analytic comparison of the variability between two groups enables us to ask completely new questions and to gain fresh insights from existing datasets. We encourage researchers to take advantage of these convenient new effect size measures for the meta-analysis of variation.


Sign in / Sign up

Export Citation Format

Share Document