Thresholds, Text Coverage, Vocabulary Size, and Reading Comprehension in Applied Linguistics

<p>The inextricable link between vocabulary knowledge and reading comprehension is incontrovertible. However, questions remain regarding the nature of the interaction. One question which remains unresolved is whether there is an optimum text coverage, or ratio of known to unknown words in a text, such that any deleterious effects of the unknown words on reading comprehension are minimised. A related question is what vocabulary size would a reader need to have in order to achieve the optimum text coverage for a given text or class of texts. This thesis addresses these questions in three ways. First, a replication and expansion of a key study (Hu & Nation, 2000)1 was performed. In that study, 98% text coverage was found to be optimal for adequate reading comprehension of short fiction texts when reading for pleasure. To replicate that study, equivalent measures of reading comprehension were collected from a more homogeneous group of participants at a university in northern Thailand (n = 138), under stricter conditions and random assignment to one of three text coverage conditions, to verify the generalisability of the results. The original study was also expanded by measuring reader characteristics thought to contribute to reading comprehension, such as vocabulary size, l1 and l2 literacy, and reading attitudes, in an effort to improve the explainable reading comprehension variance. In order to more accurately calculate the text coverage a reader experiences for a particular text, both the vocabulary profile of the text and the vocabulary size of the reader must be known as precisely as possible. Therefore, to contribute to the question of vocabulary size, changes such as measuring item completion time and varying the order of item presentation were made to the VST (P. Nation & Beglar, 2007) to improve its sensitivity and accuracy. This may ultimately lead to increased precision when using text coverage to predict reading comprehension. Finally, l2 English vocabulary size norms were established to supplement the diagnostic usefulness of the VST. Data were collected through an online version of the VST created for this thesis from primarily self-selected participants (n 1:31 105) located in countries (n 100) around the world representing several l1 and age groups. Analysis of the data collected for this thesis suggest that text coverage explains much less reading comprehension variance than previously reported while vocabulary size may be a more powerful predictor. An internal replication of Hu and Nation (2000) found errors in the calculation of optimum text coverage and in the reported size of the effect on reading comprehension. A critical review of the theoretical foundations of the text coverage model of reading comprehension found serious flaws in construct operationalisation and research design. Due to these flaws, most research which has purported to measure the effect of text coverage on reading comprehension actually measured the effect of an intervening variable: readers’ vocabulary size. Vocabulary size norms derived from data collected through an online version of the VST appear to be reliable and representative. Varying item presentation order appears to increase test sensitivity. Despite a moderate effect for l1 English users, item completion time does not seem to account for any variance in vocabulary size scores for l2 English learners. Based on the finding that vocabulary size may explain both reading comprehension and text coverage, the putative power of text coverage to predict reading comprehension is challenged. However, an alternative measure which may offer greater power to predict reading comprehension, the VST, has been modified and made available online. This version of the VST may provide greater sensitivity and ease of use than the offline, paper-based version.</p>

Thresholds, Text Coverage, Vocabulary Size, and Reading Comprehension in Applied Linguistics

<p>The inextricable link between vocabulary knowledge and reading comprehension is incontrovertible. However, questions remain regarding the nature of the interaction. One question which remains unresolved is whether there is an optimum text coverage, or ratio of known to unknown words in a text, such that any deleterious effects of the unknown words on reading comprehension are minimised. A related question is what vocabulary size would a reader need to have in order to achieve the optimum text coverage for a given text or class of texts. This thesis addresses these questions in three ways. First, a replication and expansion of a key study (Hu & Nation, 2000)1 was performed. In that study, 98% text coverage was found to be optimal for adequate reading comprehension of short fiction texts when reading for pleasure. To replicate that study, equivalent measures of reading comprehension were collected from a more homogeneous group of participants at a university in northern Thailand (n = 138), under stricter conditions and random assignment to one of three text coverage conditions, to verify the generalisability of the results. The original study was also expanded by measuring reader characteristics thought to contribute to reading comprehension, such as vocabulary size, l1 and l2 literacy, and reading attitudes, in an effort to improve the explainable reading comprehension variance. In order to more accurately calculate the text coverage a reader experiences for a particular text, both the vocabulary profile of the text and the vocabulary size of the reader must be known as precisely as possible. Therefore, to contribute to the question of vocabulary size, changes such as measuring item completion time and varying the order of item presentation were made to the VST (P. Nation & Beglar, 2007) to improve its sensitivity and accuracy. This may ultimately lead to increased precision when using text coverage to predict reading comprehension. Finally, l2 English vocabulary size norms were established to supplement the diagnostic usefulness of the VST. Data were collected through an online version of the VST created for this thesis from primarily self-selected participants (n 1:31 105) located in countries (n 100) around the world representing several l1 and age groups. Analysis of the data collected for this thesis suggest that text coverage explains much less reading comprehension variance than previously reported while vocabulary size may be a more powerful predictor. An internal replication of Hu and Nation (2000) found errors in the calculation of optimum text coverage and in the reported size of the effect on reading comprehension. A critical review of the theoretical foundations of the text coverage model of reading comprehension found serious flaws in construct operationalisation and research design. Due to these flaws, most research which has purported to measure the effect of text coverage on reading comprehension actually measured the effect of an intervening variable: readers’ vocabulary size. Vocabulary size norms derived from data collected through an online version of the VST appear to be reliable and representative. Varying item presentation order appears to increase test sensitivity. Despite a moderate effect for l1 English users, item completion time does not seem to account for any variance in vocabulary size scores for l2 English learners. Based on the finding that vocabulary size may explain both reading comprehension and text coverage, the putative power of text coverage to predict reading comprehension is challenged. However, an alternative measure which may offer greater power to predict reading comprehension, the VST, has been modified and made available online. This version of the VST may provide greater sensitivity and ease of use than the offline, paper-based version.</p>

Analyzing variability in L2 ultimate attainment

Ultimate attainment is typically more heterogeneous among second-language (L2) learners than among native speakers (e.g. Bley-Vroman, 1990). The present study offers a suite of simple analytical procedures aimed at exploring types and loci of variability in L2 attainment vis-à-vis those in the corresponding first language (L1), with special attention to certain learner-external factors that might condition such variabilities. To demonstrate the methods and their potential, we apply these procedures to learner and native acceptability judgment data published in Birdsong (1992). Under means analyses of item ratings and coefficients of variation (CV), a group of adult Anglophone learners of L2 French (ENS) are found to resemble native French controls (FNS). In contrast, under correlational analyses of ratings and CVs, ENS resemble FNS on grammatical items, but diverge on ungrammatical items. Correlational and means analyses of both CV and acceptability ratings reveal that ENS-FNS convergence is not predictable from the degree of FNS ratings variability, contra DeKeyser (2012). For both groups, we observe an interaction between FNS ratings variability and the grammatical status of items (ungrammatical vs. grammatical). Finally, for neither group do we find a relationship between the order of item presentation and ratings severity or CVs. We present our perspectives as a road map for future analyses of variabilities inherent in language learning outcomes.

Gaze and Event Tracking for Evaluation of Recommendation-Driven Purchase

Recommendation systems play an important role in e-commerce turnover by presenting personalized recommendations. Due to the vast amount of marketing content online, users are less susceptible to these suggestions. In addition to the accuracy of a recommendation, its presentation, layout, and other visual aspects can improve its effectiveness. This study evaluates the visual aspects of recommender interfaces. Vertical and horizontal recommendation layouts are tested, along with different visual intensity levels of item presentation, and conclusions obtained with a number of popular machine learning methods are discussed. Results from the implicit feedback study of the effectiveness of recommending interfaces for four major e-commerce websites are presented. Two different methods of observing user behavior were used, i.e., eye-tracking and document object model (DOM) implicit event tracking in the browser, which allowed collecting a large amount of data related to user activity and physical parameters of recommending interfaces. Results have been analyzed in order to compare the reliability and applicability of both methods. Observations made with eye tracking and event tracking led to similar results regarding recommendation interface evaluation. In general, vertical interfaces showed higher effectiveness compared to horizontal ones, with the first and second positions working best, and the worse performance of horizontal interfaces probably being connected with banner blindness. Neural networks provided the best modeling results of the recommendation-driven purchase (RDP) phenomenon.

Amnesic patients' impaired category exemplar production priming

This study examined the effect of varying the number of potential target words on amnesic patients' category exemplar production performance. In Experiment 1, 4 words from each of 6 categories were presented to amnesic patients and normal control participants. This was followed by an indirect task in which each participant produced the first 8 words that came to mind when presented with a category cue. On this task the amnesic patients were impaired. This outcome stands in sharp contrast to most other category exemplar production tasks that have been reported. However, these other paradigms tend to restrict participants' processing during target item presentation while our procedure allowed them to analyze the target words as they chose. Our procedure may have allowed the control participant more opportunity to “cluster” target words from the same category during list presentation and this, in turn, may have given them an advantage at the time of category exemplar production. Therefore, in Experiment 2, only 1 word per category was presented in the target list and only 2 words per category were requested during category exemplar production. Surprisingly, the amnesic patients still exhibited impaired performance. Therefore, it was suggested that perhaps amnesic patients' known inability to perform semantic levels of processing during individual target word presentation may have resulted in impaired priming for categorical features for these patients. (JINS, 1998, 4, 576–583.)

“The Whole is More than the Sum of its Parts”: The Effects of Grouping and Randomizing Items on the Reliability and Validity of Questionnaires

This investigation was concerned with the widely spread practice to extract subscales from extensive psychodiagnostic inventories and to present their items in questionnaires as homogeneous item-blocks. By way of examining the effects of the mode of item-presentation on the reliability and validity, the frequently used and validated SCL-90-R was analyzed as a prototype of multidimensional symptom self-report inventories. Two studies were conducted in different contexts of application (study I: controlled group testing, study II: non-controlled individual testing) and involved different groups of subjects (study I: 130 nursing school students, study II: 134 university students). In both studies the standard item-arrangement was contrasted with the item-block presentation, which groups together items measuring the same dimension. The results revealed significant effects of item-blocking on the mean values, on the reliability, and the validity of the questionnaire. The findings seriously call into question the admissibility of subscale-extraction for self-report inventories. We conclude that the Gestalt paradigm, “The whole is more than the sum of its parts,” is valid for multidimensional psychodiagnostic inventories.