In Search of New Benchmarks: Using L2 Lexical Frequency and Contextual Diversity Indices to Assess Second Language Writing

Lexical items that are encountered more frequently and in varying contexts have important effects on second language (L2) development because frequent and contextually diverse words are learned faster and become more entrenched in a learner’s lexicon (Ellis 2002a, b). Despite evidence that L2 learners are generally exposed to non-native input, most frequency and contextual diversity metrics used in L2 research represent what is produced by native speakers of English. This study develops and tests indices of lexical frequency and contextual diversity based on L2 output. The L2 indices were derived from an L2 English learner adult corpus that contained three sub-corpora based on language levels (i.e. low, medium, and high). These indices were used to predict human scores of 480 independent essays from the Test of English as a Foreign Language (TOEFL). First language (L1) indices reported by the Tool for the Automatic Analysis of Lexical Sophistication (TAALES) were also calculated. Three regression analyses were run to predict human scores using L2 indices, L1 indices, and combined indices. The results suggested that the L2 model explained a greater amount of variance in the writing scores and that the L2 model was statistically superior to the L1 model. The findings also suggested that contextual diversity indices are better predictors of writing proficiency than lexical frequency for both the L2 and the L1 models. Finally, an index from the lower level learner sub-corpus was found to be the strongest predictor. The findings have important implications for the analysis of L2 writing in that the L2 benchmarks are more predictive than the L1 benchmarks. These findings could extend human and machine scoring approaches as well as help explain L2 writing quality.