Investigating cross-linguistic similarity ratings: A Rasch analysis

A robust finding in psycholinguistics is that cognates and loanwords, which are words that typically share some degree of form and meaning across languages, provide the second language learner with benefits in language use when compared to words that do not share form and meaning across languages. This cognate effect has been shown to exist for Japanese learners of English; that is, words such as table are processed faster and more accurately in English because they have a loanword equivalent in Japanese (i.e., テーブル /te:buru/ ‘table’). Previous studies have also shown that the degree of phonological and semantic similarity, as measured on a numerical scale from ‘completely different’ to ‘identical’, also influences processing. However, there has been relatively little appraisal of such cross-linguistic similarity ratings themselves. Therefore, the present study investigated the structure of the similarity ratings using Rasch analysis, which is an analytic approach frequently used in the design and validation of language assessments. The findings showed that a 4-point scale may be optimal for phonological similarity ratings of cognates and a 2-point scale may be most appropriate for semantic similarity ratings. Furthermore, this study reveals that while a few raters and items misfitted the Rasch model, there was substantial agreement in ratings, especially for semantic similarity. The results validate the ratings for use in research and demonstrate the utility of Rasch analysis in the design and validation of research instruments in psychology.

The Construction and Validation of a New Listening Span Task

The listening span task is a measure of working memory that requires participants to process sets of increasing numbers of utterances and store the last word of each utterance for recall at the end of each set. Measures to date have contained an exceedingly demanding processing component, possibly leading to insufficient resources to meet the word recall requirement, which may have affected the sensitivity of the measure to distinguish different levels of working memory. Further, tasks thus far have asked participants to verify the content utterances based on knowledge, which may have confounded the measurement of working memory capacity with world knowledge. An additional weakness is that they lack sound psychometric construct validity evidence, which clouds what these tools actually measure. This pilot study presents a listening span task that accounts for preceding methodological shortcomings, which was administered to 31 Japanese junior high school students. The participants listened to ten sets (two sets of equal length of two, three, four, five and six utterances) of short casual utterances, judged whether they made sense in Japanese, and recalled the last word of each utterance in the set. Performance was assessed through a scoring procedure new to listening span tasks in which credit is given for the words recalled in order of appearance until memory failure. The data was analyzed through the Rasch model, which produces evidence for different aspects of validity and indicates if the items in a test measure a unidimensional construct. The results provided validity evidence for the use of the new listening span task and revealed that the instrument measured a single unidimensional construct.

Assessing critical thinking in L2: An exploratory study

Critical thinking (CT) is taking on an increasingly important role in Japanese tertiary education. Teachers tasked with developing CT in a second-language (L2) context may need a way of assessing students’ abilities. However, a number of difficulties face L2 students taking a test designed for first-language (L1) speakers. They may be disadvantaged by linguistic and perhaps cultural issues. This study describes an exploratory attempt to make a CT test that can be administered to learners of English and which allows them to display selected elements of CT, specifically analyzing arguments and judging or evaluating. A comparison of L1 and L2 performance in the test showed the results to be comparable. Analysis of two different question topics showed differences in CT skills displayed. Issues with rating accuracy are linked to the format of the test. We argue that this test format is suitable for many students in Japan and elsewhere who have intermediate levels of English.

Coming Full Circle —From CEFR to CEFR-J and back

The CEFR-J project was launched in Japan in 2008. The CEFR-J gives a set of Can Do descriptors for 10 CEFR sub- levels (Pre-A1 to B2.2) and related Reference Level Description (RLD) work, whilst including developed profiling for vocabulary, grammar, and textual features were developed. In this article, the English resources created for the CEFR-J are applied in preparing teaching resources for other major European languages as well as Asian languages. To achieve this, a series of teaching/learning resources including the CEFR-J Wordlist and Phrase List initially developed for English were translated into 27 other languages using neural machine translation. These translated word and phrase lists were then manually corrected by a team of language experts. The automatic conversion of English to other languages was evaluated against human judgments as well as frequency analysis referencing web corpora. Three types of e-learning resources were created, taking into consideration the wordlists and the phrase lists for teaching those languages to undergraduate students: (1) a flash-card app for learning vocabulary, which allows for classification by both thematic topic and CEFR level, (2) an online syntax writing tool for the study of grammar and vocabulary, and (3) an online spoken and written production corpus collection tool. Keywords: CEFR-J, multilingual resources, e-learning, machine translation, automatic conversion, NLP, multilingual corpora, web-based, writing tool, spoken production