Automated annotation of learner English

2021 ◽  
Vol 7 (1) ◽  
Author(s):  
Adriana Picoral ◽  
Shelley Staples ◽  
Randi Reppen
Keyword(s):  
Language Processing ◽  
Academic Writing ◽  
Gold Standard ◽  
Applied Linguistics ◽  
English Speakers ◽  
Automated Annotation ◽  
Linguistic Annotation ◽  
Argumentative Texts ◽  
L1 And L2 ◽  
Learner Language

Abstract This paper explores the use of natural language processing (NLP) tools and their utility for learner language analyses through a comparison of automatic linguistic annotation against a gold standard produced by humans. While there are a number of automated annotation tools for English currently available, little research is available on the accuracy of these tools when annotating learner data. We compare the performance of three linguistic annotation tools (a tagger and two parsers) on academic writing in English produced by learners (both L1 and L2 English speakers). We focus on lexico-grammatical patterns, including both phrasal and clausal features, since these are frequently investigated in applied linguistics studies. Our results report both precision and recall of annotation output for argumentative texts in English across four L1s: Arabic, Chinese, English, and Korean. We close with a discussion of the benefits and drawbacks of using automatic tools to annotate learner language.

scholarly journals Comparison of natural language processing tools for automatic gene ontology annotation of scientific literature

2018 ◽  
Author(s):  
Lucas Beasley ◽  
Prashanti Manda
Keyword(s):  
Gene Ontology ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Gold Standard ◽  
Scientific Literature ◽  
Biomedical Literature ◽  
Automated Annotation ◽  
Manual Curation ◽  
Gold Standard Reference

Manual curation of scientific literature for ontology-based knowledge representation has proven infeasible and unscalable to the large and growing volume of scientific literature. Automated annotation solutions that leverage text mining and Natural Language Processing (NLP) have been developed to ameliorate the problem of literature curation. These NLP approaches use parsing, syntactical, and lexical analysis of text to recognize and annotate pieces of text with ontology concepts. Here, we conduct a comparison of four state of the art NLP tools at the task of recognizing Gene Ontology concepts from biomedical literature using the Colorado Richly Annotated Full-Text (CRAFT) corpus as a gold standard reference. We demonstrate the use of semantic similarity metrics to compare NLP tool annotations to the gold standard.

scholarly journals Comparison of natural language processing tools for automatic gene ontology annotation of scientific literature

2018 ◽  
Author(s):  
Lucas Beasley ◽  
Prashanti Manda
Keyword(s):  
Gene Ontology ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Gold Standard ◽  
Scientific Literature ◽  
Biomedical Literature ◽  
Automated Annotation ◽  
Manual Curation ◽  
Gold Standard Reference

Manual curation of scientific literature for ontology-based knowledge representation has proven infeasible and unscalable to the large and growing volume of scientific literature. Automated annotation solutions that leverage text mining and Natural Language Processing (NLP) have been developed to ameliorate the problem of literature curation. These NLP approaches use parsing, syntactical, and lexical analysis of text to recognize and annotate pieces of text with ontology concepts. Here, we conduct a comparison of four state of the art NLP tools at the task of recognizing Gene Ontology concepts from biomedical literature using the Colorado Richly Annotated Full-Text (CRAFT) corpus as a gold standard reference. We demonstrate the use of semantic similarity metrics to compare NLP tool annotations to the gold standard.

scholarly journals Lexical Bundles of L1 and L2 English Professional Scholars: A Contrastive Corpus-Driven Study on Applied Linguistics Research Articles

2020 ◽  
Vol 6 (4) ◽  
pp. 76-89
Author(s):  
Muchamad Sholakhuddin Al Fajri ◽  
Angkita Wasito Kirana ◽  
Celya Intan Kharisma Putri
Keyword(s):  
Academic Writing ◽  
Academic Discipline ◽  
Applied Linguistics ◽  
Lexical Bundles ◽  
English Writing ◽  
Functional Types ◽  
Frequent Use ◽  
L1 And L2 ◽  
Professional Writers ◽  
L1 English

The current study examined the structural and functional types of four-word lexical bundles in two different corpora of applied linguistics scientific articles written by L1 English and L1 Indonesian professional writers. The findings show that L2 writers employed a higher number of bundles than L1 writers, but L2 writers underused some of the most typical lexical bundles in L1 English writing. Structurally, unlike previous studies, this study reports the frequent use of prepositional phrase (PP) - based bundles in the articles of L2 writers. However, besides the high frequency of PP-based bundles, L2 authors also used a high number of verbal phrase-based bundles, suggesting that these L2 writers were still acquiring more native-like bundles. In terms of functional types, L2 writers employed fewer quantification bundles than their counterparts. This study has potential implications for teaching English for academic writing. Teachers need to raise their students’ awareness of the most frequently used lexical bundles in a specific academic discipline and pay attention to the discourse conventions of academic writing, helping L2 students transition from clausal to phrasal styles.

CORPUS LINGUISTICS, LANGUAGE CORPORA AND LANGUAGE TEACHING

2020 ◽  
Vol 14 (2) ◽  
pp. 75
Author(s):  
Eska Perdana Prasetya ◽  
Anita Dewi Ekawati ◽  
Deni Sapta Nugraha ◽  
Ahmad Marzuq ◽  
Tiara Saputri Darlis
Keyword(s):  
Academic Writing ◽  
Corpus Linguistics ◽  
Language Use ◽  
Language Teaching ◽  
Applied Linguistics

<span lang="EN-GB">This research is about Corpus Linguistics, Language Corpora, And Language Teaching. As we know about this science is relatively new and is associated with technology. There are several areas discussed in this study such as several important parts of the corpus, the information generated in the corpus, four main characteristics of the corpus, Types of Corpora, Corpora in Language Teaching, several types that could be related to corpus research, Applications of corpus linguistics to language teaching may be direct or indirect. The field of applied linguistics analyses large collections of written and spoken texts, which have been carefully designed to represent specific domains of language use, such as informal speech or academic writing.</span>

scholarly journals From “No, she does” to “Yes, she does”: Negation processing in negative yes–no questions by Mandarin speakers of English

2021 ◽  
pp. 1-31
Author(s):  
Haoruo Zhang ◽  
Norbert Vanek
Keyword(s):  
Second Language ◽  
Reaction Time ◽  
English Speakers ◽  
Production Experiment ◽  
Mandarin Speakers ◽  
L1 And L2 ◽  
Negative Questions

Abstract In response to negative yes–no questions (e.g., Doesn’t she like cats?), typical English answers (Yes, she does/No, she doesn’t) peculiarly vary from those in Mandarin (No, she does/Yes, she doesn’t). What are the processing consequences of these markedly different conventionalized linguistic responses to achieve the same communicative goals? And if English and Mandarin speakers process negative questions differently, to what extent does processing change in Mandarin–English sequential bilinguals? Two experiments addressed these questions. Mandarin–English bilinguals, English and Mandarin monolinguals (N = 40/group) were tested in a production experiment (Expt. 1). The task was to formulate answers to positive/negative yes–no questions. The same participants were also tested in a comprehension experiment (Expt. 2), in which they had to answer positive/negative questions with time-measured yes/no button presses. In both Expt. 1 and Expt. 2, English and Mandarin speakers showed language-specific yes/no answers to negative questions. Also, in both experiments, English speakers showed a reaction-time advantage over Mandarin speakers in negation conditions. Bilingual’s performance was in-between that of the L1 and L2 baseline. These findings are suggestive of language-specific processing of negative questions. They also signal that the ways in which bilinguals process negative questions are susceptible to restructuring driven by the second language.

scholarly journals Hidden features in global knowledge production: (re)positioning theory and practice in academic writing

2017 ◽  
Vol 17 (2) ◽  
pp. 355-378 ◽  
Author(s):  
Joel Windle
Keyword(s):  
Academic Writing ◽  
Knowledge Production ◽  
Applied Linguistics ◽  
Positioning Theory ◽  
Theory And Practice ◽  
Academic Literacies ◽  
Global Knowledge ◽  
Post Colonial ◽  
Colonial Theory ◽  
Global North

ABSTRACT A key challenge for applied linguistics is how to deal with the historical power imbalance in knowledge production between the global north and south. A central objective of critical applied linguistics has been to provide new epistemological foundations that address this problem, through the lenses of post-colonial theory, for example. This article shows how the structure of academic writing, even within critical traditions, can reinforce unequal transnational relations of knowledge. Analysis of Brazilian theses and publications that draw on the multiliteracies framework identifies a series of discursive moves that constitute “hidden features” (STREET, 2009), positioning “northern” theory as universal and “southern” empirical applications as locally bounded. The article offers a set of questions for critical reflection during the writing process, contributing to the literature on academic literacies.

scholarly journals Learning from Disagreement: A Survey

2021 ◽  
Vol 72 ◽  
pp. 1385-1470
Author(s):  
Alexandra N. Uma ◽  
Tommaso Fornaciari ◽  
Dirk Hovy ◽  
Silviu Paun ◽  
Barbara Plank ◽  
...  
Keyword(s):  
Language Processing ◽  
Gold Standard ◽  
Training Methods ◽  
High Quality ◽  
Training Models ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
Growing Body ◽  
Research Questions ◽  
Speech Tagging

Many tasks in Natural Language Processing (NLP) and Computer Vision (CV) offer evidence that humans disagree, from objective tasks such as part-of-speech tagging to more subjective tasks such as classifying an image or deciding whether a proposition follows from certain premises. While most learning in artificial intelligence (AI) still relies on the assumption that a single (gold) interpretation exists for each item, a growing body of research aims to develop learning methods that do not rely on this assumption. In this survey, we review the evidence for disagreements on NLP and CV tasks, focusing on tasks for which substantial datasets containing this information have been created. We discuss the most popular approaches to training models from datasets containing multiple judgments potentially in disagreement. We systematically compare these different approaches by training them with each of the available datasets, considering several ways to evaluate the resulting models. Finally, we discuss the results in depth, focusing on four key research questions, and assess how the type of evaluation and the characteristics of a dataset determine the answers to these questions. Our results suggest, first of all, that even if we abandon the assumption of a gold standard, it is still essential to reach a consensus on how to evaluate models. This is because the relative performance of the various training methods is critically affected by the chosen form of evaluation. Secondly, we observed a strong dataset effect. With substantial datasets, providing many judgments by high-quality coders for each item, training directly with soft labels achieved better results than training from aggregated or even gold labels. This result holds for both hard and soft evaluation. But when the above conditions do not hold, leveraging both gold and soft labels generally achieved the best results in the hard evaluation. All datasets and models employed in this paper are freely available as supplementary materials.

scholarly journals Similarity-based interference and relative clauses in second language processing

2021 ◽  
pp. 026765832110635
Author(s):  
Ian Cunnings ◽  
Hiroki Fujita
Keyword(s):  
Language Processing ◽  
Relative Clause ◽  
Relative Clauses ◽  
Large Body ◽  
Interference Effects ◽  
Second Language Processing ◽  
L2 Processing ◽  
Relative Disadvantage ◽  
L1 And L2 ◽  
Object Relative

Relative clauses have long been examined in research on first (L1) and second (L2) language acquisition and processing, and a large body of research has shown that object relative clauses (e.g. ‘The boy that the girl saw’) are more difficult to process than subject relative clauses (e.g. ‘The boy that saw the girl’). Although there are different accounts of this finding, memory-based factors have been argued to play a role in explaining the object relative disadvantage. Evidence of memory-based factors in relative clause processing comes from studies indicating that representational similarity influences the difficulty associated with object relatives as a result of a phenomenon known as similarity-based interference. Although similarity-based interference has been well studied in L1 processing, less is known about how it influences L2 processing. We report two studies – an eye-tracking experiment and a comprehension task – investigating interference in the comprehension of relative clauses in L1 and L2 readers. Our results indicated similarity-based interference in the processing of object relative clauses in both L1 and L2 readers, with no significant differences in the size of interference effects between the two groups. These results highlight the importance of considering memory-based factors when examining L2 processing.

Sign in / Sign up

Export Citation Format

Share Document