Evaluating RBMT output for -ing forms: A study of four tar-get languages

-ing forms in English are reported to be problematic for Machine Transla-tion and are often the focus of rules in Controlled Language rule sets. We investigated how problematic -ing forms are for an RBMT system, translat-ing into four target languages in the IT domain. Constituent-based human evaluation was used and the results showed that, in general, -ing forms do not deserve their bad reputation. A comparison with the results of five automated MT evaluation metrics showed promising correlations. Some issues prevail, however, and can vary from target language to target lan-guage. We propose different strategies for dealing with these problems, such as Controlled Language rules, semi-automatic post-editing, source text tagging and “post-editing” the source text.

An in-depth analysis of the individual impact of controlled language rules on machine translation output: a mixed-methods approach

Examining the general impact of Controlled Language (CL) rules in the context of Machine Translation (MT) has been an area of research for many years. The present study focuses on the following question: how do CL rules impact MT output individually? By analysing a German corpus-based test suite of technical texts that have been translated into English by different MT systems, this study endeavours to answer this question at different levels: the general impact of CL rules (rule- and system-independent), their impact at rule level (system-independent) as well as at rule and system level. The results of five MT systems are analysed and contrasted: a rule-based system, a statistical system, two differently constructed hybrid systems, and a neural system. For this, a mixed-methods triangulation approach that includes error annotation, human evaluation, and automatic evaluation was applied. The data was analysed both qualitatively and quantitatively in terms of CL influence on the following parameters: number and type of MT errors, style and content quality, and scores of two automatic evaluation metrics. In line with many studies, the results show a general positive impact of the applied CL rules on the MT output. However, at rule level, only four rules proved to have positive effects on the aforementioned parameters; three rules had negative effects on the parameters; and two rules did not show any significant impact. At rule and system level, the rules affected the MT systems differently, as expected. Rules that had a positive impact on earlier MT approaches did not show the same impact on the neural MT approach. Furthermore, neural MT delivered distinctly better results than earlier MT approaches, namely the highest error-free, style and content quality rates both before and after applying the rules, which indicates that neural MT offers a promising solution that no longer requires CL rules for improving the MT output.

Designing the Lexical Rules for the Parsing of ASD-STE100 Function Words in ARTEMIS from a Role and Reference Grammar Perspective

ARTEMIS (Automatically Representing Text Meaning via an Interlingua-based System), is a natural language processing device, whose ultimate aim is to be able to understand natural language fragments and arrive at their syntactic and semantic representation. Linguistically, this parser is founded on two solid linguistic theories: the Lexical Constructional Model and Role and Reference Grammar. Although the rich semantic representations and the multilingual character of Role and Reference Grammar make it suitable for natural language understanding tasks, some changes to the model have proved necessary in order to adapt it to the functioning of the ARTEMIS parser. This paper will deal with one of the major modifications that Role and Reference Grammar had to undergo in this process of adaptation, namely, the substitution of the operator projection for feature-based structures, and how this will influence the description of function words in ARTEMIS, since they are strongly responsible for the encoding of the grammatical information which in Role and Reference Grammar is included in the operators. Currently, ARTEMIS is being implemented for the controlled natural language ASD-STE100, the Aerospace and Defence Industries Association of Europe Simplified Technical English, which is an international specification for the preparation of technical documentation in a controlled language. This controlled language is used in the belief that its simplified nature makes it a good corpus to carry out a preliminary testing of the adequacy of the parser. In this line, the aim of this work is to create a catalogue of function words in ARTEMIS for ASD-STE100, and to design the lexical rules necessary to parse the simple sentence and the referential phrase in this controlled language.

Errors in search strategies used in systematic reviews and their effects on information retrieval

Objectives: Errors in search strategies negatively affect the quality and validity of systematic reviews. The primary objective of this study was to evaluate searches performed in MEDLINE/PubMed to identify errors and determine their effects on information retrieval.Methods: A PubMed search was conducted using the systematic review filter to identify articles that were published in January of 2018. Systematic reviews or meta-analyses were selected from a systematic search for literature containing reproducible and explicit search strategies in MEDLINE/PubMed. Data were extracted from these studies related to ten types of errors and to the terms and phrases search modes.Results: The study included 137 systematic reviews in which the number of search strategies containing some type of error was very high (92.7%). Errors that affected recall were the most frequent (78.1%), and the most common search errors involved missing terms in both natural language and controlled language and those related to Medical Subject Headings (MeSH) search terms and the non-retrieval of their more specific terms.Conclusions: To improve the quality of searches and avoid errors, it is essential to plan the search strategy carefully, which includes consulting the MeSH database to identify the concepts and choose all appropriate terms, both descriptors and synonyms, and combining search techniques in the free-text and controlled-language fields, truncating the terms appropriately to retrieve all their variants.