scholarly journals The FEMTI guidelines for contextual MT evaluation: principles and resources

2021 ◽  
Vol 8 ◽  
Author(s):  
Paula Estrella ◽  
Andrei Popescu-Belis ◽  
Maghi King
Keyword(s):  
Machine Translation ◽  
Quality Characteristics ◽  
Evaluation Metrics ◽  
Design Evaluation ◽  
Software Evaluation ◽  
Web Based ◽  
Mt Evaluation ◽  
Context Of Use ◽  
Contextual Evaluation ◽  
Selection Of

A large number of evaluation metrics exist for machine translation (MT) systems, but depending on the intended context of use of such a system, not all metrics are equally relevant. Based on the ISO/IEC 9126 and 14598 standards for software evaluation, the Framework for the Evaluation of Machine Translation in ISLE (FEMTI) provides guidelines for the selection of quality characteristics to be evaluated depending on the expected task, users, and input characteristics of an MT system. This approach to contextual evaluation was implemented as a web-based application which helps its users design evaluation plans. In addition, FEMTI offers experts in evaluation the possibility to enter and share their knowledge using a dedicated web-based tool, tested in several evaluation exercises.

scholarly journals On Post-Editability of Machine Translated Texts

2021 ◽  
Vol 15 (1) ◽  
Author(s):  
Ch Ram Anirudh ◽  
Kavi Narayana Murthy
Keyword(s):  
Machine Translation ◽  
Quantitative Method ◽  
Evaluation Metrics ◽  
Indian Language ◽  
Actual Time ◽  
Mt Evaluation ◽  
Statistical Mt ◽  
Modern Machine

Machine Translated texts are often far from perfect and postediting is essential to get publishable quality. Post-editing may not always be a pleasant task. However, modern machine translation (MT) approaches like Statistical MT (SMT) and Neural MT (NMT) seem to hold greater promise. In this work, we present a quantitative method for scoring translations and computing the post-editability of MT system outputs.We show that the scores we get correlate well with MT evaluation metrics as also with the actual time and effort required for post-editing. We compare the outputs of three modern MT systems namely phrase-based SMT (PBMT), NMT, and Google translate for their Post-Editability for English to Hindi translation. Further, we explore the effect of various kinds of errors in MT outputs on postediting time and effort. Including an Indian language in this kind of post-editability study and analyzing the influence oferrors on postediting time and effort for NMT are highlights of this work.

Sistem Informasi Geografis (Sig) Properti Di Kota Tanjungpinang Berbasis Website

2019 ◽  
Vol 8 (1) ◽  
pp. 39-47
Author(s):  
Zulfachmi ◽  
Angger Andrea Amanda ◽  
Dedy Jauhari
Keyword(s):  
Information System ◽  
Geographic Information System ◽  
Unified Modeling Language ◽  
Geographic Information ◽  
Web Based ◽  
Unified Modeling ◽  
The Public ◽  
Property Specification ◽  
System Requirements ◽  
Selection Of

The increasing need for property in Tanjungpinang City is very growing, especially in the housing sector. Selection of property based on location and facilities and infrastructure is always a consideration for the community in making decisions to buy a property. Difficulty finding property location information in a certain area often occurs, resulting in people not getting references about the properties offered in Tanjungpinang City. The purpose of this research is to create a web-based geographic information system (GIS) regarding the distribution of the number of properties on offer, especially in Tanjungpinang City using a web-based mapping approach. In the development of Property GIS the author uses the Waterfall method and in the analysis of system requirements it is modeled with UML (Unified Modeling Language) and implemented with the PHP programming language and MySQLI database. It is hoped that the results of making this property's geographic information system can help the public to find out information about the distribution of properties offered, such as the location of property coordinates, addresses, prices, property photos, property specification data and property developer data.

UUX Method Selection

i-com ◽  
2016 ◽  
Vol 15 (1) ◽  
Author(s):  
Holger Fischer ◽  
Michaela Kauer-Franz ◽  
Dominique Winter ◽  
Stefan Latt
Keyword(s):  
Software Development ◽  
Present Article ◽  
User Experience ◽  
Interactive System ◽  
Human Centered Design ◽  
Method Selection ◽  
Development Processes ◽  
Context Of Use ◽  
Selection Of

AbstractThe establishment of human-centered design within software development processes is still a challenge. Numerous methods exist that aim to increase the usability and user experience of an interactive system. Nevertheless, the selection of appropriate methods remains to be challenging, as there are multiple different factors that have a significant impact on the appropriateness of the methods in their context of use. The present article investigates current strategies of method selection based on a conference workshop with practitioners. The results show that usability and user experience professionals concentrate on five to seven well-known methods and will need more support to select and use further ones.

scholarly journals An Open-Source Web-Based Tool for Resource-Agnostic Interactive Translation Prediction

2014 ◽  
Vol 102 (1) ◽  
pp. 69-80 ◽  
Author(s):  
Torregrosa Daniel ◽  
Forcada Mikel L. ◽  
Pérez-Ortiz Juan Antonio
Keyword(s):  
Open Source ◽  
Machine Translation ◽  
Web Application ◽  
Statistical Machine Translation ◽  
Black Box ◽  
Translation System ◽  
Web Tool ◽  
Web Based ◽  
Strongly Coupled ◽  
Machine Translation System

Abstract We present a web-based open-source tool for interactive translation prediction (ITP) and describe its underlying architecture. ITP systems assist human translators by making context-based computer-generated suggestions as they type. Most of the ITP systems in literature are strongly coupled with a statistical machine translation system that is conveniently adapted to provide the suggestions. Our system, however, follows a resource-agnostic approach and suggestions are obtained from any unmodified black-box bilingual resource. This paper reviews our ITP method and describes the architecture of Forecat, a web tool, partly based on the recent technology of web components, that eases the use of our ITP approach in any web application requiring this kind of translation assistance. We also evaluate the performance of our method when using an unmodified Moses-based statistical machine translation system as the bilingual resource.

scholarly journals Five Challenges for Intelligent Text Entry Methods

AI Magazine ◽  
2009 ◽  
Vol 30 (4) ◽  
pp. 85 ◽  
Author(s):  
Per Ola Kristensson
Keyword(s):  
Error Correction ◽  
Effective Means ◽  
Historical Context ◽  
Research Literature ◽  
Error Rates ◽  
Localization Error ◽  
Text Entry ◽  
Context Of Use ◽  
Selection Of

For text entry methods to be useful they have to deliver high entry rates and low error rates. At the same time they need to be easy-to-learn and provide effective means of correcting mistakes. Intelligent text entry methods combine AI techniques with HCI theory to enable users to enter text as efficiently and effortlessly as possible. Here I sample a selection of such techniques from the research literature and set them into their historical context. I then highlight five challenges for text entry methods that aspire to make an impact in our society: localization, error correction, editor support, feedback, and context of use.

scholarly journals Methods and applications for visualization of SNOMED CT concept sets

2014 ◽  
Vol 05 (01) ◽  
pp. 127-152 ◽  
Author(s):  
E. Sundvall ◽  
K.R. Gøeg ◽  
A.R. Højen
Keyword(s):  
Content Management ◽  
Graph Layout ◽  
Snomed Ct ◽  
Focus Group Interviews ◽  
Web Based ◽  
Hierarchical Relations ◽  
Potential Applications ◽  
Group Interviews ◽  
And Training ◽  
Selection Of

SummaryInconsistent use of SNOMED CT concepts may reduce comparability of information in health information systems. Terminology implementation should be approached by common strategies for navigating and selecting proper concepts. This study aims to explore ways of illustrating common pathways and ancestors of particular sets of concepts, to support consistent use of SNOMED CT and also assess potential applications for such visualizations.The open source prototype presented is an interactive web-based re-implementation of the terminology visualization tool TermViz that provides an overview of concepts and their hierarchical relations. It provides terminological features such as interactively rearranging graphs, fetching more concept nodes, highlighting least common parents and shared pathways in merged graphs etc.Four teams of three to four people used the prototype to complete a terminology mapping task and then, in focus group interviews, discussed the user experience and potential future tool usage. Potential purposes discussed included SNOMED CT search and training, consistent selection of concepts and content management.The evaluation indicated that the tool may be useful in many contexts especially if integrated with existing systems, and that the graph layout needs further tuning and development.Citation: Højen AR, Sundvall E, Gøeg KR. Methods and applications for visualization of SNOMED CT concept sets. Appl Clin Inf 2014; 5: 127–152http://dx.doi.org/10.4338/ACI-2013-09-RA-0071

scholarly journals Selection of Evaluation Metrics for Dialogue Breakdown Detection in Dialogue Breakdown Detection Challenge 3

2020 ◽  
Vol 35 (1) ◽  
pp. DSI-G_1-10
Author(s):  
Yuiko Tsunomori ◽  
Ryuichiro Higashinaka ◽  
Tetsuro Takahashi ◽  
Michimasa Inaba
Keyword(s):  
Evaluation Metrics ◽  
Selection Of

scholarly journals Using Word Embeddings to Enforce Document-Level Lexical Consistency in Machine Translation

2017 ◽  
Vol 108 (1) ◽  
pp. 85-96 ◽  
Author(s):  
Eva Martínez Garcia ◽  
Carles Creus ◽  
Cristina España-Bonet ◽  
Lluís Màrquez
Keyword(s):  
Machine Translation ◽  
Evaluation Metrics ◽  
Automatic Evaluation ◽  
Word Embeddings ◽  
Standard Document ◽  
Sentence Level ◽  
Word Translation ◽  
Stochastic Mechanism ◽  
Document Level

Abstract We integrate new mechanisms in a document-level machine translation decoder to improve the lexical consistency of document translations. First, we develop a document-level feature designed to score the lexical consistency of a translation. This feature, which applies to words that have been translated into different forms within the document, uses word embeddings to measure the adequacy of each word translation given its context. Second, we extend the decoder with a new stochastic mechanism that, at translation time, allows to introduce changes in the translation oriented to improve its lexical consistency. We evaluate our system on English–Spanish document translation, and we conduct automatic and manual assessments of its quality. The automatic evaluation metrics, applied mainly at sentence level, do not reflect significant variations. On the contrary, the manual evaluation shows that the system dealing with lexical consistency is preferred over both a standard sentence-level and a standard document-level phrase-based MT systems.

scholarly journals Duration and speed of speech events: A selection of methods

2015 ◽  
Vol 56 (1) ◽  
pp. 59-83
Author(s):  
Dafydd Gibbon ◽  
Katarzyna Klessa ◽  
Jolanta Bachan
Keyword(s):  
Software Tool ◽  
Worked Examples ◽  
Three Dimensions ◽  
Speech Technology ◽  
Web Based ◽  
Language Typology ◽  
Cart Analysis ◽  
Speech Timing ◽  
Speech Events ◽  
Selection Of

AbstractThe study of speech timing, i.e. the duration and speed or tempo of speech events, has increased in importance over the past twenty years, in particular in connection with increased demands for accuracy, intelligibility and naturalness in speech technology, with applications in language teaching and testing, and with the study of speech timing patterns in language typology. H owever, the methods used in such studies are very diverse, and so far there is no accessible overview of these methods. Since the field is too broad for us to provide an exhaustive account, we have made two choices: first, to provide a framework of paradigmatic (classificatory), syntagmatic (compositional) and functional (discourse-oriented) dimensions for duration analysis; and second, to provide worked examples of a selection of methods associated primarily with these three dimensions. Some of the methods which are covered are established state-of-the-art approaches (e.g. the paradigmatic Classification and Regression Trees, CART , analysis), others are discussed in a critical light (e.g. so-called ‘rhythm metrics’). A set of syntagmatic approaches applies to the tokenisation and tree parsing of duration hierarchies, based on speech annotations, and a functional approach describes duration distributions with sociolinguistic variables. Several of the methods are supported by a new web-based software tool for analysing annotated speech data, the Time Group Analyser.

Sign in / Sign up

Export Citation Format

Share Document