The FEMTI guidelines for contextual MT evaluation: principles and resources

A large number of evaluation metrics exist for machine translation (MT) systems, but depending on the intended context of use of such a system, not all metrics are equally relevant. Based on the ISO/IEC 9126 and 14598 standards for software evaluation, the Framework for the Evaluation of Machine Translation in ISLE (FEMTI) provides guidelines for the selection of quality characteristics to be evaluated depending on the expected task, users, and input characteristics of an MT system. This approach to contextual evaluation was implemented as a web-based application which helps its users design evaluation plans. In addition, FEMTI offers experts in evaluation the possibility to enter and share their knowledge using a dedicated web-based tool, tested in several evaluation exercises.

METHODOLOGY FOR EVALUATING THE VALUE OF DEPLOYING IT SERVICES TO AUTOMATE ORGANIZATIONS

В работе рассмотрен функционал программного обеспечения, разработанного для программной оценки ценности реализации ИТ-сервисов для автоматизации процесса создания ИТ-сервисов организации. Предложена стоимостная функция для получения эффекта от ИТ-сервисов организации. The paper considers the functionality of software developed for software evaluation of the value of implementing it services to automate the process of creating it services for an organization. A cost function is proposed for getting the effect of an organization's it services.

Software Evaluation for De Novo Detection of Transposons

Transposable elements (TEs) are major genomic components in most eukaryotic genomes and play an important role in genome evolution. However, despite their relevance the identification of TEs is not an easy task and a number of tools were developed to tackle this problem. To better understand how they perform, we tested several widely used tools for de novo!TE detection and compared their performance on both simulated data and well curated genomic sequences. The results will be helpful for identifying common issues associated with TE-annotation and for evaluating how comparable are the results obtained with different tools.

Software Evaluation for de novo Detection of Transposons

Transposable elements (TEs) are major genomic components in most eukaryotic genomes and play an important role in genome evolution. However, despite their relevance the identification of TEs is not an easy task and a number of tools were developed to tackle this problem. To better understand how they perform, we tested several widely used tools for de novo TE detection and compared their performance on both simulated data and well curated genomic sequences. The results will be helpful for identifying common issues associated with TE-annotation and for evaluating how comparable are the results obtained with different tools.

Establishing Evaluation Criteria for Digital Dental Simulation Systems and the Differences between Software and Evaluators

Purpose: Cavity and tooth preparation are essential curriculum components in dental education for undergraduate students, and faculty staff determine whether the standard of the work is acceptable. However, evaluators are inevitably subject to variability and bias, so quantification of the evaluation is desirable. Therefore, various digital devices have been developed to quantify evaluations, such as SURFLACER, Simodont, and DentSim. In this study, we aimed to investigate the use of evaluation software and to compare human evaluation with software evaluation using a new digital dental education device with a realtime evaluation function. Material and Method: Dental students prepared a full metal crown on an artificial mandibular left first molar, which was evaluated by the software and a human evaluator (a dentist). Results: The results showed that the software evaluation was more suitable with eight cross-sections than with four crosssections. The dentist evaluated the same teeth according to a set of evaluation criteria, but there were no statistically significant differences in any of the evaluation items. Finally, multivariate analysis was conducted to assess the relationship between the software evaluation using eight cross-sections and the human evaluator using dummy variables, and it revealed that the margin form evaluation by the dentist was a significant explanatory variable for the score by the software. Conclusion: This preliminary study is the first to demonstrate that the new digital dental education device has considerable potential as a digital educational tool.