Productivity and quality in the post-editing of outputs from translation memories and machine translation

Ana Guerberof Arenas, Productivity and quality in the post-editing of outputs from translation memories and machine translation

Abstract

Machine-translated segments are included as fuzzy matches within the translation-memory systems in the localization workflow. This study presents results on the correlation between these two types of segments in terms of productivity, final quality and experience. In order to test these variables, we set up an experiment with a group of twenty four professional translators using an on-line post-editing tool and a customized Moses statistical-base machine translation engine with a Bleu score of 0.6. The translators were asked to translate from English to Spanish no match, machine-translated and translation-memory segments from the 85-94 percent value using a post-editing tool without actually knowing the origin of each segment, and to complete an on-line questionnaire after the task. These translations were analyzed through a TER score calculation and they were reviewed by three professional translators to see the resulting quality of the assignment. The findings suggest that translators have higher productivity and quality when using machine-translated output than when translating on their own and that this productivity and quality is no significantly different than the one obtained when processing fuzzy matches from translation memories in the range 85-94 percent. Furthermore, translators’ experience and training seems to have an impact on the productivity and on the quality produced by translators.

Ana Guerberof Arenas did her PhD as part of the PhD program in Translation and Intercultural Studies, at the Universitat Rovira i Virgili, Spain. Her supervisors were Dr. Anthony Pym, Universitat Rovira i Virgili and Dr. Sharon O’Brien, Dublin City University. Her PhD was funded by the EAMT, CNGL, CrossLang and Logoscript.