by Silvia Terribile
In today’s fast-paced translation industry, productivity is a top priority. More and more language service providers (LSPs) offer post-editing services, aiming to limit time and cost. But does machine translation really make linguists faster? This is one of the key questions I aimed to answer during my PhD at the University of Manchester.
I collaborated with a leading LSP, TranslateMedia (now part of Toppan Digital Language), to conduct the first large-scale investigation of translation and revision speed in human translation and in the post-editing of neural machine translation. My analysis considered real-world data from the LSP for 90 million words translated over 2.5 years by 879 translators. These linguists worked across 11 language pairs – English to Danish, Dutch, Finnish, French, German, Italian, Polish, Portuguese, Spanish, and French to English. They were requested to aim for the highest possible quality in both translation services.
So, what did I find? At the translation stage, post-editing was on average 66% faster than human translation. But things are not always this straightforward. The impact of post-editing on speed varied dramatically among different language pairs, from an average speed increase of +130% in English to French, to an average decrease of -7% in English to Swedish. This may be due to a wide range of reasons, such as variations in machine translation quality in different language combinations, individual translators’ skills and ways of working, and many more. Surprisingly, revision was on average 38% faster in post-editing than in human translation. The revision process is not supposed to differ considerably whether carried out after the translation stage of post-editing or human translation, but many factors could have contributed to this result. For example, I hypothesised that some revisers might perceive post-editing as ‘inferior’ to human translation. This could lead them to be less accurate and complete post-editing revision faster. However, due to the uncontrolled conditions under which the analysed data were collected, it was not possible for me to determine exactly what led to these results.
To complicate things further, average speed values can mislead us: in the data for English to Polish translation, average values would suggest that post-editing was 18% faster than human translation. However, an analysis of the data distribution revealed that post-editing had been on average 4% slower than human translation in 89% of the translation tasks.
There was also very high variability in the speed obtained in different translation tasks, in the average values achieved by different linguists, and in the speed obtained by the same translator when working on different texts. This was the case both in post-editing and human translation, and both at the translation and revision stages. A key takeaway? Technology has great potential to enhance productivity, but we cannot assume that it will always make us faster.
Does post-editing speed mostly depend on the number of corrections that linguists make to the machine translation output? Not really. In the data analysed in this research, edit distance – a metric that measures the number of characters edited in the machine translation output – did not correlate strongly with speed. Some LSPs use edit distance as a proxy for determining the remuneration of freelance post-editors. LSPs, please don’t do this! This is unfair for your translators, and it may encourage counterproductive behaviour like intentional ‘overediting’.
Silvia Terribile is a researcher in Translation Studies at the University of Manchester. You can find and follow her on LinkedIn.
