Machine translation (MT) tools like Google Translate can overcome language barriers and increase ... more Machine translation (MT) tools like Google Translate can overcome language barriers and increase access to information. These tools also carry risks, and their societal role remains understudied. This article investigates typical uses and perceptions of MT based on a survey of 1200 United Kingdom residents who were representative of the national population in terms of age, sex, and ethnicity. We highlight three main findings from our analysis. First, participants often used MT for non-essential purposes that rarely justified professional human translations. Second, while they were highly satisfied with MT they also expressed desires for higher MT quality. These desires were usually motivated by expectations of perfection rather than fitness for purpose. Third, participants' future vision for MT involved increasingly blurred boundaries between text and speech. The article calls for more MT research on the interface between written and spoken communication and on the ethical implications of rare but significant high-risk uses of the technology.
Machine translation (MT) tools are widely available. They may be present in different spaces in w... more Machine translation (MT) tools are widely available. They may be present in different spaces in ways that consumers of the content do not necessarily control or realise, and research to date has paid little attention to these human-MT encounters. This article draws on the philosophy of technology literature to consider implications of MT’s permeating presence in online environments as well as in face-to-face interactions. The focus of the article is on two situations where humans can come across MT: while browsing websites and when speaking with figures of authority. The article highlights ways in which humans’ relationship with MT transcends conscious decisions to operate an MT tool directly. It argues that the human-MT relationship can also be one of immersion where MT blends with the environment in ways that, on the one hand, break language barriers but, on the other, influence, persuade, and on occasion misinform.
The ready availability of machine translation (MT) systems such as Google Translate has profoundl... more The ready availability of machine translation (MT) systems such as Google Translate has profoundly changed how society engages with multilingual communication practices. In addition to private use situations, this technology is now used to overcome language barriers in high-risk settings such as hospitals and courts. MT errors pose serious risks in environments like these, but there is little understanding of the nature of these risks and of the wider implications of using this technology. This article is the first structured study of the consequences of uninformed MT use in healthcare and law. Based on a critical literature review, the article presents a qualitative meta-analysis of official documents and published research on the use of MT in these two fields. Its findings prompt calls for action in three areas. First, the review shows that research on MT use in healthcare and law can often disregard the complexities of language and language translation. The article calls for cross-disciplinary research to address this gap by ensuring that a growing body of relevant knowledge in translation studies informs research conducted within the medical and legal sectors. Second, the review highlights a broad societal need for higher levels of awareness of the specific strengths and, crucially, of the limitations of MT. Finally, the article concludes that MT technology can in its current state exacerbate social inequalities and put certain communities of users at greater risk. We highlight this as a persistent issue that merits further attention from researchers and policymakers.
The use of machine translation (MT) in professional translation tasks can change not only how tra... more The use of machine translation (MT) in professional translation tasks can change not only how translators work, but also how projects are managed and the expectations they entail across translation supply chains. Previous research has looked extensively into translators’ attitudes to MT but has often ignored important aspects of how translators’ views interact with those of other language industry stakeholders. This article presents a contrastive analysis of attitudes to MT which covers management and production perspectives. The discussion draws on semi-structured interviews which were thematically coded and qualitatively examined. The study shows how MT adds uncertainty to translation production networks. It argues that the challenges posed by MT are exacerbated by how the current makeup of the language industry restricts translators’ field of influence to texts while possibly alienating them from wider aspects of business strategy. The article makes two suggestions. First, it calls for increased translator involvement in the management aspects of service provision. Second, it emphasises the need for a deeper discussion of MT which, rather than framing the technology itself as a potential ‘threat’, addresses broader societal issues involving misguided perceptions and mismatched expectations.
Translation is currently described as a profession under pressure from automation, falling prices... more Translation is currently described as a profession under pressure from automation, falling prices and globalized competition. Translators’ stance on machine translation (MT) is famously negative, but the economic dimension of this positioning is scarcely researched and often unclear. This article provides an analysis of translators’ blog and forum postings contextualized within general trends in employment, the economy and work automation. The analysis concentrates on MT and pay. Two key findings are reported. First, MT was found to be a secondary issue in translators’ comments on pay; most grievances were based on business practices themselves. Second, most criticisms of MT were rooted not in fears of being outperformed by MT systems, but rather in the technology’s limitations and market consequences. This article calls for a broadening of translators’ role across areas of specialization and argues that, in the debate on translation’s future, MT cannot be decoupled from its economic effects.
This report provides an insight into the use of machine translation (MT) in human written transla... more This report provides an insight into the use of machine translation (MT) in human written translation settings.
The report has two parts.
Part I presents findings from interviews conducted with technology specialists, project managers, managing directors and professional translators between March 2016 and October 2017. Thirty interviews lasting nearly twenty hours in total were conducted with participants in eleven countries. The interview findings were presented at an industry-academia knowledge exchange event held at the University of Bristol in January 2018.
Part II presents findings from this event.
Recommendations emerging from the interview findings and from discussions held at the Bristol knowledge exchange event include:
Avoiding the use of measures based on edit distance as the only parameter used to calculate post-editing rates; A need for more research and development initiatives that investigate reliable alternatives to word rates; Preventing negative and non-transparent uses of activity tracking (i.e. tracking of translating time and/or keyboarding); Improving communication and transparency to ensure that all members of translation supply chains are in synch regarding concepts, expectations and product specifications; More training to match editing skills and knowledge of machine translation across professional translation teams; A need to educate society and end-clients of what to expect from machine translation technology.
Post-editing of machine translation (MT) is now increasingly implemented in the human translation... more Post-editing of machine translation (MT) is now increasingly implemented in the human translation workflow after studies in both industry and academia have demonstrated the efficacy of this practice. Post-editing still involves open questions, however, such as how best to train post-editors and how to estimate the effort required by post-editing tasks. In attempting to address some of these questions, many previous studies investigate the post-editing process, but less research has focused on the post-edited product. This chapter examines the link between the process and product of post-editing by checking to see how post-editing effort data related to the quality of post-edited texts, assessed in terms of fluency (linguistic quality) and adequacy (translation accuracy). A statistical analysis indicated that the association between editing operations and the fluency of post-edited texts is dependent on the quality of the raw MT output. Interestingly, a negative association was observed between the number of eye fixations on the text and the quality of the post-edited translations. The chapter shows empirical evidence supporting the distinction between the concepts of translation fluency and adequacy, and postulates that automatic processes play a central role in post-editing performance.
This paper is an exercise of imagination. Based on Kay's (1980) inspiring idea of a translator's ... more This paper is an exercise of imagination. Based on Kay's (1980) inspiring idea of a translator's amanuensis, we attempt to describe a post-editing tool that enables ubiquitous translation (Cronin 2010). We argue that a parallelism exists between media remediation (Bolter and Grusin 1999) and the shifting phase translation is undergoing, with machine translation post-editing having an impact on the global workflow of translated content. We take the hybridisation of traditional and machine translation processes as a starting point to envisage the features of forthcoming translation technologies. Results of previous surveys helped us to select features expected to play a central role: versatile devices to which we broadly refer as displayers would enable ubiquity; a relevant knowledge feature would provide human translators with a well-assorted repertoire of reliable sources; and an effort prediction feature would provide post-editors with reliable estimates of how much work lay ahead. Interacting with the Translator's Amanuensis 2020 would not always be straightforward, however. Translators will have to adapt to richer ways of reading and visualising information. Ultimately, we argue that the Translator's Amanuensis 2020 could benefit from existing Translation Studies concepts: the study of translation problems, translation competence models, and the ethics and sociology of translation.
Post-editing of machine translation is gaining popularity as a solution to the ever-increasing de... more Post-editing of machine translation is gaining popularity as a solution to the ever-increasing demands placed on human translators. There has been a great deal of research in this area aimed at determining the feasibility of post-editing and at predicting post-editing effort based on source-text features and machine translation errors. However, considerably less is known about the mental workings of post-editing and post-editors’ decision-making or, in particular, the relationship between post-editing effort and different mental processes. This paper investigates these issues by analysing data from a think-aloud study through the lens of eye movements and subjective ratings obtained in a separate task. The results show that mental processes associated with grammar and lexis are significantly associated with cognitive effort in post-editing. This association was not observed for other aspects of the task concerning, for example, discourse or the real-world use of the text. In addition, it was noted that lexical issues are linked to long sequences of thought processes. The paper shows that lexis plays a central role in post-editing, and argues that more emphasis should be placed on this issue in future research and in post-editor training.
In the context of non-literary translation, and under appropriate circumstances, it is now largel... more In the context of non-literary translation, and under appropriate circumstances, it is now largely uncontroversial that the use of machine translation can increase translators' productivity without a detrimental effect on product quality. Advances in technology are giving rise to new forms of human-computer interaction, and the use of machine translation in human translation workflows is increasingly commonplace. In this changing context, this issue aims at mapping and evaluating current post-editing practices in the industry, academia, and society in general, acting as a platform for the discussion of various issues in this area. We envisage two connected, but distinct, approaches to the study of post-editing as being particularly suitable to addressing the issue's aim: a strictly controlled approach that looks closely at how post-editors undertake the tasks at hand, and a more sociology-oriented approach that looks at broader aspects of how different parties react to the use of machine translation. Articles adopting either of these approaches will be of interest to the issue, and contributions that attempt to combine them will be especially welcome.
There has been growing interest of late in the cognitive effort required by post-editing of machi... more There has been growing interest of late in the cognitive effort required by post-editing of machine translation. Compared to number of editing operations, cogni-tive (or mental) effort is frequently considered a more decisive indicator of the overall effort expended by post-editors. Estimating cognitive effort is not straightforward, however. Previous studies often triangulate different measures to obtain a consensus, but little post-editing research to date has attempted to show how measures of cogni-tive effort relate to each other in a multivariate analysis. This paper addresses this by presenting an exploratory comparison of cognitive measures based on eye tracking, pauses, editing time, and subjective ratings collected in a post-editing task carried out by professional and non-professional participants. All measures correlated with each other, but a principal components analysis showed that the measures cluster together in different ways. In particular, measures that increase with task time alone behaved differently from the others, with higher mutual associations and higher reliability. Regarding differences between professional and non-professional participants, it was observed that subjective ratings were overall more strongly associated with objective measures in the case of professionals. Surprising findings from previous research based on pause ratio are discussed. The paper argues that a pause typology will benefit the study of pause lengths and cognitive effort in post-editing.
This thesis investigates the expenditure of cognitive effort in post-editing of machine translati... more This thesis investigates the expenditure of cognitive effort in post-editing of machine translation. A mixed-method approach involving the use of eye movements, subjective ratings and think-aloud protocols was adopted for the investigation. The project aims at revealing connections between cognitive effort and variables including linguistic characteristics of the source text and the machine-translation output, post-editors’ individual traits, different linguistic aspects of the activity attended to during the task, and the quality of the post-edited texts, assessed by human translators in terms of fluency (linguistic quality) and adequacy (faithfulness to the source text). Two tasks were conducted to pursue these aims: one involving eye tracking and a self-report scale of cognitive effort, and another carried out by a different, but comparable, sample of participants, under a think-aloud condition. Results indicate that variables such as an automatic machine-translation quality score and source-text type-token ratio are good predictors of cognitive effort in post-editing. The relationship between cognitive effort and post-editors’ traits was found to be a complex one, with significant links in this respect only appearing in the context of interactions between variables. A complex relationship was also found between editing behaviour and the quality of the post-edited text: the number of changes implemented was found to have a generally positive association with post-edited fluency, though cognitive effort was found to be negatively correlated with both the fluency and adequacy of the post-edited texts. Mental processes involving grammar and lexis were significantly related to the levels of cognitive effort expended by participants. These were also the aspects most frequently attended to in the activity. From a methodological perspective, despite the criticisms received by the think-aloud method in previous research, empirical data obtained in this project indicates that think-aloud protocols correlate with eye movements and subjective ratings as measures of cognitive effort.
Identifying indices of effort in post-editing of machine translation can have a number of applica... more Identifying indices of effort in post-editing of machine translation can have a number of applications, including estimating machine translation quality and calculating post-editors’ pay rates. Both source-text and machine-output features as well as subjects’ traits are investigated here in view of their impact on cognitive effort, which is measured with eye tracking and a subjective scale borrowed from the field of Educational Psychology. Data is analysed with mixed-effects models, and results indicate that the semantics-based automatic evaluation metric Meteor is significantly correlated with all measures of cognitive effort considered. Smaller effects are also observed for source-text linguistic features. Further insight is provided into the role of the source text in post-editing, with results suggesting that consulting the source text is only associated with how cognitively demanding the task is perceived in the case of those with a low level of proficiency in the source language. Subjects’ working memory capacity was also taken into account and a relationship with post-editing productivity could be noticed. Scaled-up studies into the construct of working memory capacity and the use of eye tracking in models for quality estimation are suggested as future work.
Machine translation (MT) tools like Google Translate can overcome language barriers and increase ... more Machine translation (MT) tools like Google Translate can overcome language barriers and increase access to information. These tools also carry risks, and their societal role remains understudied. This article investigates typical uses and perceptions of MT based on a survey of 1200 United Kingdom residents who were representative of the national population in terms of age, sex, and ethnicity. We highlight three main findings from our analysis. First, participants often used MT for non-essential purposes that rarely justified professional human translations. Second, while they were highly satisfied with MT they also expressed desires for higher MT quality. These desires were usually motivated by expectations of perfection rather than fitness for purpose. Third, participants' future vision for MT involved increasingly blurred boundaries between text and speech. The article calls for more MT research on the interface between written and spoken communication and on the ethical implications of rare but significant high-risk uses of the technology.
Machine translation (MT) tools are widely available. They may be present in different spaces in w... more Machine translation (MT) tools are widely available. They may be present in different spaces in ways that consumers of the content do not necessarily control or realise, and research to date has paid little attention to these human-MT encounters. This article draws on the philosophy of technology literature to consider implications of MT’s permeating presence in online environments as well as in face-to-face interactions. The focus of the article is on two situations where humans can come across MT: while browsing websites and when speaking with figures of authority. The article highlights ways in which humans’ relationship with MT transcends conscious decisions to operate an MT tool directly. It argues that the human-MT relationship can also be one of immersion where MT blends with the environment in ways that, on the one hand, break language barriers but, on the other, influence, persuade, and on occasion misinform.
The ready availability of machine translation (MT) systems such as Google Translate has profoundl... more The ready availability of machine translation (MT) systems such as Google Translate has profoundly changed how society engages with multilingual communication practices. In addition to private use situations, this technology is now used to overcome language barriers in high-risk settings such as hospitals and courts. MT errors pose serious risks in environments like these, but there is little understanding of the nature of these risks and of the wider implications of using this technology. This article is the first structured study of the consequences of uninformed MT use in healthcare and law. Based on a critical literature review, the article presents a qualitative meta-analysis of official documents and published research on the use of MT in these two fields. Its findings prompt calls for action in three areas. First, the review shows that research on MT use in healthcare and law can often disregard the complexities of language and language translation. The article calls for cross-disciplinary research to address this gap by ensuring that a growing body of relevant knowledge in translation studies informs research conducted within the medical and legal sectors. Second, the review highlights a broad societal need for higher levels of awareness of the specific strengths and, crucially, of the limitations of MT. Finally, the article concludes that MT technology can in its current state exacerbate social inequalities and put certain communities of users at greater risk. We highlight this as a persistent issue that merits further attention from researchers and policymakers.
The use of machine translation (MT) in professional translation tasks can change not only how tra... more The use of machine translation (MT) in professional translation tasks can change not only how translators work, but also how projects are managed and the expectations they entail across translation supply chains. Previous research has looked extensively into translators’ attitudes to MT but has often ignored important aspects of how translators’ views interact with those of other language industry stakeholders. This article presents a contrastive analysis of attitudes to MT which covers management and production perspectives. The discussion draws on semi-structured interviews which were thematically coded and qualitatively examined. The study shows how MT adds uncertainty to translation production networks. It argues that the challenges posed by MT are exacerbated by how the current makeup of the language industry restricts translators’ field of influence to texts while possibly alienating them from wider aspects of business strategy. The article makes two suggestions. First, it calls for increased translator involvement in the management aspects of service provision. Second, it emphasises the need for a deeper discussion of MT which, rather than framing the technology itself as a potential ‘threat’, addresses broader societal issues involving misguided perceptions and mismatched expectations.
Translation is currently described as a profession under pressure from automation, falling prices... more Translation is currently described as a profession under pressure from automation, falling prices and globalized competition. Translators’ stance on machine translation (MT) is famously negative, but the economic dimension of this positioning is scarcely researched and often unclear. This article provides an analysis of translators’ blog and forum postings contextualized within general trends in employment, the economy and work automation. The analysis concentrates on MT and pay. Two key findings are reported. First, MT was found to be a secondary issue in translators’ comments on pay; most grievances were based on business practices themselves. Second, most criticisms of MT were rooted not in fears of being outperformed by MT systems, but rather in the technology’s limitations and market consequences. This article calls for a broadening of translators’ role across areas of specialization and argues that, in the debate on translation’s future, MT cannot be decoupled from its economic effects.
This report provides an insight into the use of machine translation (MT) in human written transla... more This report provides an insight into the use of machine translation (MT) in human written translation settings.
The report has two parts.
Part I presents findings from interviews conducted with technology specialists, project managers, managing directors and professional translators between March 2016 and October 2017. Thirty interviews lasting nearly twenty hours in total were conducted with participants in eleven countries. The interview findings were presented at an industry-academia knowledge exchange event held at the University of Bristol in January 2018.
Part II presents findings from this event.
Recommendations emerging from the interview findings and from discussions held at the Bristol knowledge exchange event include:
Avoiding the use of measures based on edit distance as the only parameter used to calculate post-editing rates; A need for more research and development initiatives that investigate reliable alternatives to word rates; Preventing negative and non-transparent uses of activity tracking (i.e. tracking of translating time and/or keyboarding); Improving communication and transparency to ensure that all members of translation supply chains are in synch regarding concepts, expectations and product specifications; More training to match editing skills and knowledge of machine translation across professional translation teams; A need to educate society and end-clients of what to expect from machine translation technology.
Post-editing of machine translation (MT) is now increasingly implemented in the human translation... more Post-editing of machine translation (MT) is now increasingly implemented in the human translation workflow after studies in both industry and academia have demonstrated the efficacy of this practice. Post-editing still involves open questions, however, such as how best to train post-editors and how to estimate the effort required by post-editing tasks. In attempting to address some of these questions, many previous studies investigate the post-editing process, but less research has focused on the post-edited product. This chapter examines the link between the process and product of post-editing by checking to see how post-editing effort data related to the quality of post-edited texts, assessed in terms of fluency (linguistic quality) and adequacy (translation accuracy). A statistical analysis indicated that the association between editing operations and the fluency of post-edited texts is dependent on the quality of the raw MT output. Interestingly, a negative association was observed between the number of eye fixations on the text and the quality of the post-edited translations. The chapter shows empirical evidence supporting the distinction between the concepts of translation fluency and adequacy, and postulates that automatic processes play a central role in post-editing performance.
This paper is an exercise of imagination. Based on Kay's (1980) inspiring idea of a translator's ... more This paper is an exercise of imagination. Based on Kay's (1980) inspiring idea of a translator's amanuensis, we attempt to describe a post-editing tool that enables ubiquitous translation (Cronin 2010). We argue that a parallelism exists between media remediation (Bolter and Grusin 1999) and the shifting phase translation is undergoing, with machine translation post-editing having an impact on the global workflow of translated content. We take the hybridisation of traditional and machine translation processes as a starting point to envisage the features of forthcoming translation technologies. Results of previous surveys helped us to select features expected to play a central role: versatile devices to which we broadly refer as displayers would enable ubiquity; a relevant knowledge feature would provide human translators with a well-assorted repertoire of reliable sources; and an effort prediction feature would provide post-editors with reliable estimates of how much work lay ahead. Interacting with the Translator's Amanuensis 2020 would not always be straightforward, however. Translators will have to adapt to richer ways of reading and visualising information. Ultimately, we argue that the Translator's Amanuensis 2020 could benefit from existing Translation Studies concepts: the study of translation problems, translation competence models, and the ethics and sociology of translation.
Post-editing of machine translation is gaining popularity as a solution to the ever-increasing de... more Post-editing of machine translation is gaining popularity as a solution to the ever-increasing demands placed on human translators. There has been a great deal of research in this area aimed at determining the feasibility of post-editing and at predicting post-editing effort based on source-text features and machine translation errors. However, considerably less is known about the mental workings of post-editing and post-editors’ decision-making or, in particular, the relationship between post-editing effort and different mental processes. This paper investigates these issues by analysing data from a think-aloud study through the lens of eye movements and subjective ratings obtained in a separate task. The results show that mental processes associated with grammar and lexis are significantly associated with cognitive effort in post-editing. This association was not observed for other aspects of the task concerning, for example, discourse or the real-world use of the text. In addition, it was noted that lexical issues are linked to long sequences of thought processes. The paper shows that lexis plays a central role in post-editing, and argues that more emphasis should be placed on this issue in future research and in post-editor training.
In the context of non-literary translation, and under appropriate circumstances, it is now largel... more In the context of non-literary translation, and under appropriate circumstances, it is now largely uncontroversial that the use of machine translation can increase translators' productivity without a detrimental effect on product quality. Advances in technology are giving rise to new forms of human-computer interaction, and the use of machine translation in human translation workflows is increasingly commonplace. In this changing context, this issue aims at mapping and evaluating current post-editing practices in the industry, academia, and society in general, acting as a platform for the discussion of various issues in this area. We envisage two connected, but distinct, approaches to the study of post-editing as being particularly suitable to addressing the issue's aim: a strictly controlled approach that looks closely at how post-editors undertake the tasks at hand, and a more sociology-oriented approach that looks at broader aspects of how different parties react to the use of machine translation. Articles adopting either of these approaches will be of interest to the issue, and contributions that attempt to combine them will be especially welcome.
There has been growing interest of late in the cognitive effort required by post-editing of machi... more There has been growing interest of late in the cognitive effort required by post-editing of machine translation. Compared to number of editing operations, cogni-tive (or mental) effort is frequently considered a more decisive indicator of the overall effort expended by post-editors. Estimating cognitive effort is not straightforward, however. Previous studies often triangulate different measures to obtain a consensus, but little post-editing research to date has attempted to show how measures of cogni-tive effort relate to each other in a multivariate analysis. This paper addresses this by presenting an exploratory comparison of cognitive measures based on eye tracking, pauses, editing time, and subjective ratings collected in a post-editing task carried out by professional and non-professional participants. All measures correlated with each other, but a principal components analysis showed that the measures cluster together in different ways. In particular, measures that increase with task time alone behaved differently from the others, with higher mutual associations and higher reliability. Regarding differences between professional and non-professional participants, it was observed that subjective ratings were overall more strongly associated with objective measures in the case of professionals. Surprising findings from previous research based on pause ratio are discussed. The paper argues that a pause typology will benefit the study of pause lengths and cognitive effort in post-editing.
This thesis investigates the expenditure of cognitive effort in post-editing of machine translati... more This thesis investigates the expenditure of cognitive effort in post-editing of machine translation. A mixed-method approach involving the use of eye movements, subjective ratings and think-aloud protocols was adopted for the investigation. The project aims at revealing connections between cognitive effort and variables including linguistic characteristics of the source text and the machine-translation output, post-editors’ individual traits, different linguistic aspects of the activity attended to during the task, and the quality of the post-edited texts, assessed by human translators in terms of fluency (linguistic quality) and adequacy (faithfulness to the source text). Two tasks were conducted to pursue these aims: one involving eye tracking and a self-report scale of cognitive effort, and another carried out by a different, but comparable, sample of participants, under a think-aloud condition. Results indicate that variables such as an automatic machine-translation quality score and source-text type-token ratio are good predictors of cognitive effort in post-editing. The relationship between cognitive effort and post-editors’ traits was found to be a complex one, with significant links in this respect only appearing in the context of interactions between variables. A complex relationship was also found between editing behaviour and the quality of the post-edited text: the number of changes implemented was found to have a generally positive association with post-edited fluency, though cognitive effort was found to be negatively correlated with both the fluency and adequacy of the post-edited texts. Mental processes involving grammar and lexis were significantly related to the levels of cognitive effort expended by participants. These were also the aspects most frequently attended to in the activity. From a methodological perspective, despite the criticisms received by the think-aloud method in previous research, empirical data obtained in this project indicates that think-aloud protocols correlate with eye movements and subjective ratings as measures of cognitive effort.
Identifying indices of effort in post-editing of machine translation can have a number of applica... more Identifying indices of effort in post-editing of machine translation can have a number of applications, including estimating machine translation quality and calculating post-editors’ pay rates. Both source-text and machine-output features as well as subjects’ traits are investigated here in view of their impact on cognitive effort, which is measured with eye tracking and a subjective scale borrowed from the field of Educational Psychology. Data is analysed with mixed-effects models, and results indicate that the semantics-based automatic evaluation metric Meteor is significantly correlated with all measures of cognitive effort considered. Smaller effects are also observed for source-text linguistic features. Further insight is provided into the role of the source text in post-editing, with results suggesting that consulting the source text is only associated with how cognitively demanding the task is perceived in the case of those with a low level of proficiency in the source language. Subjects’ working memory capacity was also taken into account and a relationship with post-editing productivity could be noticed. Scaled-up studies into the construct of working memory capacity and the use of eye tracking in models for quality estimation are suggested as future work.
Uploads
Papers
The report has two parts.
Part I presents findings from interviews conducted with technology specialists, project managers, managing directors and professional translators between March 2016 and October 2017. Thirty interviews lasting nearly twenty hours in total were conducted with participants in eleven countries. The interview findings were presented at an industry-academia knowledge exchange event held at the University of Bristol in January 2018.
Part II presents findings from this event.
Recommendations emerging from the interview findings and from discussions held at the Bristol knowledge exchange event include:
Avoiding the use of measures based on edit distance as the only parameter used to calculate post-editing rates;
A need for more research and development initiatives that investigate reliable alternatives to word rates;
Preventing negative and non-transparent uses of activity tracking (i.e. tracking of translating time and/or keyboarding);
Improving communication and transparency to ensure that all members of translation supply chains are in synch regarding concepts, expectations and product specifications;
More training to match editing skills and knowledge of machine translation across professional translation teams;
A need to educate society and end-clients of what to expect from machine translation technology.
We envisage two connected, but distinct, approaches to the study of post-editing as being particularly suitable to addressing the issue's aim: a strictly controlled approach that looks closely at how post-editors undertake the tasks at hand, and a more sociology-oriented approach that looks at broader aspects of how different parties react to the use of machine translation. Articles adopting either of these approaches will be of interest to the issue, and contributions that attempt to combine them will be especially welcome.
Talks
The report has two parts.
Part I presents findings from interviews conducted with technology specialists, project managers, managing directors and professional translators between March 2016 and October 2017. Thirty interviews lasting nearly twenty hours in total were conducted with participants in eleven countries. The interview findings were presented at an industry-academia knowledge exchange event held at the University of Bristol in January 2018.
Part II presents findings from this event.
Recommendations emerging from the interview findings and from discussions held at the Bristol knowledge exchange event include:
Avoiding the use of measures based on edit distance as the only parameter used to calculate post-editing rates;
A need for more research and development initiatives that investigate reliable alternatives to word rates;
Preventing negative and non-transparent uses of activity tracking (i.e. tracking of translating time and/or keyboarding);
Improving communication and transparency to ensure that all members of translation supply chains are in synch regarding concepts, expectations and product specifications;
More training to match editing skills and knowledge of machine translation across professional translation teams;
A need to educate society and end-clients of what to expect from machine translation technology.
We envisage two connected, but distinct, approaches to the study of post-editing as being particularly suitable to addressing the issue's aim: a strictly controlled approach that looks closely at how post-editors undertake the tasks at hand, and a more sociology-oriented approach that looks at broader aspects of how different parties react to the use of machine translation. Articles adopting either of these approaches will be of interest to the issue, and contributions that attempt to combine them will be especially welcome.