Pupil dilation

This paper reports on a study that analyses the impact of two different machine
translation (MT) outputs on the cognitive effort required to post-edit machine-translated
metaphors by means of eye tracking and think-aloud protocols. We hypothesise that the
statistical MT output would have a positive effect on reducing cognitive effort. In order to
test this hypothesis, a post-editing experiment was conducted with two different groups
of participants. Each experimental group had two post-editing tasks using the language
pair English into Brazilian Portuguese. On Task 1 (T1), participants were asked to postedit a Google machine-translated output whereas on Task 2 (T2) the same participants
were assigned to post-edit a Systran machine translated output. Data collection was
conducted under the experimental paradigm of data triangulation in translation process
research. Data analysis focuses on eye tracking data related to fixation duration and
pupil dilation as well as think-aloud protocols. This analysis shows that the cognitive
effort required to post-edit the pure statistical MT output might be lower in comparison to
the hybrid output when conventional metaphors are machine translated.

Eye tracking has been used successfully as a technique for measuring cognitive load in reading, psycholinguistics, writing, language acquisition etc. for some time now. Its application as a technique for measuring the reading ease of MT output has not yet, to our knowledge, been tested. We report here on a preliminary study testing the use and validity of an eye tracking methodology as a means of semi-automatically evaluating machine translation output. 50 French machine translated sentences, 25 rated as excellent and 25 rated as poor in an earlier human evaluation, were selected. Ten native speakers of French were instructed to read the MT sentences for comprehensibility. Their eye gaze data were recorded non-invasively using a Tobii 1750 eye tracker. The average gaze time and fixation count were found to be higher for the “bad” sentences, while average fixation duration and pupil dilations were not found to be substantially different for output rated as good and output rated as bad. Comparisons between HTER scores and eye gaze data were also found to correlate well with gaze time and fixation count, but not with pupil dilation and fixation duration. We conclude that the eye tracking data, in particular gaze time and fixation count, correlate reasonably well with human evaluation of MT output but fixation duration and pupil dilation may be less reliable indicators of reading difficulty for MT output. We also conclude that eye tracking has promise as a semi-automatic MT evaluation technique, which does not require bi-lingual knowledge, and which can potentially tap into the end users’ experience of machine translation output.

Pupil dilation (PD) dynamics reflect the interactions
of sympathetic and parasympathetic innervations in the iris muscle. Different pupillary responses have been observed with respect to emotionally characterized stimuli. Evidences of the correlation between PD and respiration, heart rate variability (HRV) and blood pressure (BP) are present in literature, making
the pupil dilation a candidate for estimating the activity state of the Autonomic Nervous System (ANS), in particular during stressful and/or emotionally characterized stimuli. The aim of this study is to investigate whether both slow and fast oscillations
of the PD can be addressed to characterized different
affective states. Two different frequency band were considered: the classical autonomic band [0-0.45] Hz and a very high frequency (VHF) band [0.45-5] Hz. The pupil dilation signals from 13 normal subjects were recorded during a psychological protocol suitable to evoke particular affective states. An elaborate
reconstruction of the missing data (blink events and artifacts) was performed to obtain a more reliable signal, particularly in the VHF band. Results show a high correlation between the arousal of the event and the power characteristics of the signal, in all frequencies. In particular, for the “Anger” condition, we can observe 10 parameters out of 14 significantly different with
respect to “Baseline” counterparts. These preliminary results suggest that both slow and fast oscillations of the PD can be used to characterize affective states.