Keywords

1 Introduction

A key concept for road safety is mental fatigue consequent to extensive driving. It is defined as the “state of reduced mental alertness that impairs performance” [1] and it is usually occasioned by prolonged periods of a cognitive, high-demanding, and sustained activity requiring mental efficiency [2]. Driving fatigue is the cause of up to 24% of road crashes in European countries, with a mean of 15% per country [3, 4].

Nowadays, the theoretical models underpinning mental fatigue (for a broad discussion see Ref. [5]), as well as its practical implications (e.g., impairs in cognitive and physical performance, as well as in emotion regulation [6,7,8]) are well known. However, the measurement of mental fatigue in complex and dynamic tasks, such as driving, is still challenging due to the difficulties in establishing an objective, reliable, non-intrusive and real-time tracking method of mental fatigue [9]. Indeed, fatigue, especially when assessed in a clinical context, has frequently been assessed by means of self-report measures (see Ref. [10] for a review). However, because of intrinsic limits of subjective measures, which cannot be employed to track fatigue variations in real-time, the use of neurobehavioral objective indices for fatigue measurement is gaining increasing attention. Although neurobehavioral measures have proved to be useful for fatigue detection and measurement, some limitations need to be considered when it comes to assess their suitability to real field settings, such as the driving context. A summary of limitations present in these measures is reported below.

Neurobehavioral Measures for Driving Fatigue

Among the most-employed neurobehavioral measures for fatigue tracking in driving contexts, eye movements play surely a key role [11,12,13,14,15]. Eye movements consists of frequent, quick movements, called saccades, interspersed with periods of steady gaze, called fixations. For instance, saccade-based metrics, particularly the saccadic peak velocity (i.e., the highest velocity reached within a saccade [11, 12, 15]) emerged as crucial means for tracking mental fatigue during driving [11]. Thus, eye-movement recording is for sure a widely used and reliable technique, able to track real-time fatigue variations. However, in real field settings some problems may emerge due to troubles that most eye-tracking systems have in data recording when people wear eyeglasses [16].

Another central psychophysiological index, frequently employed to assess driving fatigue is the EEG power spectra, which is the relative prevalence of the sinusoidal waves composing the EEG signal. In fatigue conditions, it is usually found a decrease of frontal beta and gamma power and an increase of frontal theta, alpha and delta [17,18,19,20]. For instance, during a 2-h driving along a simulated scenario, an inverted U-shaped quadratic trend in frontal delta power has been reported [18, 19]. Event-related potentials (i.e., changes in the brain activity occurring in response to a stimulus; ERPs) are sometimes employed to track driving fatigue [21]. Among them, ERPs linked to early and mid-stages of stimulus processing (such as the P3 component) have been investigated for their relation to automatic attentional processes that are crucial to driving. All the mentioned EEG indices have proved to accurately monitor mental fatigue. However, EEG measurement, even when conducted by means of wearable systems, is quite uncomfortable for the participant.

Concerning peripheral indices, cardiac activity (especially heart rate) is frequently employed to measure fatigue [9], also in driving contexts [9, 22, 23]. A recent review [9] identified at least fourteen studies reporting a linear increase in heart rate with the increase in the time spent driving, both in real and simulated contexts. Thus, heart rate has proved not only to be an easily measurable index, but also directly correlated with fatigue [9, 22, 23]. Another reliable and easy measurable index, electrodermal activity, has proved to be related to fatigue while driving [24,25,26]. Both skin conductance level and frequency of skin conductance responses show a general increasing trend with increasing stress and fatigue [27]. These indices have been recently included as predictors in algorithms aimed at classifying different drivers’ state, along with other fatigue indices such as heart rate and respiration rate [28]. Some recent evidence pointed out a relation between superficial electromyography (EMG) [29] (i.e., the measurement of the muscular electrical activity) and fatigue, with an increase in the signal frequency with increasing fatigue. For instance, EMG signal recorded from Brachioradialis muscle located at the forearm during a 2-h simulated driving showed good degree of sensitivity to increasing levels of fatigue [29].

The measurement of heart rate, electromyography and electrodermal activity, although more comfortable than EEG, is generally motion sensitive (e.g., Ref. [30]): complex tasks, such as driving a vehicle, often require the person to move, thus movement artifacts could affect the reliability of the data. Finally, respiratory activity, in terms of respiration rate, although quite responsive to changes in driving fatigue [31, 32], is an index that should be always coupled with other measures (e.g., EEG), since breath variations can occur for a variety of reasons [see Ref. 33].

Another measure for assessing driving fatigue is actigraphy [9, 34], which involves the measurement of movements frequency over time, usually employing a wearable device, typically a wristband. The amount of movements within a given time is used to derive the typical metrics of this method, that, in comparison to a threshold, can discriminate between “sleep” and “awake” states. Thus, actigraphy is a well-known and widely employed method to produce reliable estimates of sleep/wake timing and duration [35] that are crucial inputs for fatigue-prediction models. However, the devices usually employed with this aim are scarcely suitable for monitoring alertness directly [35]; as a matter of fact, wrist movement activity has only been proved to consistently vary between wake and sleep, being unable to systematically track different levels of alertness during awake states [35].

Facial Infrared Thermography

Considering the above-mentioned shortcomings, an emerging and challenging research trend (e.g., HADRIAN project [36]) focuses on the identification of a non-invasive, yet reliable method, suitable to track real-time driver’s state variations in real field settings. The ideal future scenario [36], as already pointed out by various researchers [32, 36], should involve several neurobehavioral measures, chosen among the most comfortable and precise, so as to obtain the most accurate and comprehensive estimate of the driver’s state. An example of successful integration of several fatigue-sensitive measures is a recent technology developed by Panasonic (Japan) [37] that uses contactless sensing devices of neurobehavioral indices (body temperature, blink features, and facial expressions) to track the driver’s states and in-vehicle environment (e.g., temperature, light conditions, air velocity) in real-time. This technology integrates all this information to assess drivers’ drowsiness level, to adjust the inside-vehicle conditions and, in case of high levels of fatigue, to suggest resting. This application is just an example of how body temperature can be useful to detect driving fatigue in combination with other indices. Indeed, facial infrared thermography, might represent a good candidate to reach this aim [38]. In a recent study from our laboratory [20], we used infrared thermography to remotely measure nasal-skin temperature during a 2-h driving task. As the driving session progressed, arousal levels decreased. It was confirmed by the increased frontal delta EEG activity (inverted U-shaped quadratic trend), as well as subjective ratings of alertness and fatigue. The behavior of the nasal-skin temperature was coherent: it increased over the first 45 min of session, tending to gradually decrease in the last part of the task (for more details see Ref. [20]). Nevertheless, the presence of make-up and sweat, which cannot be controlled outside the experimental environment, can alter the reliability of this technique. A body region that is commonly uninfluenced by make-up or sweat is the dorsal part of the hands. Thus, hand-skin temperature measurement might represent a valid alternative to overcome these limitations. Studies monitoring hand-skin temperature to study driver fatigue, however, are just anecdotic (e.g., Ref. [39]).

Here, we investigated the effects of a monotonous 2-h simulated driving session, a common method to induce fatigue at the wheel (e.g., Ref. [11]), while we continuously monitored the drivers’ hand-skin temperature. As the experimental session progressed, we expected the driver to reduce his/her grip force on the steering wheel [40]; and consequently we expected an increase in the driver’s hand-skin temperature, due to changes in hand peripheral circulation and to the gradual mechanical decompression of the blood vessels (see, among others, Ref. [41]).

2 Materials and Methods

2.1 Participants

Eleven active drivers (mean age ± standard deviation = 25.36 ± 1.70 years; 6 women) were voluntary enrolled to take part to the present study (University of Granada’s Institutional Review Board approval #484/CEIH/2018). All of them were non-smokers, held a valid driving license, had normal or corrected-to-normal vision, and were naïve to the hypotheses of the experiment.

2.2 Experimental Design

The experiment consisted of a 2-h simulated driving session along a monotonous course [11, 18] in a virtual environment. We chose this temporal window to be close to the maximum driving time that professional drivers are allowed before a mandatory break [42]. The study followed a within-subjects design with the driving time as the independent variable. As dependent variables, we considered right-hand-skin temperature over the 2-h driving session.

2.3 Thermographic Recordings and Analyses

We used the ThermoVision A320G Researcher Infrared Camera (FLIR Systems, USA). The camera (resolution of 320 × 240 pixels) was placed on a tripod 110 cm above the floor and ~140 cm from the driver. The camera has automatic focus that was always employed to focus the image recording. We stored the recorded signal using the program Researcher TermaCAMP 2.9 (FLIR Systems, USA).

The main relevant point (pixel) of interest (POI) was the skin surface of the dorsal proximal phalangeal joint of the third finger of the right hand (approximately, 2 cm below the knuckle). To control for possible room temperature changes, we selected another POI on the wall behind the driver seat. The coordinates for this POI were kept constant within the recording session. Two independent researchers manually performed the collection of the temperature for each POI and participant. The ICC estimate for the driver’s right-hand-skin temperature was 0.98 [95% C.I. 0.97–0.98], which indicates excellent reliability. For the temperature analysis (see Sect. 3.1 Right-hand-skin temperature), we used the mean value of two researchers.

2.4 Procedure

The experimental protocol was designed following the recommendations of the consensus statement on thermographic imaging studies by Moreira and colleagues [43]. The experiment took place in a simulation laboratory (two adjacent windowless rooms of about 8.5 m2 each: the test/preparation unit and the simulator unit) free from thermal noise sources, located at the Mind, Brain, and Behavior Research Center (Granada, Spain) (see Fig. 1). Once the participant signed the informed consent form, we recorded sociodemographic and health data in the test/preparation unit. All participants wore the same upper body clothing (a clean cotton t-shirt). Approximately 15 min after his/her arrival (i.e., adaption period to room temperature), the participant positioned him/herself on the car seat (in the simulator unit), he/she filled several questionnaires (see Ref. [20] for more details) and the thermographic camera was turned on to allow for its sensor to be stabilized. Participants drove a middle-sized car for two hours without breaks, around the same road without any other traffic present.

Fig. 1.
figure 1

Experimental setting. Two adjacent rooms (A and B), separated by a small corridor, constitutes the driving simulation laboratory

Full size image

We developed a virtual two-lane, rounded rectangle monotonous grassy meadow road scenario using the OpenDS 2.5 software (OpenDS, Saarbrücken, Germany). A speed limit of 60 km/h was set. The driving simulator recorded the car speed at 20 Hz. The simulator is described in Ref. [20]. After a five-minute driving familiarization session, we calibrated the thermographic camera, and the driving simulation started. We instructed participants to hold the steering wheel at two marked points (10 o’clock and 2 o’clock positions). Furthermore, in order to avoid diurnal fluctuations that affect arousal levels [44], the experimental protocol always started at 8:30 am and the driving session, at 9:00 am.

2.5 Statistical Analyses

We categorized the full driving period into sixty 2-min bins, discarding the data from the first and the last bin to remove possible transient effects caused by the starting and ending of the driving session. To analyze changes over time (2-h driving session, 58 time points) in the right-hand-skin temperature, we tested different individual growth curve models (linear, quadratic, and cubic) using a maximum likelihood estimation to study intraindividual differences in the patterns of change of the dependent variables [45] (see Ref. [20] for more details). To select the best model, we used the Akaike’s Information Criterion (AIC, a lower AIC value indicates a better fit). The results of a given growth model were shown, only if its AIC was lower than AIC for the precedent model. For each model, we present parameter estimates and standard errors (SE). Significance levels were set at α < 0.05. This is a complementary report to reference [20], thus data concerning the effects of driving time on nasal skin temperature, EEG power activity, and speeding behavior have been already reported (see Ref. [20]).

3 Results

3.1 Right-Hand-Skin Temperature

To study trajectory changes over time in the driver’s right-hand-skin temperature, we first tested the linear growth curve model (Model 2). The mean right-hand-skin temperature was 30.57 ℃ and decreased with time. The decrease was not significant (p = 0.142), although there was a decline in the residual variance of 1.18 from Model 1 to Model 2 (13.10 to 11.92). To test the quadratic rate of change (Model 3), we added a quadratic parameter in the previous model. The linear effect for the driver’s right-hand-skin temperature was positive, revealing that it increased linearly over time, but not significantly (β = 0.06, SE = 0.04, p = 0.161). The significant quadratic effect was negative (β = −0.001, SE = 0.0006, p = 0.012), showing that the increasing effect gradually diminished after 20 min (i.e., inverted U-shaped curve). Compared to the linear change trajectory (0.06), the rate of quadratic growth (−0.001) was small. We also tested any cubic changes in individual trajectories over time (Model 4), but they were non-significant. Thus, the hand-skin temperature showed a quadratic trajectory, y · ij = 29.59 + (0.059 × time) + (−0.0015 × time · 2) (see Fig. 2). The patterns of change of the right-hand-skin temperature did not vary when we considered the room temperature. The ratio between the right-hand-skin and the room temperatures presented a similar significant cubic trajectory: yij = 1.42 + (0.014 × time) + (−0.0004 × time2) + (0.000003 × time3). From this, we can affirm that small variations in the environmental temperature (less than 1 ℃) did not affect the sensitivity of the hand-skin temperature for detecting arousal changes.

Fig. 2.
figure 2

Effects of driving time (time on task) on the drivers’ hand-skin temperature. The blue curve represents the quadratic fit trajectory of the drivers’ hand-skin temperature over the 2-h driving session. Insert: A driver during the experimental session with her two hands on the steering wheel. The point of interest is marked with a grey arrow.

Full size image

Table 1 presents the results of the fitting the unconditional means model and the growth curve models.

Table 1. Results of fitting growth curve models for the trajectory of the driver’s right-hand-skin temperature over a 2-h simulated driving session (n = 11). It presents parameter estimates and, in brackets, standard errors.
Full size table

4 Discussion

The present study aimed to assess the use of the hand-skin temperature to monitor overall arousal variations due to the effect of fatigue, when engaging in a complex and dynamic everyday task as driving. We tracked participants’ right-hand-skin temperature using an infrared thermography camera during a 2-h uninterrupted driving course along a monotonous virtual scenario. As the experimental session progressed, we expected to find an increase in the drivers’ temperature. Indeed, during the first ≈20 min we found an increase of the hand-skin temperature (less than 1 ℃); afterward, it began to decrease, ending up to 27.8 ℃, following a quadratic trajectory. The initial increase, consistently with our hypothesis, probably reflects participants’ gradual lowering in griping force on the steering wheel [40, 41], as a result of the increasing fatigue. The explanation of the decrease of the temperature in the last part of the task is less clear. One option may rely on executive control difficulties [46], that is difficulties in the ability to regulate perceptual and motor processes for goal-directed behavior. Fatigued drivers might have difficulties in regulating their perceptual and motor processes while driving [46, 47]; as a result, they would need to put more effort in regulating the vehicle trajectory. This, in turn, might lead them to increase their grip force on the steering wheel. This explanation is coherent with the results reported by independent investigations in non-driving scenarios, which showed a decrease in hand-skin temperature when the experimental subjects exerted constant pression (see for instance Ref. [41]). Future studies should control for the griping force on the steering wheel while monitoring hand-skin temperature variations.

Another explanation may rely on the role of stress [48]. Tasks requiring sustained attention (e.g., vigilance tasks or even prolonged driving tasks) can be simultaneously (and paradoxically) de-arousing and stressful, due to the multidimensionality of stress responses [48]. Actually, the driving task employed in the present study, although monotonous, requested a constant level of attention that may have caused an increase in participants’ stress. A well-known effect of stress is the decrease in the peripheral temperature [49]: stress seems to induce peripheral vasoconstriction, causing a rapid, short-term drop in skin temperature [49]. Thus, the sustained attention required by the monotonous driving task here employed may have cause stress responses, leading to vasoconstriction and to the decrease in the hand-skin temperature in the second part of the session. Further studies are needed to disentangle the effects caused by stress from those caused by fatigue.

Nevertheless, the present results are in favor of the validity of thermography to track hand-skin temperature variations as an index of drivers’ fatigue, without interfering with task performance or compromising drivers’ comfort and safety. A limitation of this method relies on the need to cope with the expectable variations in environmental temperature in a real vehicle. This might be overcome by using a ratio between skin and environmental temperatures (see Ref. [20]). Thermal variations could also be used to reduce driver distractions. For instance, Jaguar-Land Rover [50, 51] recently proposed a sensory steering wheel that uses haptic (mainly thermic) feedback technologies.

By slightly increasing or decreasing the temperature in various zones of the steering wheel, the technology would signal to the driver that he/she should change roadway or keep the eyes on the road because of a risky situation, such as a difficult crossroad.

In conclusion, the present results may represent a first step toward the development of a complex system with a huge impact on driver’s safety, potentially able to detect (and even notify) safety-critical increases in fatigue, such as situations where the drivers should take a break, monitoring their drowsiness in a non-invasive, yet accurate and safe way.