GB2635163A

GB2635163A - Monitoring driver behaviour

Info

Publication number: GB2635163A
Application number: GB2316666.3A
Authority: GB
Inventors: Hoermann Stefan; Kramadhari Prashant; Widmann Ludwig
Original assignee: Jaguar Land Rover Ltd
Current assignee: Jaguar Land Rover Ltd
Priority date: 2023-10-31
Filing date: 2023-10-31
Publication date: 2025-05-07
Also published as: GB202316666D0

Abstract

A control system for monitoring a divergence from a baseline driving behaviour of an operator of a vehicle. The control system is configured to: calculate a trajectory score based on a comparison of a monitored trajectory with a driver behaviour model. Monitor using a sensor 304 the environment of the vehicle and calculate an environment-based driving score 310 based on the based on the monitored trajectory and environment. Calculate a driving score 316 based on the trajectory score and the environment score and compare the score with a threshold value to identify a divergence from a baseline driving behaviour to generate a signal indicating the divergence. The driver model may included an auto encoder model and the control system may produce a birds-eye-view representation of the environment. Interior settings of the vehicle may include air conditioning, interior illumination, infotainment or drivers seat settings.

Description

MONITORING DRIVER BEHAVIOUR

TECHNICAL FIELD

The present disclosure relates to monitoring driving behaviour of a driver of a vehicle. Aspects of the invention relate to a control system, a method of monitoring driving behaviour for a divergence from a baseline driving behaviour, a computer program product and a vehicle.

BACKGROUND

It is recognized that operating a vehicle for a prolonged period may lead to tiredness or fatigue of a driver, reducing the ability of the driver to concentrate, and increasing the risk of driver error. Modern vehicles may include a number of safety features to help mitigate such driver errors, for example active emergency braking and lane guidance systems. These safety features are intended to intervene when the vehicle is at risk of an accident. However, at the point when these systems intervene, the vehicle may already be in a dangerous situation and the intervention may not be sufficient to completely avoid occurrence of an accident. Moreover, drivers may become accustomed to the presence of such features, relying on them to safeguard the vehicle when the driver is distracted. This may lead to the driver being more careless when driving, thereby negating the increase in vehicle safety provided by the safety features.

It is an aim of the present invention to address one or more of the disadvantages associated with the prior art.

SUMMARY OF THE INVENTION

Aspects and embodiments of the invention provide a control system, a method of adjusting an interior environment setting of a vehicle to promote driver alertness, a computer program product and a vehicle as claimed in the appended claims.

According to an aspect of the present invention there is provided a control system for monitoring a divergence from a baseline driving behaviour of an operator of a vehicle, the control system comprising one or more processors collectively configured to monitor a trajectory of the vehicle; calculate a trajectory score based on a comparison of the monitored trajectory with a driver behaviour model; monitor, using at least one sensor of the vehicle, an environment around the vehicle; calculate an environment-based driving score based on the monitored trajectory and the monitored environment; calculate a driving score value based on the trajectory score and the environment-based driving score; compare the driving score value with a threshold value to identify a divergence from the baseline driving behaviour; and generate a signal indicating the divergence from the baseline driving behaviour.

Advantageously, determining a driving score based on a combination of an assessment of the monitored trajectory compared to normal driver behaviour along with an assessment of the trajectory based on environmental factors may allow a more accurate detection of a deviation from a normal driving style, for example, by accounting for unusual driving characteristics caused by external factors such as the actions of another vehicle.

The control system comprises one or more controllers collectively comprising at least one electronic processor having an electrical input for receiving an input signal; and at least one memory device electrically coupled to the at least one electronic processor and having instructions stored therein; and wherein the at least one electronic processor is configured to access the at least one memory device and execute the instructions thereon so as to: monitor a trajectory of the vehicle; calculate a trajectory score based on a comparison of the monitored trajectory with a driver behaviour model; monitor, using at least one sensor of the vehicle, an environment around the vehicle; calculate an environment-based driving score based on the monitored trajectory and the monitored environment; calculate a driving score value based on the trajectory score and the environment-based driving score compare the driving score value with a threshold value to identify a divergence from the baseline driving behaviour; and generate a signal indicating the divergence from the baseline driving behaviour.

In an embodiment, the one or more processors may be collectively configured to calculate the trajectory score by: encoding the monitored trajectory based on the driver behaviour model, the driver behaviour model comprising data representative of the baseline driving behaviour; determining a reconstructed trajectory based on the encoded trajectory and the driver behaviour model; and calculating the trajectory score based on a distance between the monitored trajectory and the reconstructed trajectory.

Advantageously, calculating the trajectory score in this way provides a simple and accurate method of identifying any deviation of monitored driving behaviour from a driver behaviour model of 'normal' driving behaviour.

In an embodiment, the driver behaviour model comprises an autoencoder model pre-trained on monitored trajectories of a plurality of drivers.

Advantageously, a pre-trained autoencoder model can be used either to allow assessment of self-driving algorithms against a model of normal driver behaviour, or as an initial model that may be finetuned to actual behaviour of a driver monitored during a training period, thereby significantly reducing the amount of training required to provide a model adapted to a particular driver's behaviour.

In an embodiment, the one or more processors may be collectively configured to calculate the environment-based driving score by: generating a cost map for a plurality of possible trajectories based on an input of the at least one sensor; and comparing the monitored trajectory with the cost map.

Advantageously, the use of a cost map of possible trajectories may allow embodiments to use outputs of existing perception models already in place on a vehicle to determine an environment-based driving score, for example by using the outputs as inputs to an Al model trained to generate the cost map.

In an embodiment, the one or more processors may be collectively configured to compare the monitored trajectory with the cost map by, for each point compare the monitored trajectory with the cost map by, for each point (xt, yt) of the monitored trajectory, summing a cost value of the cost map associated with the point of the monitored trajectory.

Advantageously, summing the point of the trajectory across the cost map provides a flexible way to generate a cost value from an arbitrary trajectory that may be different from any of the model trajectories used to train the Al model to generate the cost map.

In an embodiment, the one or more processors may be collectively configured to generate the cost map by: transforming the input of the at least one sensor into a bird's-eye-view (BEV) space to generate a BEV representation of the environment around the vehicle; and processing the BEV representation using a neural network to generate the cost map.

Advantageously, sensors and models to generate the BEV representation may already be provided on the vehicle for use by ADAS features. Reusing these features to generate a BEV representation to be input to the model is therefore efficient and provides a useful input data set including sufficient information of the environment to allow the cost map to be generated.

In an embodiment, the neural network is trained in conjunction with a perception model to predict a cost map, wherein the perception model is configured to receive the at least one sensor input and generate the BEV representation, the training data comprising training example input/output pairs of at least one sensor input and an associated trajectory.

Advantageously, the neural network can be trained as a separate 'head' comprising one or more layers of a neural network to receive an output of a BEV encoder, i.e. perception model. This facilitates reuse of the perception model and may simplify the training data examples.

In an embodiment, at least a portion of the set of training data comprises example input/output pairs captured during a training period while the vehicle is operated by the driver.

Advantageously use of training data examples captured while the driver is operating the vehicle allows the finetuning of the neural network to generate the cost map to a particular drivers normal behaviour in a particular environment.

In an embodiment, the one or more processors may be collectively configured to train the neural network by: recording, during a training period, a plurality of training examples, the training examples comprising example input/output pairs of the at least one sensor input and an associated monitored trajectory; inputting the at least one sensor input from one or more of the example input/output pairs to obtain a predicted cost map result; characterizing an error between the predicted cost map result and the monitored trajectory of the input/output pair to minimize a cost associated with the monitored trajectory; and using an optimization algorithm to update weights of the neural network based on the characterized error.

Advantageously, the control system is able to train the neural network by updating weights of the network based on training data, such as training data captured during operation of the vehicle by a particular driver. Thereby allowing finetuning adaptation to a particular driver's driving style to be performed on the vehicle. This arrangement may mitigate privacy issues associated with transmission of monitored driving data off the platform for processing.

In an embodiment, the baseline driving behaviour comprises a learned baseline driving behaviour associated with a driver of the vehicle, and the autoencoder model is finetuned based on monitored trajectories of the vehicle for a training period while the vehicle is operated by the driver.

Advantageously, the driving behaviour model to generate the trajectory score can be finetuned according to monitored trajectories during a training period, allowing the driving behaviour model to be adapted to a particular driver.

In an embodiment, the one or more processors may be collectively configured to finetune the autoencoder model based on a trajectory training set comprising a plurality of monitored trajectories recorded during the training period by: for each monitored trajectory in the trajectory training set, inputting the trajectory to the autoencoder model to obtain a reconstructed trajectory; characterizing an error between the monitored trajectory and the reconstructed trajectory; and updating weights of the autoencoder model using an optimization algorithm based on the characterized error.

Advantageously, the control system is able to train the driver behaviour model based on training data, such as training data captured during operation of the vehicle by a particular driver, allowing finetuning adaptation to a particular driver's driving style to be performed on the vehicle. May avoid privacy issues caused by transmitting monitored driving data off platform for processing.

In an embodiment, the one or more processors may be collectively configured to calculate the driving score value based on a weighted average over a predefined period of time of at least one of the trajectory score and the environment-based driving score.

Advantageously, using a weighted average to smooth a driving score may avoid artificially raising the driver score in response to a single occurrence, providing a better measurement of any deterioration in a driver score as, for example, a driver becomes tired.

In an embodiment, the calculated driving score value is indicative of an alertness level of the driver, and wherein the divergence from the baseline driving behaviour is indicative of a reduction in alertness of the driver.

Advantageously, the driving score can be used to determine when a driver has become tired or distracted, for example based on a comparison with a threshold level.

In an embodiment, the one or more processors may be collectively configured to in response to the signal indicating the divergence from the baseline driving behaviour, modify a setting associated with an ambient condition of an interior environment of the vehicle.

Advantageously, one or more actions can be initiated based on the driving score to attempt to increase alertness of the driver, for example providing increased cool airflow to the driver.

In an embodiment, the setting associated with the ambient condition of the interior environment of the vehicle comprises one of an air conditioning setting; an interior illumination setting; an infotainment volume setting; and a heated driver's seat setting.

Advantageously, a number of different cabin environment settings can be adjusted to help increase the alertness of the driver.

In an embodiment, the one or more processors may be collectively configured to operate the vehicle using an advance driving assistance system of the vehicle; and generate the driving score value, the driving score value representative of a divergence of a trajectory controlled by the advance driving assistance system and the baseline driving behaviour.

Advantageously, the control system may be used to score self-driving features against models of normal driving behaviour, providing an objective measure of performance for self-driving algorithms/systems. In some embodiments, ADAS features may operate in conjunction with the driver operating the vehicle, for example with features that assist the driver while driving, and a driver score may be calculated for both the behaviour of the driver and the operation of the vehicle by the ADAS feature.

According to another aspect of the invention, there is provided a method of determining a driving score representative of a divergence from a baseline driving behaviour, the method comprising: monitoring a trajectory of a vehicle; calculating a trajectory score based on a comparison of the monitored trajectory with a driver behaviour model; monitoring, using at least one sensor, an environment around the vehicle; calculating an environment-based driving score based on the monitored trajectory and the monitored environment; calculating a driving score value based on the trajectory score and the environment-based driving score; comparing the driving score value with a threshold value to identify a divergence from the baseline driving behaviour; and generating a signal indicative of the divergence from the baseline driving behaviour.

Advantageously, determining a driving score based on a combination of an assessment of the monitored trajectory compared to normal driver behaviour and also an assessment of the trajectory based on environmental factors allows a more accurate detection of a deviation from the normal driving style, for example, by accounting for unusual driving characteristics caused by external factors such as the actions of another vehicle.

In an embodiment, the method may comprise calculating the trajectory score by: encoding the monitored trajectory based on the driver behaviour model, the driver behaviour model comprising data representative of the baseline driving behaviour; determining a reconstructed trajectory based on the encoded trajectory and the driver behaviour model; and calculating the trajectory score based on a distance between the monitored trajectory and the reconstructed trajectory.

In an embodiment, the method may comprise calculating the environment-based driving score by: generating a cost map for a plurality of possible trajectories based on an input of the at least one sensor; and comparing the monitored trajectory with the cost map.

In an embodiment, the method may comprise comparing the monitored trajectory with the cost map by, for each point compare the monitored trajectory with the cost map by, for each point (xt,yt) of the monitored trajectory, summing a cost value of the cost map associated with the point of the monitored trajectory.

In an embodiment, the method may comprise generating the cost map by: transforming the input of the at least one sensor into a bird's-eye-view (BEV) space to generate a BEV representation of the environment around the vehicle; and processing the BEV representation using a neural network to generate the cost map.

Advantageously, method may make use of sensors and models already be provided on the vehicle for use by ADAS features to generate the BEV representation. Reusing these features to generate a BEV representation to be input to the model is therefore efficient and provides a useful input data set including sufficient information of the environment to allow the cost map to be generated.

In an embodiment, the method may comprise training the neural network by: recording, during a training period, a plurality of training examples, the training examples comprising example input/output pairs of the at least one sensor input and an associated monitored trajectory; inputting the at least one sensor input from one or more of the example input/output pairs to obtain a predicted cost map result; characterizing an error between the predicted cost map result and the monitored trajectory of the input/output pair to minimize a cost associated with the monitored trajectory; and using an optimization algorithm to update weights of the neural network based on the characterized error.

Advantageously, the method may include training the neural network by updating weights of the network based on training data, such as training data captured during operation of the vehicle by a particular driver. Thereby allowing finetuning adaptation to a particular driver's driving style to be performed on the vehicle. This arrangement may mitigate privacy issues associated with transmission of monitored driving data off the platform for processing.

In an embodiment, the method may comprise finetuning the autoencoder model based on a trajectory training set comprising a plurality of monitored trajectories recorded during the training period by: for each monitored trajectory in the trajectory training set, inputting the trajectory to the autoencoder model to obtain a reconstructed trajectory; characterizing an error between the monitored trajectory and the reconstructed trajectory; and updating weights of the autoencoder model using an optimization algorithm based on the characterized error.

Advantageously, the method may include training the driver behaviour model based on training data, such as training data captured during operation of the vehicle by a particular driver, allowing finetuning adaptation to a particular driver's driving style to be performed on the vehicle. May avoid privacy issues caused by transmitting monitored driving data off platform for processing.

In an embodiment, the method may include calculating the driving score value based on a weighted average over a predefined period of time of at least one of the trajectory score and the environment-based driving score.

In an embodiment, the method may include, in response to the signal indicating the divergence from the baseline driving behaviour, modifying a setting associated with an ambient condition of an interior environment of the vehicle.

In an embodiment, the method may include operating the vehicle using an advance driving assistance system of the vehicle; and generating the driving score value, the driving score value representative of a divergence of a trajectory controlled by the advance driving assistance system and the baseline driving behaviour.

Advantageously, the control system may be used to score self-driving features against models of normal driving behaviour, providing an objective measure of performance for self-driving algorithms/systems.

According to a further aspect of the invention, there is provided a computer program product comprising computer program code that when executed on a processor of a control system of a vehicle including at least one environment sensor causes the control system to perform a method as described above.

According to another aspect of the invention, there is provided a vehicle including the control system as described above.

Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and/or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and/or features of any embodiment can be combined in any way and/or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and/or incorporate any feature of any other claim although not originally claimed in that manner.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Figure 1 shows a control system for a vehicle in accordance with an embodiment of the present invention Figure 2 shows a vehicle including the control system in accordance with embodiments of the invention; Figure 3 shows a method of calculating a driving score in accordance with embodiments of the invention; Figure 4 shows a method of monitoring driving behaviour according to embodiments of the invention; Figure 5 shows an autoencoder model; Figure 6 shows a neural network architecture that can be used to generate a trajectory cost map in accordance with embodiments of the invention; Figure 7 shows a method of adjusting an interior environment setting of a vehicle in accordance with embodiments of the invention; and Figure 8 shows a device comprising a computer-readable storage medium in accordance with embodiments.

DETAILED DESCRIPTION

According to embodiments of the invention, operation of a vehicle can be monitored and a driver score calculated that is indicative of a similarity between the way the vehicle is currently being operated and a normal, or baseline, driving behaviour. The driving score may then be compared with a threshold value to determine when the monitored driving behaviour diverges from the baseline driving behaviour by a significant amount. For example, divergence of the driving behaviour from the baseline driving behaviour may indicate that the driver is becoming fatigued and driver alertness has been reduced. Embodiments may be able to identify a divergence of the driving behaviour from the baseline driving behaviour before the intervention of any active safety measures, allowing action to be initiated to increase the alertness of the driver before the vehicle is placed in a potentially dangerous situation due to tiredness of the driver, that would trigger intervention by any active safety systems. According to some embodiments, the baseline driving behaviour may be learned from the actions of a particular driver, allowing the current actions of the driver to be compared against the normal behaviour of that driver.

When operating a vehicle, a driver may be exposed to a range of external factors, such as weather conditions, the behaviour of other road users, the condition of the road, etc that may affect the driving style or behaviour of the driver. Furthermore, over time, the alertness of the driver varies according to their ability to concentrate, which is affected by physiological parameters. While fresh and concentrated, the driver will be able to perform the driving task (e.g. operating the accelerator and brake pedals, applying steering wheel torque) according to their typical driving behaviour. However, as the alertness of the driver reduces, the driver behaviour when operating the vehicle controls may change, diverging from the driver's normal driving behaviour, and such changes may provide an indication of the alertness of the driver.

However, operation of the vehicle controls by the driver may also be highly dependent on environmental conditions (e.g. the type of road -city, countryside, highway, the presence of special landmarks like tunnels, the curvature profile of the road ahead) as well the presence and behaviour of other road users (e.g. a sudden cut-in of another vehicle, a slow vehicle driving ahead at low speed for a long time, a large lorry in the lane nearby). Simply monitoring the operation of the vehicle by the user without considering the environmental conditions in which the vehicle is being operated may, therefore, not provide an accurate indication of the alertness of the driver.

According to embodiments of the invention, operation of a vehicle can be monitored and assigned a trajectory score compared to a model of driving behaviour independent of any environmental conditions. At the same time, an environment-based driving score can be determined based on the trajectory of the vehicle compared to a model of operation of the vehicle taking into account features of the environment in which the vehicle is being operated. By combining the trajectory score and the environment-based score, a driving score value can be calculated that can be compared against a threshold value to determine whether the driving behaviour has deviated from the baseline driving behaviour, while taking into account environmental factors that may have caused unusual yet appropriate operation of the vehicle.

Driving scores may take account of vehicle operation (i.e. monitored trajectories) over a period of time prior to the driving score value being calculated, for example the driving score may comprise a weighted average over a preceding predefined period of time, such as the preceding thirty minutes. Thus, individual minor driving errors may not significantly affect the calculated driving score. but accumulated minor errors within the predefined period of time will cause the driving score to change significantly indicating that the driver's behaviour has diverged significantly from their usual driving style. Major driving errors that may be safety-relevant may initiate an intervention by one or more safety systems, such as active emergency braking, and may also be reflected in the driving score value.

According to some embodiments, when it is determined that the driving behaviour has deviated from the base line driving behaviour, for example indicating that the driver is tired or distracted, action may be taken to raise alertness of the driver and to reduce the chance of further poor driving. Example actions include modifying a setting associated with an ambient interior environment of the vehicle to promote alertness, such as adjusting interior light settings (e.g. light level, intensity, temperature or colour), changing a temperature or airflow setting of the climate control system or car seat, and/or changing a setting related to an infotainment system (switch on/off, change volume, select playlist, change pitch/tone, etc.). Through subtle actions, such as reducing a climate control temperature setting by one or two degrees, alertness of the driver may be increased without active interaction by the driver and before a major driving error is made that might cause an accident, or that results in intervention by an active safety feature.

According to some embodiments, the driving scores may be calculated based on a personalized driving behaviour model based on the normal driving behaviour of a particular driver. Thus, the driving score may provide an indication of how well a driver is driving relative to that driver's learned driving style, that is how similar the driver's current actions are compared to normal activity when operating the vehicle. The use of a learned driving behaviour model may allow embodiments to more accurately determine when a particular driver has changed their behaviour, rather than comparing driver's actions against a generic driver model encompassing all types of drivers.

With reference to Figure 1, there is illustrated a control system 100 for a vehicle in accordance with an embodiment of the present invention. The control system 100 comprises at least one sensor 102 for monitoring an environment around the vehicle; a trajectory monitoring module 104 for monitoring a trajectory of the vehicle as it is operated; and one or more controller 108. The at least one sensor 102 and the trajectory monitoring module 104 are coupled to the one or more controller 108 and provide sensor data and vehicle trajectory data, respectively, to the one or more controller 108.

The control system 100 as illustrated in Figure 1 comprises one controller 108, although it will be appreciated that this is merely illustrative. The controller 108 comprises processing means 110 and memory means 112. The processing means 110 may be one or more electronic processing device 110 which operably executes computer-readable instructions. The memory means 112 may be one or more memory device 112. The memory means 112 is electrically coupled to the processing means 110. The memory means 112 is configured to store instructions, and the processing means 110 is configured to access the memory means 112 and execute the instructions stored thereon.

Figure 2 illustrates a vehicle 200 including the control system 100 in accordance with embodiments of the invention.

Figure 3 illustrates a method 300 of calculating a driving score in accordance with embodiments of the invention that can be implemented using the control system 100 illustrated in Figure 1. According to the illustrated method 300, sensor data 302 associated with an environment around the vehicle 200 can be received from the at least one sensor 102. For example, sensor data 302 may comprise images, radar measurements, or lidar measurements of the area surrounding the vehicle 200. The sensor data 302 is provided to a perception system 304. The perception system 304 processes the sensor data 302 and generates a representation of the area surrounding the vehicle, for example by transforming sensor measurements provided in sensor data 302 to into a Bird's-Eye-View (BEV) space centred on the vehicle 200. The perception system 304 further generates a cost map 306 that assigns a cost to coordinates in the representation of the area surrounding the vehicle 200.

Figure 6 illustrates a neural network architecture 600 that can be used to generate the cost map 306 in accordance with embodiments of the invention. According to the illustrated architecture 600, a pretrained perception stack can be used that is capable of receiving sensor input 602 of the space around the vehicle 200 (e.g. camera, radar or lidar images) and transforming the sensor input 602 into a bird's-eye-view space that provides a top-view feature representation of the environment around the vehicle 200, including any other entities such as other vehicles, pedestrians or obstructions nearby. The illustrated architecture 600 includes a number of modules, wherein each module may include one or more layers of nodes of a neural network. As illustrated in Figure 6 sensor data 602 is received at an image view encoder 604 that performs image feature extraction. An output of the image view encoder 604 is provided to a view transformer module 606 that transforms the extracted features from the image view to the Bird's-eye-view. In some examples, the image view encoder 606 may generate a point cloud representation 608 of the scene. The output of the image view encoder 606 is then input to the bird'seye-view (BEV) encoder 610 that encodes the BEV features. Image view encoder 604, view transformer module 606, and BEV encoder 610 may form part of a pretrained perception stack present on the vehicle 200 and that can be used to implement other features of the vehicle 200, such as active safety features or advanced driving assistance features.

To generate a cost map, a dedicated cost map head 612 is provided that receives the output of the BEV encoder 610. To train the neural network architecture 600 to generate the cost map, the cost map head 612 is trained while weights associated with nodes of the image view encoder 604, view transformer module 606 and BEV encoder 610 remain fixed. According to embodiments, a training data set comprising input/output pairs of captured sensor data (e.g. camera, radar, or lidar data) and a corresponding monitored trajectory is used to train the neural network by modifying weights of the cost map head 612 using optimizing on softmax cross-entropy loss. Trajectories may be clustered into one of a plurality of template trajectories that capture all possible trajectories. Then, from the cluster of trajectories a class may be selected, which best represents the monitored trajectory based on a distance measure, e.g. Euclidian (L2) distance. The selected class may then be used as a ground-truth class for optimizing the weights in the neural network.

The training data may include trajectories collected over a body of drives in different conditions, e.g. 30 hours of driving in urban, countryside and highway scenarios, varying weather conditions and seasons, observing a set of behaviours from other road users (cut/-in, slow drive ahead, overtaking), each training data pair comprising trajectory data for the previous five seconds, together with the perception system's 304 sensor input (camera/radar/lidar). Based on the training data, the neural network 600 may be trained to associate a low cost value with a target trajectory that matches the monitored trajectory of the output training sample for corresponding input sensor data. Thus, the neural network 600 learns to generate a cost map 306 for each set of input sensor data which associates a low cost to the target trajectory.

Further according to the method 300, trajectory information 308 describing a monitored trajectory over a predefined period of time, that resulted in the vehicle 200 arriving at the current position, is received from the trajectory monitoring module 104. For example, the trajectory may be monitored over five seconds. The trajectory information 308 is compared with the cost map 306 that has been generated based on the input sensor data to generate an environment-based driving score in block 310.

For example, the cost map 306 may define a cost value for each of a plurality of points in BEV space through which possible trajectories may pass and a cost for a trajectory can then be computed by summing over all costs values of trajectory points in the cost map for the actual trajectory as described by the trajectory information 308. That is, for a cost map C E RnY with X and Y denoting the width and length of the BEV space, for any trajectory ri defined by ri = pct, yt, tth its cost C(C,-rj) can be computed by summing over all costs of trajectory's points (xt, yt) c(c,T,) = c(xtryt) (rt hiett In embodiments, the environment-based driving score may be generated by determining, using the cost map 306, a cost associated with the monitored trajectory, normalizing the cost, and calculating a weighted average of the costs over a predefined period of time, for example a preceding thirty minutes.

In some examples, fine-tuning training of the cost map head 612 may be performed based on driver specific trajectories and sensor data collected during operation of the vehicle 200 by a particular driver. Such trajectories may be collected over an initial training period, e.g. a first 30 hours of driving, and should preferably encompass a wide range of driving conditions, e.g. urban, countryside and highway scenarios, varying weather conditions, etc. Sensor data and corresponding trajectories captured during the initial training period may form input/output pairs of a fine-tuning training set and used to fine-tune the neural network to reflect a driving style of a particular driver by further modifying the weights of the cost map head using an optimization algorithm as described above.

According to the illustrated method 300, trajectory information 308 is further provided to the trajectory reconstruction module 312. The reconstructed trajectory together with the monitored trajectory 308 is forwarded to determine a trajectory driving score in isolation of any environmental factors (314). According to embodiments, trajectory reconstruction module 312 may comprise an autoencoder model 500 as illustrated in Figure 5. The autoencoder model 500 comprises an encoder 504 that encodes an input 502 into a low dimension representation in latent space 506, and a decoder 508 that reverses the operation of the encoder 504 to decode the encoded representation and recreate a predicted input 510. Thus, by applying monitored a training set of trajectories as input data to the autoencoder model 500, the autoencoder 500 can be trained to correctly reconstruct normal trajectories, i.e. for an input trajectory Ti 502 similar to one of the training set, the autoencoder model 500 will generate a very similar output trajectory 510 ti = Ti. However, for abnormal/unusual trajectories not falling within the training set, the output trajectory 510 will differ from the input 502 so that a distance measure between the input trajectory 502 and the reconstructed trajectory 510, e.g. -Till. can be used as a metric to measure a normalness of trajectories.

Thus, a monitored trajectory with unusual braking/acceleration or steering patterns will result in a large distance measure between the input and recreated trajectories, and thus a lower trajectory driving score.

In some embodiments, the autoencoder model 500 may comprises a pretrained model, i.e. pretrained encoder/decoder, that has been trained on a large number of normal trajectories captured from a wide range of drivers having different driving styles (e.g. calm, hesitant, fast, etc.).

Further training of the autoencoder model 500 may be performed to tailor the driver behaviour model captured in the autoencoder model 500 to a particular driver based on trajectory information captured while that driver is operating the vehicle 200. For example, during a training period trajectory information may be monitored while the particular driver operates the vehicle 200 and used to fine-tune the autoencoder 500 to recreate trajectories similar to a behaviour of that driver. Thus, trajectory driver scores may be based on a baseline driver model that reflects a normal driving style of that particular driver. Such an approach may avoid false positive detections of divergence from the baseline driving behaviour that might occur if a common driving behaviour model is used to evaluate trajectories for a driver with a harsh driving style. Alternatively, a generally calm and accurate driver that becomes fatigued may not be detected when evaluated against a common driving behaviour model.

A final driving score value 316 is then generated based on the trajectory driving score and the environment-based driving score, for example by summing the trajectory driving score and the environment based driving score. The final driving score value 316 may be normalized to be a value ranging from zero to one, where zero is a worst driving style compared to the baseline driving behaviour and one of a driving style in line with the base line driving behaviour.

In embodiments, the final driving score value is compared against a threshold value. The driving score value falling below the threshold value may be indicative of the monitored trajectory significantly diverging from the baseline behaviour, for example indicating that a driver has become fatigued. Alternatively, in some embodiments, a driver score value greater than the threshold value may be considered indicative of the monitored trajectory significantly diverging from the baseline behaviour.

Figure 4 is a flowchart illustrating a method 400 according to embodiments of the invention. According to the method 400, at block 402, a current trajectory of a vehicle is monitored. The monitored trajectory is then used to calculate 404 a trajectory score based on a comparison of the monitored trajectory with a driver behaviour model, for example using autoencoder model 500. At block 406, sensor input from one or more sensors of the vehicle is captured to monitor the environment around the vehicle, e.g. including the presence and movement of other vehicles, etc. Based on the monitored trajectory and the sensor input, an environment-based driving score is calculated 408, for example as described above. In block 410, a driver score value is calculated by combining the trajectory score and the environment-based driving score. In block 412, the driver score value is compared against a threshold value to determine whether the driver score value indicates that the monitored driver behaviour diverges from a baseline driving behaviour. If not, the method returns to blocks 402 and 406 and continues monitoring the trajectory and environment of the vehicle. If the driver score value is determined to be indicative of the driver behaviour diverging from the baseline driver behaviour, a signal indicative of the divergence from the baseline driver behaviour is generated 414.

Figure 7 is a flowchart illustrating a method 700 according to embodiments of the invention. According to the method 700, at block 702 a driving score indicative of an alertness level of a driver of the vehicle is obtained. For example, the driving score may be calculated using the method 300 of Figure 3 and/or the method 400 of Figure 4 described above. The obtained driving score is compared against a threshold value at block 704 to determine whether the driving score indicates that there is a reduction in alertness of the driver, for example by indicating a divergence of the driving behaviour of the operator of the vehicle as compared to the baseline driving behaviour, or from the learned model of the driver's usual driving behaviour. In response to determining that the driving score indicates there is a reduction in alertness of the driver, a setting associated with an ambient condition of the interior environment of the vehicle may be modified 706. In particular, the modified setting may relate to an action that promotes alertness of the driver.

In some embodiments, in response to the determination that the driving score indicates there is a reduction in alertness of the driver, an indication may be generated, such as in block 414 of method 400, and the modification of the setting performed in block 706 of method 700 may then be in response to receiving the generated indication.

According to embodiments, one or more actions may be taken by a control system of the vehicle in response to the signal indicative of the divergence from the baseline driving behaviour, for example to increase an alertness of the driver to mitigate any tiredness that may have caused the divergence from the driver's normal behaviour. Such actions may include modifying a setting relating to an ambient condition of an internal environment of the vehicle, that is the ambient conditions surrounding the driver such as a setting relating to in-car lighting, climate control or audio settings. As an example, in response to the indication, the climate control system could be controlled to provide a flow of cooler fresh air that might be expected to 'wake-up' the driver and lead to improved alertness.

Particular examples of actions that can be taken in response to the generated signal include: * Lighting o an intensity of interior lighting o an illumination temperature or colour of the interior lights * Climate control o a cabin temperature setting o a driver's seat temperature setting o a heated or ventilated driver's seat setting o an airflow level setting * Audio (e.g. infotainment system) o a volume level o selection of a predefined playlist In some embodiments, the action taken may be, at least in part, proportional to the driver score value. For example, a lower driver score value indicating a greater level of divergence from the baseline driving model, may cause a greater reduction in cabin temperature setting to promote awareness of the driver. Mapping between the driver score value and the action performed may be linear or may be based on a calibration equation defined by a manufacturer or by a user via a human machine interface.

In some embodiments, the vehicle 200 may be operated by an autonomous control system of an advanced driving assistance system, and the method 300 may be used to generate a metric comparing the operation of the vehicle by the autonomous control system against the baseline driver behaviour, for example to score the accuracy of the autonomous system versus a learned driver behaviour model. Such a score may be useful in determining how close the actions of the autonomous system are to a typical human driver in the same conditions or situations.

Certain methods and systems as described herein may be implemented by one or more processors that processes program code that is retrieved from a non-transitory storage medium. Figure 8 shows an example 800 of a device comprising a computer-readable storage medium 830 coupled to at least one processor 820. The computer-readable media 830 can be any media that can contain, store, or maintain programs and data for use by or in connection with an instruction execution system. Computer-readable media can comprise any one of many physical media such as, for example, electronic, magnetic, optical, electromagnetic, or semiconductor media. More specific examples of suitable machine-readable media include, but are not limited to, a hard drive, a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory, or a portable disc.

In Figure 8, the computer-readable storage medium comprises program code to perform a method corresponding to the embodiment shown in Figure 7, that is: obtain 702 a driving score indicative of an alertness level of a driver of the vehicle; determine 704 based on a comparison of the driving score with a threshold value that the driving score indicates that there is a reduction in alertness of the driver; and modify 706 a setting associated with an ambient condition of the interior environment of the vehicle.

In embodiments, the computer-readable storage medium may comprise program code to perform a method corresponding to the embodiments of Figures 3 or 4.

It will be appreciated that various changes and modifications can be made to the present invention without departing from the scope of the present application.

Claims

CLAIMS1. A control system for monitoring a divergence from a baseline driving behaviour of an operator of a vehicle, the control system comprising one or more processors collectively configured to: monitor a trajectory of the vehicle; calculate a trajectory score based on a comparison of the monitored trajectory with a driver behaviour model; monitor, using at least one sensor of the vehicle, an environment around the vehicle; calculate an environment-based driving score based on the monitored trajectory and the monitored environment; calculate a driving score value based on the trajectory score and the environment-based driving score; compare the driving score value with a threshold value to identify a divergence from the baseline driving behaviour; and generate a signal indicating the divergence from the baseline driving behaviour.
2. The control system of claim 1, wherein the one or more processors are collectively configured to calculate the trajectory score by: encoding the monitored trajectory based on the driver behaviour model, the driver behaviour model comprising data representative of the baseline driving behaviour; determining a reconstructed trajectory based on the encoded trajectory and the driver behaviour model; and calculating the trajectory score based on a distance between the monitored trajectory and the reconstructed trajectory.
3. The control system of claim 1 or claim 2, wherein the driver behaviour model comprises an autoencoder model pre-trained on monitored trajectories of a plurality of drivers.
4. The control system of any of claims 1 to 3, wherein the one or more processors are collectively configured to calculate the environment-based driving score by: generating a cost map for a plurality of possible trajectories based on an input of the at least one sensor; and comparing the monitored trajectory with the cost map.
5. The control system of claim 4, wherein the one or more processors are collectively configured to compare the monitored trajectory with the cost map by, for each point (xe, yt) of the monitored trajectory, summing a cost value of the cost map associated with the point of the monitored trajectory.
6. The control system of claim 4 or claim 5, wherein the one or more processors are collectively configured to generate the cost map by: transforming the input of the at least one sensor into a bird's-eye-view (BEV) space to generate a BEV representation of the environment around the vehicle; and processing the BEV representation using a neural network to generate the cost map.
7. The control system of claim 6, wherein the neural network is trained in conjunction with a perception model to predict a cost map, wherein the perception model is configured to receive the at least one sensor input and generate the BEV representation, the training data comprising training example input/output pairs of at least one sensor input and an associated trajectory.
8. The control system of claim 3, wherein the baseline driving behaviour comprises a learned baseline driving behaviour associated with a driver of the vehicle, and wherein the autoencoder model is finetuned based on monitored trajectories of the vehicle for a training period while the vehicle is operated by the driver.
9. The control system of any preceding claim, wherein the calculated driving score value is indicative of an alertness level of the driver, and wherein the divergence from the baseline driving behaviour is indicative of a reduction in alertness of the driver.
10. The control system of claim 9, wherein the one or more processors are collectively configured to, in response to the signal indicating the divergence from the baseline driving behaviour, modify a setting associated with an ambient condition of an interior environment of the vehicle.
11. The control system of claim 10, wherein the setting associated with the ambient condition of the interior environment of the vehicle comprises one of: an air conditioning setting; an interior illumination setting; an infotainment volume setting; and a heated driver's seat setting.
12. The control system of any of claims 1 to 7 wherein the one or more processors are collectively configured to: operate the vehicle using an advance driving assistance system of the vehicle; and generate the driving score value, the driving score value representative of a divergence of a trajectory controlled by the advance driving assistance system and the baseline driving behaviour.
13. A method of determining a driving score representative of a divergence from a baseline driving behaviour, the method comprising: monitoring a trajectory of a vehicle; calculating a trajectory score based on a comparison of the monitored trajectory with a driver behaviour model; monitoring, using at least one sensor, an environment around the vehicle; calculating an environment-based driving score based on the monitored trajectory and the monitored environment; calculating a driving score value based on the trajectory score and the environment-based driving score; comparing the driving score value with a threshold value to identify a divergence from the baseline driving behaviour; and generating a signal indicative of the divergence from the baseline driving behaviour.
14. A computer program product comprising computer program code that when executed on a processor of a control system of a vehicle including at least one environment sensor causes the control system to perform the method claim 13.
15. A vehicle comprising the control system of any of claims 1 to 12.