NL2033199B1

NL2033199B1 - System for establishing indirect communication between a driver of a vehicle and a road user located nearby the vehicle

Info

Publication number: NL2033199B1
Application number: NL2033199A
Authority: NL
Inventors: Adrianus Wernaart Hendricus
Original assignee: Daf Trucks Nv
Priority date: 2022-09-30
Filing date: 2022-09-30
Publication date: 2024-04-08

Abstract

A system for establishing indirect communication between a driver of a vehicle and a road user located nearby the vehicle. A first camera device is mounted to an exterior side of the vehicle and faces an area adjacent to the vehicle, for recording a ﬁrst image that comprises the road user’s eyes. Based on a recorded first image, an image processor is arranged for determining a first gaze direction of the road user’s eyes. Upon determining that the ﬁrst gaze direction connects the road user’s eyes With the signalling device, the image processor transmits a ﬁrst output to a safety system for operating a signalling device to generate a first alert signal.

Description

P133664NL00

Title: SYSTEM FOR ESTABLISHING INDIRECT COMMUNICATION

BETWEEN A DRIVER OF A VEHICLE AND A ROAD USER

LOCATED NEARBY THE VEHICLE

The invention relates to a system for establishing indirect communication between a driver of a vehicle, such as a truck, and a road user located nearby the vehicle. The invention further relates to a truck or tractor-trailer combination, provided with such a system.

Nowadays, more and more vehicles, such as trucks, are provided with external cameras, e.g. to replace mirrors. In this way, fuel economy of the vehicle can be improved, as well as the view of the vehicle’s surroundings. External cameras may also enable (partially) automated driving manoeuvres.

In traffic situations, it is important that there is some kind of communication between the driver of the vehicle and other road users, such as pedestrians, cyclists, or drivers of other vehicles, for creating mutual awareness of each other’s presence, priority and space on the road. In this way, hazardous situations can be avoided. Conventionally, this can be done by establishing eye contact. By introducing external cameras to visualize the vehicle’s surroundings, however, eye contact between the driver and other road users may be limited.

It is an object of the present invention to improve communication between a driver of a vehicle provided with external cameras, and other road users.

SUMMARY

In summary, the invention pertains to a system for establishing indirect communication between a driver of a vehicle and a road user located nearby the vehicle. The system comprises a first camera device, an image processor, and a safety system.

The first camera device is mounted to an exterior side of the vehicle and faces an area adjacent to the vehicle, for recording a first image that comprises the road user's eyes. The image processor is in communication with the first camera device, and is arranged for processing the recorded first image. The safety system comprises a signalling device mounted to the exterior side and arranged for generating alert signals to the road user and/or the driver. The safety system is in communication with the image processor and is arranged for operating the signalling device.

Based on the recorded first image, the image processor is arranged for determining a first gaze direction of the road user’s eyes. Upon determining that the first gaze direction connects the road user’s eyes with the signalling device, the image processor transmits a first output to the safety system for operating the signalling device to generate a first alert signal.

By having the signalling device generate the first alert signal, e.g. a visual and/or auditive signal, the road user is acknowledged of his attempt to communicate with the driver, while the driver is alerted that a road user 1s present nearby the vehicle and attempting to communicate. As such, communication by an alert signal, that can e.g. represent a form of indirect eye contact, is established between the road user and the driver of the vehicle, via the safety system.

Preferably, the system further comprises a display device and a second camera device. The display device is provided inside a cabin of the vehicle and is in communication with the first camera device. The display device is arranged for displaying the recorded first image to the driver. The second camera device is mounted inside the cabin and facing the driver, for recording a second image that comprises the driver's eyes.

The image processor further is in communication with the second camera device and arranged for processing the recorded second image.

Based on the recorded second image, the image processor is arranged for determining a second gaze direction of the driver's eyes. Upon determining that the second gaze direction connects the driver's eyes with the display device and/or the signalling device, the image processor transmits a second output to the safety system for operating the signalling device to generate a second alert signal, indicating that the driver has identified the presence of a vulnerable road user.

By having the signalling device generate the second alert signal, the driver is able to notify the road user that the driver is attempting to communicate with the road user. For example, in case the road user has already triggered the first alert signal by connecting his first gaze direction with the signalling device, the driver can trigger the second alert signal by connecting his second gaze direction with the display device, thereby providing feedback to the road user, e.g. that the driver of the vehicle is aware of the road user. The second alert signal can e.g. represent a form of indirect eye contact from the driver towards the road user.

Alternatively, operation of the signalling device can be initiated by the second alert signal, when the driver spots the road user first. In such cases, the driver triggers the second alert signal by connecting his second gaze direction with the display device. The second alert signal acknowledges the drive of his attempt to communicate with the road user, while the road user is alerted that the driver is attempting to communicate. By connecting his first gaze direction with the signalling device, the road user can provide feedback to the driver, e.g. that the road user is aware of the vehicle.

The second camera device can e.g. be part of the display device.

Accordingly, while the driver watches the display device to view the first image recorded by the first camera device, the second camera device is positioned substantially opposite the viewing direction of the driver. As a result, the accuracy of the image processor for determining whether the second gaze direction connects with the display device is improved.

In another or further variant, the safety system can comprise an additional signalling device mounted inside the cabin and arranged for generating an additional alert signal to the driver, wherein the image processor is arranged for transmitting a third output to the safety system for operating the additional signalling device, in response to determining that the first gaze direction connects the road user’s eyes with the signalling device, while determining that the second gaze direction does not connect the driver’s eyes with the display device. The additional alert signal inside the cabin can thus be used for additionally alerting the driver while he is not responding to the first alert signal, which is generated by the signalling device mounted to the exterior side of the vehicle.

The additional signalling device can e.g. be part of the display device. For example, the additional signalling device can be a light source or audio speaker mounted to the display device. Alternatively, or additionally, the additional signalling device can comprise digital projection means arranged for projecting an alert signal on the display device.

For example, the additional signalling device can comprise an image overlay arranged for projecting the additional alert signal onto the first image displayed by the display device.

In some embodiments, the image processor is arranged for determining that the second gaze direction connects the driver's eyes with the road user’s eyes displayed in the first image. Accordingly, the system can determine whether indirect eye contact between the driver and the road user is established.

To provide an alert signal that is discernible regardless of the position of the signalling device relative to the road user and/or driver, the signalling device can comprise a sound generator arranged for generating sounds, e.g. a horn or an audio speaker, wherein the safety system is arranged for operating the sound generator to generate the first and/or second alert signal.

In some cases, it may not be advisable to provide an auditive alert signal, e.g. due to traffic regulations. Alternatively, or additionally, the 5 signalling device can comprise a light source arranged for generating light, wherein the safety system is arranged for operating the light source to generate the first and/or second alert signal.

Preferably, the light source is in a line of sight of the road user and the driver. In this way, the first and/or second alert signal generated by the light source is visible for both the road user as well as the driver.

The light source can e.g. be a multicolour LED, wherein the multicolour LED is switchable between an initial condition, an alert condition, and a confirm condition, wherein, in absence of the first and second output, the safety system is arranged for switching the multicolour

LED to the initial condition, wherein, in response to the first or second output, the safety system is arranged for switching the multicolour LED to the alert condition, and wherein, in response to a combination of the first and second output, the safety system is arranged for switching the multicolour LED to the confirm condition. In the initial condition, the multicolour LED can e.g. be switched off, thus not generating any light. In the alert condition, the multicolour LED can e.g. be generating a blinking or flashing light signal in a first colour, to alert the driver and/or the road user.

The first colour can e.g. be orange or red, to represent a potentially hazardous situation. In the confirm condition, the multicolour LED can e.g. be generating a solid, continuous light signal in a second colour, to confirm that mutual indirect eye contact is established between the driver and the road user. The second colour can e.g. be green or blue, to represent a safe situation.

In some embodiments, the signalling device is part of the first camera device. For example, the signalling device can be provided adjacent to the lens of the first camera device, e.g. forming a ring around the camera lens. Accordingly, when the first gaze direction of the road user connects with the signalling device, the first image recorded by the first camera device comprises the road user’s eyes gazing substantially into the camera lens. This improves the accuracy of the image processor for determining the first gaze direction, and also provides a sense of direct eye contact to the driver when he watches the recorded first image on the display device.

Preferably, the signalling device is manually operable by the driver for generating an alert signal to the road user. In this way, the driver can operate the signalling device to generate additional alert signals, when the road user is not responding to the first and/or second alert signal.

Other aspects of the invention pertain to a truck or tractor-trailer combination, provided with a system as described herein. In some embodiments of the truck or tractor-trailer combination, the first camera device is a side mirror replacing camera device. Accordingly, while the truck or tractor-trailer combination is lacking conventional side mirrors, the driver is still able to communicate with other road users via indirect eye contact, by having the truck or tractor-trailer combination provided with a system as described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be further elucidated in the figures:

FIG 1 illustrates an embodiment of a system for establishing indirect communication between a driver of a vehicle and a road user located nearby the vehicle;

FIG 2 illustrates another or further embodiment of the system;

FIG 3 illustrates yet another or further embodiment of the system;

FIG 4 illustrates an embodiment of a signaling device of the system;

FIG 5 illustrates an embodiment of a vehicle, provided with a system as described herein.

DETAILED DESCRIPTION

The invention is described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown.

In the drawings, the absolute and relative sizes of systems, components, layers, and regions may be exaggerated for clarity. Embodiments may be described with reference to schematic and/or cross-section illustrations of possibly idealized embodiments and intermediate structures of the invention. In the description and drawings, like numbers refer to like elements throughout. Relative terms as well as derivatives thereof should be construed to refer to the orientation as then described or as shown in the drawing under discussion. These relative terms are for convenience of description and do not require that the system be constructed or operated in a particular orientation unless stated otherwise.

FIG 1 illustrates a system 100 for establishing indirect communication between a driver 10 of a vehicle 1 and a road user 20 located nearby the vehicle 1. The vehicle 1 can be any type of vehicle, preferably a road vehicle. The vehicle 1 can be suitable for private applications, e.g. passenger cars, for commercial applications, e.g. buses or trucks, for heavy duty or industrial applications, e.g. wheel loaders or excavators, or for agricultural applications, e.g. tractors or tillage machines. The driver 10 is the person in control of operating the vehicle 1, e.g. in traffic. The road user 20 can be any other person in the vicinity of the vehicle 1, e.g. participating in a mutual traffic or work situation, such as a pedestrian, a (motor)cyclist, a driver of another vehicle, or a co-worker. Seen from the driver's perspective, the vicinity of the vehicle 1 can e.g. comprises an area 3 around a right-hand side to the right of the vehicle 1, around a left-hand side to the left of the vehicle 1, around a front side in front of the vehicle 1, and/or around a rear side behind the vehicle 1.

As illustrated, the system 100 comprises a first camera device 110, mounted to an exterior side 2 of the vehicle 1. The first camera device 110 faces an area 3 adjacent to the vehicle 2. The road user 20 is depicted inside the area 3. Accordingly, the first camera device 110 is arranged for recording a first image 111 that comprises at least the road user’s eyes 21.

The system 100 further comprises an image processor 120, which is in communication with the first camera device 110 and arranged for processing the recorded first image 111. Based on the recorded first image 111, the image processor 120 is arranged for determining a first gaze direction 22 of the road user’s eyes 21. The first image 111 may further comprise the body of the road user 20 entirely, or may comprise other body parts of the road user 20. As such, besides determining a first gaze direction 22, the image processor 120 can be further arranged for determining a body posture or movement direction of the road user 20.

A signalling device 131 is mounted to the exterior side 2 of the vehicle 1 and is arranged for generating alert signals to the road user 20 and/or the driver 10. The signalling device 131 can be a separate unit, independently mounted to the exterior side 2, or can be integrated in another component of the vehicle. For example, the signalling device 131 can be part of the first camera device 110.

Upon determining that the first gaze direction 22 connects the road user’s eyes 21 with the signalling device 131, the image processor 120 transmits a first output 121 to a safety system 130 for operating the signalling device 131 to generate a first alert signal 141, such as a flashing or blinking light, or a sound. For example, the signalling device 131 can comprise a sound generator 132 for generating sounds, and the safety system 130 can be arranged for operating the sound generator 132 to generate alert signals. Alternatively, or additionally, the signalling device 131 can comprise a light source 133 for generating light, and the safety system 130 can be arranged for operating the light source 133 to generate alert signals. Preferably, the light source 133 is in a line of sight of both the road user 20 as well as the driver 10.

The first alert signal 141 is generated in response to the image processor 120 determining that the first gaze direction 22 connects the road user's eyes 21 with the signalling device 131. Accordingly, the first alert signal 141 can e.g. represent indirect eye contact from the road user 20 towards the driver 10. By the first alert signal 141, the road user 20 is acknowledged of his attempt to communicate with the driver 10, while the driver 10 is alerted of the presence of the road user 20.

FIG 2 illustrates another or further embodiment of the system 100.

A display device 150, e.g. comprising a screen or other type of image projection means, is provided inside a cabin 4, or interior, of the vehicle 1.

The display device 150 is in communication with the first camera device 110 and is arranged for displaying the recorded first image 111 to the driver 10, who is seated inside the cabin 4 or interior of the vehicle 1. The display device 150 can e.g. be mounted in or to the dashboard, the rear-view mirror console, or the centre console of the vehicle 1. Accordingly, the display device is positioned in a line of sight without the driver having to change head posture during driving, to improve reaction time. Alternatively, or additionally, the system 100 can comprise a display device 150 mounted to an interior side 5 of the cabin 4, wherein the interior side corresponds to the exterior side 2 of the vehicle to which the first camera device 110 is mounted. Accordingly, the driver may need to deliberately change head posture to watch the display device, thereby improving the accuracy of the image processor for determining the second gaze direction. The system 100 can comprise more than one display device 150 of the types described above to combine their respective benefits.

As illustrated in FIG 2, a second camera device 160 1s mounted inside the cabin 4 or interior of the vehicle 1 and faces the driver 10. The second camera device 160 can e.g. be mounted in or to the dashboard, the rear-view mirror console, or a cabin panel. The second camera device 1s configured to record a second image 161 that comprises the driver's eyes 11.

The second camera device 160 can be a stand-alone unit, having its own mount for mounting to the cabin 4, or can e.g. be part of other components of the vehicle 1, such as the display device 150.

The image processor 120 is in communication with the second camera device 160 and is arranged for processing the recorded second image 161. Based on the recorded second image 161, the image processor 120 determines a second gaze direction 12 of the driver's eyes 11 and determines that the second gaze direction 12 connects the driver's eyes 11 with the display device 150. More specifically, the image processor 120 can e.g. be arranged for determining that the second gaze direction 22 connects the driver's eyes 11 with the road user’s eyes displayed in the first image 111.

Upon determining one of the above, the image processor 120 transmits a second output 122 to the safety system 130 for operating the signalling device 131 to generate a second alert signal 142 that can e.g. represent indirect eye contact from the driver 10 towards the road user 20.

Additionally, the signalling device 131 can also be manually operable by the driver 10 for generating an alert signal 140 to the road user 20. For example, the cabin 4 or vehicle interior can comprise a user interface 139, e.g. comprising a knob, push button, or lever. The user interface 139 can e.g. be part of the display device 150. When operated by the driver, the user interface 139 can e.g. be arranged for sending a control signal to the safety system 130 for operating the signalling device 131, to generate a third alert signal 143. Accordingly, the driver can alert the road user 10, without the having to connect his second gaze direction 22 with the display device 150. The third alert signal 143 can be different to the second or first alert signal 142, 141. The third alert signal 143 may have a higher intensity, or a different frequency or colour, than the second or first alert signal 142, 141.

In this way, in case the road user 20 e.g. is not responding to the second alert signal 142, the driver 10 can attempt to get his attention by manually operating the signalling device 131.

FIG 3 illustrates yet another or further embodiment of the system 100. Here, the safety system 130 comprises an additional signalling device 134 mounted inside the cabin 4. The additional signalling device 134 is able to generate an additional alert signal 144 to the driver 10, such as a light, sound, or haptic signal. The additional signalling device 134 is especially useful in cases when a road user 20 is attempting to communicate with the driver 10, but the driver 10 does not notice the first alert signal 141 generated by the signalling device 131.

In such cases, the image processor 120 transmits a third output 123 to the safety system 130 for operating the additional signalling device 134, in response to determining that the first gaze direction 22 connects the road user's eyes 21 with the signalling device 131, while determining that the second gaze direction 12 does not connect the driver's eyes 11 with the display device 150.

The additional signalling device 134 can be part of the display device 150. For example, the additional signalling device 134 can be a LED or other type of light source, or a sound speaker, mounted to the display device 150. Alternatively, or additionally, the additional signalling device 134 can be a head up display system e.g. arranged for projecting the additional alert signal 144 onto the windshield, or can be a haptic feedback device. Such a haptic feedback device can e.g. be mounted to the steering wheel or driver's seat of the vehicle 1 and arranged for providing the additional alert signal 144 in the form of a haptic signal, such as a vibration, in response to the third output 123.

The additional signalling device 134 may also comprise an image overlay arranged for projecting the additional alert signal 144 onto the first image 111 displayed by the display device 150.

FIG 4 illustrates an embodiment of a signalling device 131 of the system 100. The signalling device 131 can comprise a light source 133 provided by a switchable, multicolour LED 136. As illustrated, the multicolour LED 136 is switchable between an initial condition 136-1, an alert condition 136-2, and a confirm condition 136-3. In absence of the first and second output, the safety system can be arranged for switching the multicolour LED 136 to the initial condition 136-1. Accordingly, the initial condition 136-1 can be used for indicating that there is no communication between the driver and the road user. In the initial condition 136-1, the multicolour LED 136 can e.g. be switched off, or can e.g. be generating light of a neutral colour, such as white or blue. The light can e.g. be generated continuously at a relatively low intensity, or flashing at a relatively low frequency, to still make the signalling device 131 visible in low ambient lighting conditions.

In response to the first output or the second output, the safety system 130 can be arranged for switching the multicolour LED 136 to the alert condition 136-2. Accordingly, the alert condition 136-2 can be used for indicating that one of the driver or road user is attempting to communicate with the other. In the alert condition 136-2, the multicolour LED 136 can e.g. be arranged for generating light of a bright signal colour, such as yellow, orange, or red. The light can e.g. be generated continuously at a relatively high intensity, or flashing at a relatively high frequency, to alert the driver and/or the road user that an attempt to communicate has been made.

In response to a combination of the first output and the second output, the safety system can be arranged for switching the multicolour

LED 136 to the confirm condition 136-3. Accordingly, the confirm condition 136-3 can be used for indicating, or confirming, that both the driver as well as the road user have established mutual communication and thus are aware of each other. In the confirm condition 136-3, the multicolour LED 136-3 can e.g. be arranged for generating light of a safety colour, such as blue or green. The light can e.g. be generated continuously or blinking, flashing at a certain frequency, to indicate that mutual communication between the driver and the road user is established.

FIG 5 illustrates a truck or tractor-trailer combination 1000, provided with a system 100 as described herein. In stead of side mirrors, the truck 1000 can be provided with side mirror replacing camera devices, each of which is formed by a first camera device 110 of the system 100. As such, fuel efficiency of the truck 1000, and the view of the surroundings of the driver 10 of the truck 1000 can be improved. By having the truck 1000 equipped with a system 100 as described herein, the driver 10 remains able to communicate with road users located nearby the truck 1000.

It is thus believed that the operation and construction of the present invention will be apparent from the foregoing description and drawings appended thereto. For the purpose of clarity and a concise description features are described herein as part of the same or separate embodiments, however, it will be appreciated that the scope of the invention may include embodiments having combinations of all or some of the features described.

The invention applies not only to automotive applications where the system is used for communication purposes, but also to other technical, industrial or agricultural applications where a system for establishing indirect communication 1s used. It will be clear to the skilled person that the invention is not limited to any embodiment herein described and that modifications are possible which may be considered within the scope of the appended claims. Also, kinematic inversions are considered inherently disclosed and can be within the scope of the invention. In the claims, any reference signs shall not be construed as limiting the claim.

The terms ‘comprising’ and ‘including’ when used in this description or the appended claims should not be construed in an exclusive or exhaustive sense but rather in an inclusive sense. Thus, expression as ‘including’ or ‘comprising’ as used herein does not exclude the presence of other elements,

additional structure or additional acts or steps in addition to those listed.

Furthermore, the words ‘a’ and ‘an’ shall not be construed as limited to ‘only one’, but instead are used to mean ‘at least one’, and do not exclude a plurality.

Features that are not specifically or explicitly described or claimed may additionally be included in the structure of the invention without departing from its scope.

Expressions such as: "means for ...” should be read as: "component configured for …" or "member constructed to …" and should be construed to include equivalents for the structures disclosed. The use of expressions like: "critical", "preferred", "especially preferred" etc. is not intended to limit the invention. To the extent that structure, material, or acts are considered to be essential they are inexpressively indicated as such. Additions, deletions, and modifications within the purview of the skilled person may generally be made without departing from the scope of the invention, as determined by the claims.

Claims

Conclusions

1. A system for establishing indirect communication between a driver of a vehicle and a road user located near the vehicle, comprising: - a first camera device attached to an outside of the vehicle and directed towards a near vehicle-located area, for capturing a first image that includes the road user's eyes; - an image processor, which is in communication with the first camera device and is adapted to process the captured first image; - a safety system comprising a signaling device mounted on the outside and designed to generate attention signals towards the road user and/or the driver, wherein the safety system is in communication with the image processor and designed to operate the signaling device; wherein the image processor is designed to determine a first viewing direction of the road user's eyes based on the recorded first image; and wherein the image processor, upon causing the first viewing direction to connect the road user's eyes to the signaling device, sends a first output to the safety system for operating the signaling device to generate a first attention signal.

A system according to claim 1, further comprising: - a display device provided within a cabin of the vehicle, the display device being in communication with the first camera device and arranged to display the captured first image to the driver ; and — a second camera device, mounted inside the cabin and facing the driver, for capturing a second image including the driver's eyes; wherein the image processor is further in communication with the second camera device and is arranged to process the captured second image; wherein the image processor is adapted to determine, based on the captured second image, a second viewing direction of the driver's eyes; and wherein the image processor, upon causing the second viewing direction to connect the driver's eyes to the display device and/or the signaling device, transmits a second output to the safety system for operating the signaling device to generate a second attention signal.

3. System according to claim 2, wherein the second camera device is part of the display device.

4. System according to any one of claims 2-3, wherein the safety system comprises an additional signaling device that is mounted inside the cabin and designed to generate an additional attention signal to the driver, wherein the image processor is designed to send a third party output to the safety system for operating the additional signaling device, 1n response to ensuring that the first viewing direction connects the road user's eyes to the signaling device, while ensuring that the second viewing direction does not connect the driver's eyes to the display device.

5. System according to claim 4, wherein the additional signaling device is part of the display device.

6. System according to claim 4 or 5, wherein the additional signaling device comprises an image overlay that is adapted to project the additional attention signal on the first image shown by the display device.

7. System as claimed in any of the claims 2-6, wherein the image processor is designed to ensure that the second viewing direction connects the eyes of the driver with the eyes of the road user shown in the first image.

8. System according to any of the preceding claims, wherein the signaling device comprises a sound generator that is designed to generate sounds, and wherein the safety system is designed to operate the sound generator to generate the first and/or second attention signal.

9. System according to any of the preceding claims, wherein the signaling device comprises a light source adapted to generate light, and wherein the safety system is adapted to operate the light source to generate the first and/or second attention signal.

System according to claim 9, wherein the light source is located in a field of view of the road user and the driver.

11. System according to claim 9 or 10, wherein the light source is a multi-colored LED, wherein the multi-colored LED is switchable between an initial state, an attention state, and a confirmation state, wherein the safety system is designed to, in the absence of the first and second output , to switch the multi-coloured LED to the initial state, wherein the safety system is arranged to switch the multi-coloured LED to the attention state in response to the first or second output, and wherein the safety system is arranged to, in response to a combination of the first and second output, switch the multi-color LED to the confirmation state.

12. System according to any of the preceding claims, wherein the signaling device is part of the first camera device.

13. System according to any of the preceding claims, wherein the signaling device can be manually operated by the driver to generate an attention signal to the road user.

14. A truck or tractor-trailer combination, provided with a system according to any of the preceding claims.

15. Truck or tractor-trailer combination according to claim 14, wherein the first camera device is a side mirror replacement camera device.