DE102024119599A1 - Color-augmented lane marking for obstacle warning - Google Patents

Abstract

The invention relates to a system for a vehicle (1) for visually augmenting a vehicle's surroundings, wherein a control unit (7) detects another road user (9) in data from a sensor unit (3) and continuously updates and calculates a distance range to the other road user (9), which indicates that the lane change of the vehicle (1) is to be avoided if the distance is within the distance range, and displays a marking (11) visualizing the distance range on a display unit (5), which moves along a corridor that lies, analogous to contact, on an actual or virtually determined lane marking between the lane of the vehicle (1) and the adjacent lane, with a movement of the other road user (9).

Description

The invention relates to a system for a vehicle for visually augmenting a vehicle environment and a method for visually augmenting a vehicle environment.

When a vehicle changes lanes, the driver is typically responsible for not obstructing other road users in adjacent lanes, especially the lane being changed. To do this, the driver typically has to estimate the approach speed of another road user in the intended lane and the distance between that other road user and their own vehicle. While the driver is thus reliant on an imprecise estimate, it can also be difficult for them to accurately judge the required distances to other road users in the intended lane.

Modern vehicles increasingly feature augmentation systems that can overlay additional information onto the vehicle's screen or project augmented reality onto mirrors or glass surfaces. It is known in the prior art that displaying a lane change recommendation can depend on a check to determine whether the change is possible without a collision. Specifically, it is known that when changing lanes and no hazard is detected in the adjacent lane, a simple arrow is discreetly displayed on the head-up display. However, if a vehicle is present in the adjacent lane, the graphic can be modified to display a message to stop the lane change, for example, by flashing the indicator or changing it to red.

The DE 10 2010 013 398 A1 This relates to a method for displaying graphic images on a transparent windshield head-up display of a vehicle, describing the vehicle's journey in relation to a lane, wherein the method comprises: monitoring information inputs describing an operating environment of the vehicle, including monitoring a desired lane and the vehicle's current position; determining a relationship between the desired lane and the vehicle's current position; determining a graphic describing the relationship between the desired lane and the vehicle's current position; and displaying the graphic describing the relationship between the desired lane and the vehicle's current position on the substantially transparent windshield head-up display; and wherein the substantially transparent windshield head-up display comprises light-emitting particles or microstructures over a predefined area of the windshield, allowing a luminescent display while permitting a view through the windshield.

1. a) Aufnehmen von Bildern mit den mindestens fünf Kameras;
2. b) Erzeugen von Spurwechselinformationen aus den aufgenommenen Bildern;
3. c) Anzeigen der Spurwechselinformationen mit der Vorrichtung für eine erweiterte Realität; wobei der Schritt des Erzeugens von Spurwechselinformationen die weiteren Schritte aufweist:
   • b1) Erfassen von Fahrspuren,
   • b2) Erfassen der Position und der Geschwindigkeiten von anderen Fahrzeugen, die das Fahrzeug umgeben, und
   • b3) Erzeugen von mindestens einer Spurwechselempfehlung in einer Spurwechselempfehlungseinheit.

The WO 2017/162278 A1 also relates to a method for supporting a complete lane-changing process using an augmented reality device, wherein the method uses at least five cameras provided on a vehicle, a computing device and the augmented reality device, and wherein the method comprises the steps:

1. a) Taking pictures with at least five cameras;
2. b) Generating lane change information from the recorded images;
3. c) Displaying lane change information using an augmented reality device; wherein the step of generating lane change information comprises the following further steps:
   • b1) Lane detection,
   • b2) Determining the position and speeds of other vehicles surrounding the vehicle, and
   • b3) Generating at least one lane change recommendation in a lane change recommendation unit.

The object of the invention is to provide a visual augmentation of the environment in the vehicle, which allows the driver of the vehicle to better assess the safety of an intended lane change.

The invention is defined by the features of the independent claims. Advantageous further developments and embodiments are the subject of the dependent claims.

A first aspect of the invention relates to a system for a vehicle for visually augmenting a vehicle's surroundings, comprising a sensor unit for detecting the vehicle's surroundings, a display unit, and a control unit designed to recognize another road user in the data from the sensor unit and to determine whether the other road user is within a certain distance of the vehicle when the vehicle changes lanes to an adjacent lane. to take into account the vehicle and, if this is the case, to continuously calculate a distance range to the other road user, which indicates that the vehicle should refrain from changing lanes if the distance is within the distance range, and to display a marking on the display unit visualizing the distance range, which moves along a corridor that lies, analogous to contact on an actual or virtually determined lane marking, between the vehicle's lane and the adjacent lane, with a movement of the other road user.

The system is particularly helpful for a driver considering changing lanes from their current lane to an adjacent lane if there is a possibility that another vehicle is approaching from behind in the adjacent lane. However, the system can also be used to warn of oncoming traffic.

In particular, when a vehicle changes lanes to an adjacent lane, the distance between the other road user and the vehicle must be taken into account if the other road user is already in the adjacent lane or is entering it.

The driver of their own vehicle must maintain a certain distance from other road users who are already in or entering the intended lane. This distance serves to ensure that the other road user is not unnecessarily forced to brake, or has the opportunity to reduce their speed in good time. In any case, the distance must ensure that no collision occurs between the other road user and the driver's own vehicle when the driver's vehicle changes from its current lane into an adjacent lane in which the other road user is already located or entering.

• Von der Steuereinheit wird mithilfe der Daten einer Sensoreinheit des Fahrzeugs, welche Informationen über das Fahrzeugumfeld liefern, die Anwesenheit weiterer Verkehrsteilnehmer erkannt. Jeder erkannte weitere Verkehrsteilnehmer wird daraufhin überprüft, ob er für einen vom Fahrer potenziell intendierten Spurwechsel relevant ist. Beispielsweise können weitere Verkehrsteilnehmer, welche auf einem Standstreifen stehen oder langsam fahren, oder sich auf einem Beschleunigungsstreifen befinden, hierfür nicht beachtet werden. Primär eignet sich das Vorgehen der Steuereinheit dafür, von hinten herannahende weitere Verkehrsteilnehmer, die das eigene Fahrzeug im Begriff sind zu überholen, zu berücksichtigen, es kann jedoch auch alternativ oder ergänzend dazu Gegenverkehr auf einer benachbarten Fahrspur berücksichtigt werden.

To assist the driver in assessing whether a lane change from the current lane to the adjacent lane is safe, ensuring a necessary distance to other road users, the following procedure is followed, whereby the control unit can perform this procedure simultaneously for a large number of other road users:

• The control unit uses data from a vehicle sensor unit, which provides information about the vehicle's surroundings, to detect the presence of other road users. Each detected road user is then checked to determine if they are relevant to a lane change potentially intended by the driver. For example, other road users who are stopped or moving slowly on the hard shoulder, or who are on an acceleration lane, may be disregarded. The control unit's primary function is to take into account other road users approaching from behind who are about to overtake the vehicle; however, it can also alternatively or additionally consider oncoming traffic in an adjacent lane.

The control unit continuously updates and determines a specific distance zone assigned to each other road user. This distance zone generally moves with the other road user, but its extent can be adjusted during updates, for example, if the relative speed between the other road user and the vehicle increases or decreases. Minimum limits can be specified for the distance zone, such as 5 meters in front of and behind another road user.

In the area between another road user and your own vehicle, when the other road user is approaching, the required distance is correspondingly greater than in the area where the other road user is moving away from this intermediate area. For example, if your vehicle is about to overtake a truck by changing lanes, while another road user is approaching from behind at a higher speed, you must consider the speed and distance of the other road user before initiating the overtaking maneuver. However, once the other road user has passed, you can move into the adjacent lane with a smaller distance compared to the approaching road user, as the other road user is then moving away from your vehicle, thus increasing the distance. However, minimum distances must still be observed, which are primarily determined by your vehicle's speed.

The distance zone defines a zone relative to other road users into which the vehicle should not, or may not, enter, as otherwise a required minimum distance to the other road user would be violated. The control unit visualizes this distance zone for the driver by means of a marker superimposed on an image of the vehicle's real-world surroundings. This marker moves along a corridor, i.e., within a corridor, which corresponds to an actual or virtual lane marking between the vehicle's current lane and an adjacent lane.

This corridor extends along the lane marking and is preferably not displayed itself unless the entire corridor is color-coded, with a first color for the marking and a second color outside the marking. The corridor provides a spatial reference for contact-analogous augmentation of the driver's view of their surroundings. The marking itself is generally not contact-analogous to the surroundings, as the marking indicates the distance range, and the distance range is relative to the position of a moving road user. Therefore, the marking moves relative to the road and relative to the lane marking as the other road user to which the marking refers moves.

The lane marking can be a real, actual lane marking, for example, characterized by a series of longitudinal, bar-shaped white or orange stripes. The corridor can then be defined, for example, by image recognition of this lane marking, so that it always follows the bar-shaped stripes; it can also be smoothed to a certain extent.

The virtually determined lane marking can, for example, compensate for gaps in an actual lane marking and continue it virtually. This makes it advantageous to display the marking to indicate the distance between two lanes, even without a continuous and complete physical lane marking.

The marking itself is preferably colored, for example red. The marking extends lengthwise along the virtual or actual lane marking and has a width that essentially corresponds to the width of typical or currently existing bar-shaped marking stripes. Preferably, the marking does not protrude significantly into its own lane or the adjacent lane. The marking is also preferably represented as a continuous, uninterrupted line.

At a front and a rear end of the marking, the marking may fade or change to a different color, for example to a green colored area of the lane marking, to indicate to the driver of the vehicle the possibility of a safe lane change if he finds himself outside the marking at the level of the green area or at the level of the faded area.

If several other road users are traveling in or entering an adjacent lane, the control unit generates multiple markings, which can also overlap. Gaps are visible between the markings for the driver, which they can use to change lanes from their current lane to the adjacent lane.

In a preferred embodiment, a lane marking is continuously colored between the vehicle's own lane and an adjacent lane, the choice of color depending on whether and where a marking for a distance area is displayed.

The system is advantageously able to accurately indicate a gap in traffic approaching from the front and/or from behind with other road users, thus significantly simplifying the driving task for the driver of their own vehicle in order to anticipate the dangers of a lane change.

By having the control unit take into account, in particular, the distance and relative speed between the respective other road user and the driver's own vehicle when determining the length (i.e., the extent) of the distance range and thus the marking, it is no longer necessary for the driver to estimate the safety of a lane change based on an estimated relative speed of the other road user and their distance from it. Instead, comparing the driver's own position with the displayed marking is sufficient. If, as in the example above, the marking is indicated by a red area of the lane marking and areas outside the marking by a green area, then it is sufficient for the driver to determine whether the lane marking is safe when deciding whether to change lanes to an adjacent lane. Position of one's own vehicle at the level of the green zone and preferably assess whether it can execute and complete the lane change in the green zone.

According to an advantageous embodiment, the marking extends longitudinally along the corridor over a length of the distance range and is colored over this length.

According to another advantageous embodiment, the marking transitions smoothly into a different color in the corridor at its front and rear ends.

According to a further advantageous embodiment, the display unit comprises a screen, wherein the control unit is configured to display an image of the vehicle environment obtained from the data of the sensor unit on the screen, wherein the image includes at least part of the adjacent lane.

As an alternative to a screen, the marker can be displayed in a mirror, such as a rearview or side mirror. The marker can also be displayed in the vehicle's head-up display. Other display unit configurations are possible, such as AR glasses or VR glasses. Accordingly, the display unit can show an augmented mirror image or a synthetic image on a screen, such as a camera image. The latter can be used, for example, when a camera-monitor system is used instead of physical exterior mirrors with reflective glass. In this system, external cameras are directed backwards, against the direction of travel, and transmit a video data stream to a corresponding external monitor.

According to another advantageous embodiment, the screen is part of a camera-monitor system that replaces a side mirror.

According to a further advantageous embodiment, the control unit is designed to display the marking only when an adjacent lane and/or an actual lane marking is detected in the data of the sensor unit by the control unit, which meets a predetermined condition.

One such condition might be that a regular lane marking exists between the driver's own lane and the adjacent lane, and that the lane marking does not delineate a hard shoulder or acceleration lane. The marking might also only be displayed if changing lanes is permitted. For example, solid lines are often temporarily installed between lanes in construction zones to indicate that changing lanes is prohibited.

According to a further advantageous embodiment, the control unit is designed to display a respective crossbar directed towards the actual or virtually determined lane marking, analogous to contact with a front end of the road user and analogous to contact with a rear end of the vehicle.

The crossbar helps the driver to better judge the position of their vehicle relative to the boundaries of the marking, as well as the actual distance between their vehicle and other road users, especially when they are close together. This is particularly relevant in scenarios where the marking is used to visualize the distance of another road user approaching from behind, against the direction of travel.

According to another advantageous embodiment, the crossbar ends at a distance in front of the actual or virtually determined lane marking.

This provides a better overview of the displayed graphic elements (the marking along the lane marking and the respective crossbar), as these graphic elements do not overlap each other.

According to a further advantageous embodiment, the control unit is designed to display a single marking for each road user superimposed on a single actual or virtually determined lane marking on each side of the vehicle. The vehicle sides are a left side and a right side.

Another aspect of the invention relates to a vehicle with a system as described above and below.

Another aspect of the invention relates to a method for visually augmenting a vehicle's surroundings, wherein a sensor unit detects the vehicle's surroundings, at least part of the vehicle's surroundings is visible to the driver of the vehicle on a display unit, and a control unit detects another road user in the sensor unit's data and determines whether the other road user will enter an adjacent lane when the vehicle changes lanes. The system takes into account the distance between the road user and the vehicle and, if this is the case, continuously updates a distance range to the other road user, which indicates that the vehicle should refrain from changing lanes if the distance is within the distance range, and wherein a marker visualizing the distance range is displayed on the display unit, which moves along a corridor that lies analogously to an actual or virtually determined lane marking between the vehicle's lane and the adjacent lane, in conjunction with the movement of the road user.

Advantages and preferred further developments of the proposed procedure result from an analogous and substantive transfer of the above statements made in connection with the proposed system.

Further advantages, features and details will become apparent from the following description, in which - possibly with reference to the drawing - at least one embodiment is described in detail.

• 1: Einen Innenraum eines Fahrzeugs mit einem System für Rückspiegel gemäß einem Ausführungsbeispiel der Erfindung.
• 2: Einen Innenraum eines Fahrzeugs mit einem System für ein digitales Rückspiegelsystem gemäß einem Ausführungsbeispiel der Erfindung.
• 3: Das Fahrzeug der 1 oder 2 in Vogelperspektive.
• 4: Ein Szenario und eine Anzeige des Systems nach 1 oder 2.
• 5: Eine angezeigte Markierung gemäß einem Ausführungsbeispiel der Erfindung.
• 6: Ein Verfahren zum visuellen Augmentieren eines Fahrzeugumfelds gemäß einem Ausführungsbeispiel der Erfindung.

They show:

• 1 : An interior of a vehicle with a system for rearview mirrors according to an embodiment of the invention.
• 2 : An interior of a vehicle with a system for a digital rearview mirror system according to an embodiment of the invention.
• 3 The vehicle of 1 or 2 from a bird's-eye view.
• 4 : A scenario and a display of the system according to 1 or 2 .
• 5 : A displayed marking according to an embodiment of the invention.
• 6 : A method for visually augmenting a vehicle environment according to an embodiment of the invention.

The representations in the figures are schematic and not to scale.

1 shows the interior of a vehicle 1 with a system for visually augmenting the vehicle's surroundings as viewed by the driver of vehicle 1. Not shown in 1 , however, for example in 3 The vehicle 1 has a sensor unit 3 for detecting its surroundings. Data about the vehicle's surroundings are continuously transmitted from the sensor unit 3 to a control unit 7, which analyzes this data. During this data analysis, other road users 9 are continuously detected and identified as such. The control unit 9 then performs an examination to determine whether, in the event of a lane change by the vehicle 1 to an adjacent lane, these other road users 9 must be taken into account by maintaining a predetermined distance. This check can be performed conservatively, meaning that it is also carried out for other road users 9 who are very far away, but not for other road users 9 who are, for example, traveling on different roads. In particular, this check is always and only confirmed if the other road user 9 is located in an adjacent lane to which the vehicle 1 could potentially change lanes. Then, the control unit 7 calculates a distance range to the other road user 9, which indicates that the lane change by vehicle 1 should be avoided if the distance between vehicle 1 and the other road user 9 is within this distance range. In the example of the 1 The driver of vehicle 1 views the area behind and partly to the side of the vehicle 1 using side mirrors. These form the display unit 5, on which the marker 11, which visualizes the distance range, is superimposed on the actual mirror image. The display of marker 11 is further refined using the 4 described.

2 shows analogous to 1 The interior of a vehicle 1 is shown from the perspective of the driver of the vehicle 1, but with a design variation: Instead of side mirrors, the display unit 5 is formed by a camera-monitor system. Rear-view cameras are used instead of the side mirrors, and their images are displayed on monitors of the display unit 5 in the interior of the vehicle 1. These monitors are located on the left and right sides of the interior of the vehicle 1. Marking 11 (see 4 and 5 For example markings 11), the image is superimposed on the monitors of the display unit 5 in the image of the environment of the vehicle 1.

3 shows a vehicle 1, optionally the one of the 1 or the 2 . In the 3 An exemplary sensor unit 3 is shown to supply data about the vehicle 1's environment to the vehicle 1's control unit 7. Since this is a modern vehicle with a range of driver assistance and automation systems, vehicle 1 is equipped with highly accurate environmental sensors. The sensor unit 3 comprises a multitude of individual cameras distributed along the circumference of one exterior surface of vehicle 1. Systems, radar systems, and/or lidar systems, which allow the control unit 7 to continuously determine the respective distance and speed of each other road user 9 to the vehicle 1. Based on the respective distance and relative speed of each other road user 9, the control unit 7 can continuously determine a distance range assigned to each other road user 9 and accordingly display the marker 11 on a display unit 5, however it may be configured.

4 shows an exemplary view of a driver in a real rearview mirror (variant of display unit 5 according to 1 ) or optionally into a digital side mirror (variant of display unit 5 according to 2 When the driver of vehicle 1 looks in the side mirror or at a monitor of a camera-monitor system equivalent to a side mirror, they see, on the one hand, the other road user 9, on the other hand, areas of their own vehicle 1, and, on the other hand, a lane marking between their own lane and the adjacent lane in which the other road user 9 is located. While this lane marking is generally displayed in green on the display unit 5, the green color is replaced by a red color in the distance area; this is marking 11.

5 The diagram schematically shows, from a bird's-eye view, the area of marking 11 relative to an example position of vehicle 1 and an example position of the other road user 9. The distance zone, visualized by marking 11, is calculated such that a lane change can only be carried out safely if vehicle 1 is not at the level of marking 11. In the example of the 5 If this is not the case, the driver of vehicle 1 can still change lanes as long as road user 9 does not get any closer to vehicle 11.

6 Figure 1 shows a method for visually augmenting a vehicle's surroundings, wherein a sensor unit 3 detects a vehicle's surroundings S1, at least part of the vehicle's surroundings is visible to the driver of vehicle 1 on a display unit 5, and a control unit 7 detects another road user 9 in data from the sensor unit 3 S2 and determines S3 whether the other road user 9 must be taken into account when vehicle 1 changes lanes to an adjacent lane with regard to the distance between the road user 9 and vehicle 1, and if so, continuously updated, a distance range to the other road user 9 is calculated S4, which indicates that the lane change by vehicle 1 should be avoided if the distance is within the distance range, and wherein a marking 11 visualizing the distance range is displayed on the display unit 5 S5, which lies along a corridor that is contact-analogous to an actual or virtually determined lane marking between the lane of vehicle 1 and the adjacent lane. is moved along with a movement of road user 9.

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• DE 10 2010 013 398 A1 [0004]
• WO 2017/162278 A1 [0005]

Claims (10)

System for a vehicle (1) for visually augmenting a vehicle's surroundings, comprising a sensor unit (3) for detecting the vehicle's surroundings, a display unit (5), and a control unit (7) configured to detect another road user (9) in the data from the sensor unit (3) and to determine whether the other road user (9) must be taken into account when the vehicle (1) changes lanes to an adjacent lane with respect to a distance between the road user (9) and the vehicle (1), and if so, to continuously calculate a distance range to the other road user (9) that indicates that the lane change of the vehicle (1) should be avoided if the distance is within the distance range, and to display on the display unit (5) a marking (11) visualizing the distance range, which is located along a corridor that is contact-analogous to an actual or virtually determined lane marking between the lane of the vehicle (1) and the adjacent lane, with a movement of the other is moved along with road user (9). System according to Claim 1 , wherein the marking (11) extends elongatedly along the corridor over a length of the distance range and is colored over this length. System according to Claim 2 , wherein the marking (11) transitions to a different color in the corridor at its front and rear ends with a smooth transition. System according to one of the preceding claims, wherein the display unit (5) comprises a screen, wherein the control unit (7) is configured to display an image of the vehicle environment obtained from the data of the sensor unit (3) on the screen, wherein the image includes at least part of the adjacent lane. System according to Claim 4 , where the screen is part of a camera-monitor system that replaces a side mirror. System according to one of the preceding claims, wherein the control unit (7) is configured to display the marking (11) only when an adjacent lane and/or an actual lane marking is detected in the data of the sensor unit (3) by the control unit (7) which meets a predetermined condition. System according to one of the preceding claims, wherein the control unit (7) is configured to display a respective crossbar directed towards the actual or virtually determined lane marking, analogous to contact with a front end of the road user (9) and analogous to contact with a rear end of the vehicle (1). System according to Claim 7 , where the crossbar ends at a distance in front of the actual or virtually determined lane marking. System according to one of the preceding claims, wherein the control unit (7) is configured to display a respective marking (11) for a respective road user (9) superimposed on only a single actual or virtually determined lane marking on each side of the vehicle. A method for visually augmenting a vehicle's surroundings, wherein a sensor unit (3) detects a vehicle's surroundings (S1), at least part of the vehicle's surroundings is visible to a driver of the vehicle (1) on a display unit (5), and a control unit (7) detects another road user (9) in data from the sensor unit (3) (S2) and determines (S3) whether the other road user (9) must be taken into account when the vehicle (1) changes lanes to an adjacent lane with regard to a distance between the road user (9) and the vehicle (1), and if so, continuously calculates a distance range to the other road user (9) (S4) which indicates that the lane change by the vehicle (1) should be avoided if the distance is within the distance range, and wherein a marker (11) visualizing the distance range is displayed on the display unit (5) (S5), which runs along a corridor that is contact-analogous to an actual or virtually determined The lane marking lies between the lane of the vehicle (1) and the adjacent lane and is moved along with a movement of the road user (9).