DE102021200027B4 - Method for operating an assistance system of a motor vehicle, computer program product and assistance system - Google Patents

Abstract

Method for operating an assistance system (2) of a motor vehicle (1), in which an environment (9) of the motor vehicle (1) is detected by means of at least one detection device (3) of the assistance system (2) and, depending on the detected environment (9), an environment model (10) is generated by means of an electronic computing device (8) of the assistance system (2), wherein the environment model (10) is displayed on a display device (11) of the assistance system (2), wherein swarm data (5) relating to a plurality of further motor vehicles (6) is received by means of a swarm data receiving device (4) of the assistance system (2) and at least one item of environment information (12, 13, 14, 15) is generated on the basis of the swarm data (5), and the environment information (12, 13, 14, 15) is taken into account when generating the environment model (10) and is displayed on the display device (11), characterized in that trajectories of the plurality of motor vehicles (6) are used as Swarm data (5) are received and a swarm trajectory (12) is generated and displayed as environmental information (12, 13, 14, 15), wherein a single swarm trajectory (12) is generated and displayed from the plurality of trajectories.

Description

The invention relates to a method for operating an assistance system of a motor vehicle, in which an environment of the motor vehicle is detected by means of at least one detection device of the assistance system, and an environment model is generated by means of an electronic computing device of the assistance system as a function of the detected environment, wherein the environment model is displayed on a display device of the assistance system for a user of the motor vehicle. Furthermore, the invention relates to a computer program product and an assistance system.

It is already known from the state of the art that an environment model can display the detected or recorded environment to a motor vehicle user. For example, the tracks detected by a camera and detected objects, such as other motor vehicles or motorcycles, are displayed.

The DE 10 2018 128 315 A1 discloses a method for checking a first adaptive system model, wherein the first adaptive system model is stored in a first vehicle and corresponds to a system of the first motor vehicle, wherein the first vehicle and a plurality of second vehicles are representative of vehicles of a predetermined vehicle class of a vehicle fleet. A first characteristic of an adapted model parameter of the first adaptive system model is provided. For at least a subset of the plurality of second vehicles, a respective second characteristic of the adapted model parameter of a second adaptive system model stored in the respective second vehicle is provided for each of the second vehicles of the subset. Comparison data is determined and stored depending on the first characteristic and the second characteristic values.

The DE 10 2013 205 392 A1 discloses a backend device for providing information for driver assistance functions of vehicles, comprising a plurality of application units, a database, and an interface unit. The application units are each configured to transmit information for at least one driver assistance function of the vehicle and to provide it to the vehicles. The database stores information of an environment model assigned to the interface unit of the database and is configured to enable access to the information stored in the database, wherein the application units are each connected to the interface unit in order to retrieve information stored in the database via the interface unit.

The DE 10 2018 200 134 B3 relates to a method for acquiring training data for a driver assistance system for a motor vehicle. A criterion is specified by a server device and transmitted to the motor vehicles. If the criterion is met, the vehicles then transmit individual images of the surroundings of the respective motor vehicle captured with a respective on-board camera to the server device. The server device then generates a common image series for training the driver assistance system from various individual images transmitted by the motor vehicles.

The DE 10 2018 203 583 A1 relates to a method for predicting a position or a trajectory of at least one object in an environment of a motor vehicle, wherein the at least one object in the environment of the motor vehicle is detected by means of a detection device of the motor vehicle and the position or the trajectory of the at least one object is predicted by means of a neural network, wherein the environment is modeled to form a graph-based environment model with at least one graph edge and at least one graph node and the position or the trajectory is predicted by means of the neural network with which the graph-based environment model is implemented, depending on the at least one graph edge and/or the at least one graph node.

The DE 10 2015 225 094 A1 relates to a method for providing an information signal representing a traffic situation of a vehicle. The method comprises a step of processing sensor data acquired by at least one sensor device assigned to the vehicle, which represents at least a partial aspect of the traffic situation of the vehicle, in order to generate the information signal. The method also comprises a step of outputting the information signal to an interface to at least one vehicle-external device.

According to the republished DE 10 2019 217 429 A1 When a large number of vehicles travel along the same section of road, parameters of the respective vehicle and/or the respective vehicle environment and/or time-related parameters are recorded. Metadata for supporting longitudinal control when traveling along the section of road traveled is generated by statistically evaluating the parameters recorded by the large number of vehicles. The generated metadata is sent to a vehicle to support longitudinal control. sent. The vehicle receives the metadata and uses it to support the vehicle's longitudinal control when traveling along the same section of road. The section of road may include a curve, and the metadata is used to determine the curve, in particular the curve radius, and/or a recommended speed for the vehicle when traveling through the curve.

The object of the present invention is to provide a method, a computer program product and an assistance system by means of which a user of the motor vehicle can be supported in driving the vehicle in an improved manner.

This object is achieved by a method, a computer program product, and an assistance system according to the independent patent claims. Advantageous embodiments are specified in the subclaims.

One aspect of the invention relates to a method for operating an assistance system of a motor vehicle, in which an environment of the motor vehicle is detected by means of at least one detection device of the assistance system and, depending on the detected environment, an environment model is generated by means of an electronic computing device of the assistance system, wherein the environment model is displayed on a display device of the assistance system.

It is provided that a swarm data receiving device of an assistance system receives swarm data generated based on data from a plurality of other motor vehicles. Based on the received swarm data, at least one piece of environmental information is generated, and the environmental information is taken into account when generating the environmental model and displayed on the display device.

This allows the swarm data to be taken into account and displayed to the vehicle user using the environment model. One advantage is that the user can access an additional sensor information source, thus achieving an improved representation of the environment, thus supporting the vehicle user in their driving task.

A camera, a lidar sensor, an ultrasonic sensor, or a radar sensor is suitable as a detection device, for example. The display device can be considered, for example, a head-up display, which can also be referred to as a head-up display, or preferably as a display device of a head unit. The environment model is, in particular, a digital environment model. The environment information is then displayed on the display device in addition to the vehicle's own sensor data.

Trajectories from the multitude of motor vehicles are received as swarm data, and a swarm trajectory is generated and displayed as environmental information. In particular, in addition to the lanes, the user is also shown the path traveled from the swarm data in the environmental model. This can help increase confidence in the function, for example, in situations where the motor vehicle is driving through an intersection. In particular, a single swarm trajectory is generated and displayed from the multitude of trajectories. In other words, a cumulative evaluation of the trajectories to form the swarm trajectory takes place.

According to an advantageous embodiment, the environmental information is generated internally within the vehicle by the electronic computing device. In other words, the electronic computing device receives the swarm data from the swarm data receiving device and evaluates it internally within the vehicle. Alternatively, the evaluation can also be performed externally, for example, by a central electronic computing device external to the vehicle. In particular, the evaluation can achieve an improved representation of the environmental information on the display device.

It is further advantageous if the swarm data is received by a central electronic computing device external to the vehicle and/or the swarm data is received via vehicle-to-vehicle communication, also known as car-to-car communication. In particular, the swarm data can also be received via vehicle-to-X communication, also referred to as car-to-X or vehicle-to-x (C2X, V2X). In particular, this is a so-called near-field communication, on the basis of which corresponding swarm data can be made available. In particular, the swarm data can thus be received in a highly up-to-date manner from the immediate surroundings of the vehicle.

Alternatively or additionally, the swarm data can also be received by the vehicle's external, central electronic computing device. For this purpose, the assistance system is equipped with a swarm data receiving device. which is particularly designed as a radio communication device that allows wireless communication with the electronic computing device, particularly via the Internet. This also makes it possible to access swarm data from a large number of vehicles based on past detection events. For this purpose, the large number of vehicles send data captured by the detection devices, particularly cameras, to the electronic computing device. This allows the environment model to be generated in its entirety.

In particular, it is provided that the motor vehicle itself transmits information recorded by the recording device to the electronic computing device and/or via motor vehicle-to-motor vehicle communication or motor vehicle-to-X communication to one or more other vehicles or receivers located in the vicinity of the motor vehicle.

It is further advantageous if additional speeds from the plurality of motor vehicles are received as swarm data, and a swarm speed is additionally generated and displayed as environmental information. In particular, only a single swarm speed is generated and displayed. Thus, the swarm speed is cumulatively generated from the plurality of speeds.

It is also advantageous if the current speed of the motor vehicle is compared with the speed of the swarm and the result of the comparison is also shown on the display device. In particular, the speed of the swarm, i.e. the multitude of motor vehicles, is then known. This can then be compared with the user's own speed. If, for example, one of the user's own vehicles is traveling faster than the swarm at a certain point, this can optionally be indicated, for example, using a color. Furthermore, if one is traveling slower than the swarm, this can also be indicated. This can support the user of the motor vehicle in their driving task.

It is also advantageous if a future curve is transmitted as swarm data to the swarm data receiving device, and a future curve trajectory is generated as environmental information depending on the curve of the motor vehicle and displayed on the display device. In particular, this embodiment solves the problem that lane paths often lose quality when the road curves. In particular, with double curves, a second curved segment is very poorly detected and displayed with the known detection devices. Furthermore, a first curved segment is also very poorly displayed at low speeds; for example, below 45 km/h only a straight lane is displayed. With the help of the swarm data, which is based on the detection of many motor vehicles in the same situation, better and more accurate information is now available. In particular, the swarm data can be used to refine the environmental model to such an extent that even two curvatures in a curve can be reliably and stably displayed.

According to a further advantageous embodiment, a lane boundary is transmitted as swarm data to the swarm data receiving device and generated as environmental information and displayed on the display device. In particular, according to the current state of the art, when detecting a structural separation, such as curbs, the quality of the detection is not very high enough to be able to reliably determine the height of, for example, this structural separation or curbs. Using the swarm data, the environmental model can be expanded with the additional information and evaluated, for example, based on interpolation or with a filter. In particular, this can prevent sudden changes in height, especially frequent or repeated ones.

The method presented is, in particular, a computer-implemented method. For this purpose, a further aspect of the invention provides for a computer program product and program code means which, when the program code means is processed, cause an electronic computing device to carry out a method according to the preceding aspect. For this purpose, the invention further relates to a computer-readable storage medium with the computer program product. For this purpose, the electronic computing device has, in particular, circuits, in particular integrated circuits, and processors as well as further electronic components which are required for processing the program code means and for carrying out the method.

A further aspect of the invention relates to an assistance system for a motor vehicle, comprising at least one detection device, a swarm data receiving device, a display device and an electronic computing device, wherein the assistance system is designed to carry out a method according to the preceding aspect. In particular the procedure is carried out using the assistance system.

Yet another aspect of the invention relates to a motor vehicle with an assistance system according to the preceding motor vehicle. The motor vehicle can be configured to be at least partially assisted.

Advantageous embodiments of the method are considered to be advantageous embodiments of the computer program product, the assistance system, and the motor vehicle. The assistance system and the motor vehicle have specific features that enable the method to be carried out.

The invention also includes further developments of the assistance system according to the invention that have features already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments of the assistance system according to the invention are not described again here.

The invention also includes the combination of the features of the described embodiments.

• 1 eine schematische Draufsicht auf eine Ausführungsform eines Kraftfahrzeugs mit einer Ausführungsform eines Assistenzsystems;
• 2 eine schematische Draufsicht auf eine Ausführungsform einer Anzeigeeinrichtung eines Assistenzsystems;
• 3 ein schematisches Blockschaltbild einer Ausführungsform eines Kraftfahrzeugs mit einer Ausführungsform eines Assistenzsystems.

Exemplary embodiments of the invention are described below. Shown are:

• 1 a schematic plan view of an embodiment of a motor vehicle with an embodiment of an assistance system;
• 2 a schematic plan view of an embodiment of a display device of an assistance system;
• 3 a schematic block diagram of an embodiment of a motor vehicle with an embodiment of an assistance system.

The exemplary embodiments explained below are preferred exemplary embodiments of the invention. In the exemplary embodiments, the described components each represent individual, independently considered features of the invention, which also further develop the invention independently of one another and are thus also to be considered as components of the invention, either individually or in a combination other than that shown. Furthermore, the described exemplary embodiments can also be supplemented by further features of the invention already described.

In the figures, functionally identical elements are provided with the same reference numerals.

1 shows a schematic plan view of an embodiment of a motor vehicle 1 with an embodiment of an assistance system 2. The assistance system 2 has at least one detection device 3. The detection device 3 can be designed, for example, as a camera, a lidar sensor device, an ultrasonic sensor device and/or a radar sensor device. Furthermore, the assistance system 2 has a swarm data receiving device 4, which is designed to receive swarm data 5. In the present case, it can be provided in particular that the swarm data 5 can be received by another motor vehicle 6. For example, a corresponding transmission of the swarm data 5 can then be carried out by means of vehicle-to-vehicle communication. Furthermore, the swarm data 5 can also be received by a central electronic computing device 7 external to the motor vehicle. Furthermore, it is provided that the assistance system 2 has an electronic computing device 8. The electronic computing device 8 has, in particular, circuits, for example integrated circuits, processors and other electronic components in order to be able to execute program instructions.

According to one aspect of the invention, for operating the assistance system 2, it is provided that an environment 9 of the motor vehicle 1 is detected by means of the detection device 3 and, depending on the detected environment 9, an environment model 10 ( 2 ) is generated by means of the electronic computing device 8, wherein the environment model 10 is displayed on a display device 11 ( 2 ) of the assistance system 2 is displayed for a user of the motor vehicle 1.

It is provided that the swarm data 5 relating to a plurality of further motor vehicles 6 are received by means of the swarm data receiving device 4 and, on the basis of the swarm data 5, at least one item of environmental information 12 ( 2 ), 13 ( 3 ), 14 ( 2 ), 15 ( 2 ) is generated and the environmental information 12, 13, 14, 15 are taken into account when generating the environmental model 10 and are displayed on the display device 11.

In particular, it is provided that the environmental information 12, 13, 14, 15 is generated internally in the vehicle by the electronic computing device 4. Alternatively or additionally, the environmental information 12, 13, 14, 15 can also be generated by the central electronic Computing device 7. Furthermore, it is provided that the environmental information 12, 13, 14, 15 is generated internally in the motor vehicle by the electronic computing device 4 with the aid of swarm data 5 received from the central electronic computing device 7 external to the motor vehicle.

2 shows a schematic perspective view of the display device 11 of the assistance system 2. In the present case, the environmental model 10 is shown on the display device 11. In particular, it is shown that trajectories from the plurality of motor vehicles 6 are received as swarm data 5 and a swarm trajectory 12 is generated and displayed as environmental information 12, 13, 14, 15. In addition, it can be provided that speeds from the plurality of motor vehicles 6 are received as swarm data 5 and a swarm speed 14 is additionally generated and displayed as environmental information 12, 13, 14, 15. The current speed of the motor vehicle 1 can, for example, be compared with the swarm speed 14 and a result of the comparison can additionally be displayed on the display device 11. For example, the displayed swarm trajectory 12 can be changed in color if there is a deviation in the speed.

In particular, when evaluating the environment 9, a traveled path of the swarm data 5, including its speed and, for example, traffic light positions, can be evaluated. The traveled path, in particular the swarm trajectory 12, can then be displayed. Furthermore, the swarm trajectory 12 can optionally be colored if, for example, the motor vehicle 1 traveled faster or slower than the average time. Furthermore, a traffic light and other traffic information can also be displayed using the environment model 10.

Furthermore, the 2 that a lane boundary 15 is transmitted as swarm data 5 to the swarm data receiving device 4 and is generated as environmental information 12, 13, 14, 15 and displayed on the display device 11. In the present case, the lane boundary is indicated, for example, by curbs or kerbs. These curbs can be recognized by the detection device 3 because they can be compared with a digital map. The curbs are then displayed, for example, on the basis of height interpolation or on the basis of a filter in order to prevent sudden changes in height. If necessary, a mapping table with fixedly defined heights can also be displayed at a connection between them. The lane boundary 15 is then likewise displayed in the environmental model 10. In particular, a uniform representation for the lane boundary 15 can thus be realized.

3 shows a schematic plan view of an embodiment of the motor vehicle 1. In the present case, in particular, a future curve trajectory 13 can be represented. In particular, a future curve course 16 has thus been transmitted as swarm data 5 to the swarm data receiving device 4, and the future curve trajectory 13 can be generated as environmental information 12, 13, 14, 15 depending on the curve course 16 of the motor vehicle 1 and displayed on the display device 11. In particular, this is due to the fact that the detection device 3, should it be designed as a camera, for example, often cannot detect two curves, for example 100 meters in front of the motor vehicle 1. The first curve begins, for example, below the motor vehicle 1, and the second curve begins, if present, within the viewed 100 meters, whereby the second curve segment is displayed in very poor quality because it is too noisy to be used as a meaningful display. In particular, even at low speeds, for example less than 45 km/h, the first segment may be qualitatively very poorly recognizable, so that in the environment model 10 no curvature is displayed at low speeds.

Using the swarm data 5, the environmental model 10 is now expanded to reliably represent the two curvatures of the curve 16. In particular, the future curve trajectory 13 is then also displayed as a displayed curvature on the display device 11. For this purpose, for example, the coordinates of the lines of the other motor vehicles 6 as well as the coordinates of the possible route can be used. The curvature is generated, for example, as a polygon or clothoid. The data is then merged with the acquired information by the acquisition device 3. This is then calculated in the environmental model 10 and displayed on the display device 11.

List of reference symbols

1

motor vehicle

2

Assistance system

3

Recording device

4

Swarm data receiving device

5

Swarm data

6

Variety of motor vehicles

7

vehicle-external, central electronic computing device

8

electronic computing device

9

Vicinity

10

Environment model

11

Display device

12

Swarm trajectory

13

future curve trajectory

14

Swarm speed

15

Road markings

16

Curve progression

Claims (9)

Method for operating an assistance system (2) of a motor vehicle (1), in which an environment (9) of the motor vehicle (1) is detected by means of at least one detection device (3) of the assistance system (2) and, depending on the detected environment (9), an environment model (10) is generated by means of an electronic computing device (8) of the assistance system (2), wherein the environment model (10) is displayed on a display device (11) of the assistance system (2), wherein swarm data (5) relating to a plurality of further motor vehicles (6) is received by means of a swarm data receiving device (4) of the assistance system (2) and at least one item of environment information (12, 13, 14, 15) is generated on the basis of the swarm data (5), and the environment information (12, 13, 14, 15) is taken into account when generating the environment model (10) and is displayed on the display device (11), characterized in that trajectories of the plurality of motor vehicles (6) are used as Swarm data (5) are received and a swarm trajectory (12) is generated and displayed as environmental information (12, 13, 14, 15), wherein a single swarm trajectory (12) is generated and displayed from the plurality of trajectories. Procedure according to Claim 1 , characterized in that the environmental information (12, 13, 14, 15) is generated internally in the motor vehicle by the electronic computing device (3). Procedure according to Claim 1 or 2 , characterized in that the swarm data (5) are received by a central electronic computing device (7) external to the motor vehicle and/or the swarm data (5) are received by means of motor vehicle-to-motor vehicle communication. Method according to one of the preceding claims, characterized in that additional speeds from the plurality of motor vehicles (6) are received as swarm data (5) and a swarm speed (14) is additionally generated and displayed as environmental information (12, 13, 14, 15). Procedure according to Claim 4 , characterized in that a current speed of the motor vehicle (1) is compared with the swarm speed (14) and a result of the comparison is additionally displayed on the display device (11). Method according to one of the preceding claims, characterized in that a future curve profile (16) is transmitted as swarm data (5) to the swarm data receiving device (4) and a future curve trajectory (13) is generated as environmental information (12, 13, 14, 15) as a function of the curve profile (16) and is displayed on the display device (11). Method according to one of the preceding claims, characterized in that a lane boundary (15) is transmitted as swarm data (5) to the swarm data receiving device (4) and is generated as environmental information (12, 13, 14, 15) and displayed on the display device (11). Computer program product with program code means which cause an electronic computing device (8) when executing the program code means to carry out a method according to one of the Claims 1 until 7 to carry out. Assistance system (2) for a motor vehicle (1), with at least one detection device (3), with a swarm data receiving device (4) with a display device (11) and with an electronic computing device (8), wherein the assistance system (2) is designed to carry out a method according to one of the Claims 1 until 7 is trained.