EP1327968A2 - Vehicle with image recognition means - Google Patents

Abstract

The motor vehicle has an image acquisition arrangement (1) for acquiring images of the road and a processing and evaluation arrangement for determining any traffic signs (4a,4b) visible in the image, their regulatory content and their association with the road and for outputting at least one signal (S) for triggering an action recognizable by the driver depending on the results of the evaluation.

Description

The invention relates to a motor vehicle.

With increasing traffic it happens more often that Motorists wrong in a highway retract, so opposite to the drive the actual direction of the highway. To the driver for the To indicate it intended correct lane, one uses primarily a corresponding signage, about which the driver the permitted lane and the forbidden lane are displayed. However, it often happens that these signs are overlooked or due to the increasing number of relatively adjacent to each other standing signs confused or misaligned what ultimately lead to a choice of the wrong lane.

In order to avoid in particular a wrong driving on highways is it is known to provide devices integrated into the road surface, the at an inadmissible driveway on this road destroying the Tire of the vehicle act. To name here are, for example protruding thorns piercing the tires. This remedy is indeed effective, but carries a considerable risk to the driver, since the Bursting the tire represents a dangerous situation for the motorist possibly uncontrollable. Of course, the destruction of the Tires associated with considerable follow-up costs.

From DE 199 34 774 A1 it is also known about a suitable Detection system, e.g. a GPS (Global Positioning System) or DGPS (Differential Global Positioning System) the actual position of a vehicle too capture and with in a vehicle-mounted storage means in the form of "digital road maps" comparable data to compare This way, a deviation from the permissible target distance to to capture. If such an impermissible deviation is determined, then For example, a warning signal issued or via the engine control unit the vehicle will be switched off. The disadvantage here, however, is that the correct operation of this system inevitably on the accuracy of the actual position detection as well as the accuracy of the stored "digital Road maps "is dependent and thus there is a risk that at Inaccurate position detection or incorrect data, the system is not or recognize the wrong-way drive too late.

The invention is based on the problem to provide a motor vehicle, the a reliable determination of any wrong drive, be it when driving on a highway or, for example, the wrong retraction in one One-way street, captured.

To solve this problem, a motor vehicle according to the invention an image pickup means for taking pictures of the images with the Motor vehicle or road to travel and with Processing and evaluation means for determining a possible in the Picture visible road sign, its contents and its Assignment to the used or driven lane as well as to the Issue at least one recognizable to the driver action triggering signal depending on the determination result are formed, provided.

In the motor vehicle according to the invention are preferably continuously Pictures taken of the used or to be driven lane and evaluated to recognize any signs shown in them. The processing and evaluation are also able to Control content, ie the relevant for the driver or traffic Information to determine and the road sign of a particular lane so as to recognize whether it is for the driver or to be driven Roadway is or not. Depending on how this determination result If necessary, a signal can be output via which then triggers a suitable, recognizable by the driver action is, for example, the impending or already given Indicates dangerous situation. The entire situation recording is done so vehicle internally. The processing and evaluation have for example, via a suitable computing device, the one can perform appropriate image analysis for traffic sign detection. In a memory, for example, all nationwide applicable Road signs with their regulatory content deposited, with which one detected road sign can be compared. It is natural also the option of only a few traffic signs for those with the motor vehicle according to the invention to be detected dangerous situation relevant are to deposit.

According to the invention, the processing and evaluation means for Recognition of a sign prohibiting the entrance and to issue a Signal with recognized assignment of the sign to the traffic lane be educated. After this invention design, the simplest Form of invention design is about the processing and Analysis means analyzed the image to see if in it a "driveway "prohibited" sign is shown next to its assignment to be used or checked roadway. Are both Decision parameter positive, the signal is output, because then there is a risk of incorrect entry. For example, if a "driveway However, it is a parallel lane and not assigned to the traffic, so omits the signal output, since then the There is no danger of wrong-way driving.

Furthermore, the processing and evaluation means for detecting one the beginning of a highway or other roadway with a excellent direction indicating sign and to issue a Signal when detected assignment of the sign to be driven or to driving lane and in the presence of at least one other an evaluation of the or subsequently taken pictures resulting information about the used or to be driven roadway be educated. In this variant of the invention analyze the Processing and evaluation means a recorded image to for example, if a beginning "highway" sign shown and whether this is the traffic or road to be traveled is assigned. Is both the Case, it can be assumed that the driver is the right lane For example, has chosen to drive on the highway. nevertheless Of course, it can still happen that the driver wrongly switched from the right lane to a wrong lane. As a result, it is useful in this invention embodiment, if at least one more resulting from the evaluation Information parameter enters into the decision about the signaling. Because if, for example, as such evaluation-specific Information is determined that the previously correct lane has been changed inadmissibly, the signal must be present despite the presence of original analysis parameters indicating the correct lane be issued.

As such information, for example, the curvature of the curves in the Image or in a roadway shown below be determined and taken into account. The processing and evaluation means are trained accordingly to get out of the captured image suitable data transformation the curve curvature of the traveled Track lane. For example, the curve curvature in shape an optionally signed value, where this part of the processing and evaluation with a corresponding setpoint is compared. The signaling is in this case then dependent on the comparison result. So for example, a "Start highway" sign detected and the busy lane assigned and capture the processing and evaluation that the Road turns into a right turn, as is usually the case Highway entrances is the case, since the driveway then in the threading track of Highway passes, so a signal would not be required. But takes the driver, for example, an accidental lane change on the sometimes parallel to the driveway led exit, so would the Processing and evaluation to this then traveled lane one Determine left turn. These at usual highways driveways usually unrealized curve guidance would in this case be an analysis parameter, which indicates a wrong drive, which leads to the signal.

It is also conceivable that the processing and evaluation means a two parallel lanes bordering and separating Lane marking is recorded and checked, whether these are below is overrun, which would mean an illegal lane change. are ie e.g. the exit and the driveway of a motorway sections in parallel Guided and separated only by a solid line, so can one unlawful driving over and thus a wrong-way driving are detected.

In the event that the road shows a changing curvature, So for example, first a left turn and then a Right-hander describes transition into the Einfädelspur, so it is expedient if the processing and evaluation means for determining a changing curvature of the paved roadway are formed, wherein the output of a signal at a determined Amendment may be omitted. This is the case, for example detects that after determining a "start highway" sign the roadway first describes a left turn, which is actually an indication of a False ride is, however, later in a right turn and then in the Threading trace passes, i. that the driver so on the permitted driveway moves. This situation can according to the embodiment of the invention be detected, here either the signal output is omitted, since the System can not determine the wrong course with final security. Alternatively, there is also the possibility of changing the signal, for example, such that first an alarm signal is output, the then changed to a "release signal". Beside it exists Of course, the possibility of having a lane guidance initially "valid" right turn and then left turn, the following turn into a right turn with transition into the threading track, too to capture.

In particular, in this case, in spite of all security to that effect whether there is a correct or incorrect motorway exit, or to generally improve the safety of the assessment of the situation be provided that the processing and evaluation means for determining a Ausfädelspur leading off a highway and, where appropriate one on a motorway leading Einfädelspur as another road-related information, depending on the or if necessary, a new signal output takes place, are formed. The Processing and output means or their image analysis vornehmendes Means is capable of this invention embodiment, certain Determine lanes, as they are typical of highways. At a Motorway exit branches off from the actual right freeway lane Departure trail, which then merges into the motorway exit. These characteristic tracking can be detected by the image processing means be, for example, by detecting the location of the green area or the guardrail (which must be in the left picture area at the threading track) or by detecting the position of the parallel to Einfädelspur Motorways, etc. The same applies to the recording of Einfädelspur, here the green strip or the guardrail in the right Picture area and the parallel highway lanes in the left Picture area would have to lie. For example, when capturing a "Start Highway" sign, its assignment to the carriageway and following detected multiple curvature changes, the where appropriate, do not allow a final assessment of the situation, finally detected a Einfädelspur, so there is no wrong drive, a signal output can be omitted. However, if a Ausfädelspur detected, so is immediately the signal output and the corresponding alarm are triggered.

Another possibility to support the system is that According to the invention, the processing and evaluation means for determining Target lane data based on image acquisition and comparison the target lane data with actual data representing the current position of the Vehicle, in particular by means of a GP or DGP system recorded data and to output a signal if missing Match the data are formed. Here, therefore, in addition to already described data processing also a consideration of recorded via external detection systems, the actual position of the Vehicle specifying data.

Based on the signal from the processing and evaluation means can be issued, an audible and / or an optical warning signal be issued. For example, a warning sound can be generated or There is a voice output such that the driver on the Hazardous situation, e.g. by an announcement "Attention, it it could be the wrong lane "or something like that conceivable to output optical flashing signals that are as conspicuous and for the driver are recognizable, for example in the form of a red warning light (an optional signal to be subsequently given would then be for example, a green light), of course, too acoustic and optical signals are given simultaneously. Besides there is a possibility that the driver may manually and / or aurally, e.g. by voice input, which can suppress the warning signal, for what corresponding means, e.g. a button or switch or equivalent Voice input option are provided. So for example, a However, the driver is definitely sure that he will give the warning signal On the right lane, he can suppress the warning signal about this.

Finally, there is the possibility that the signal to a the operation of the Vehicle engine control unit is given, the reception of the signal and to vary the engine operation in response to the Signal is formed. This will realize the possibility for example, completely shut off or throttle the engine so that only minimal vehicle movements, e.g. an immediate turn at an actual wrong-way, which the driver despite timely failed to correct or correct the given warning signal, possible are. The way how the vehicle engine at a Signaling is controlled, depending on the particular situation be designed.

Fig. 1

eine Prinzipskizze zur Darstellung der relevanten Komponenten des erfindungsgemäßen Kraftfahrzeugs,

Fig. 2

eine Prinzipskizze zur Darstellung unterschiedlicher Fahrsituationen, und

Fig. 3

ein Flussdiagramm zur Ablaufdarstellung.

Further advantages, features and details of the invention will become apparent from the embodiment described below and with reference to the drawings. Showing:

Fig. 1

a schematic diagram illustrating the relevant components of the motor vehicle according to the invention,

Fig. 2

a schematic diagram showing different driving situations, and

Fig. 3

a flowchart for flow representation.

Fig. 1 shows the relevant components of a not shown in detail Motor vehicle, for the situation analysis of the invention and Evaluation and subsequent possible driver warning are required. About an image pickup means 1 in the form of a camera, for example, in the upper area of the windshield is arranged looking forward, continuous pictures of the road are taken. the Image pickup means 1 are processing and evaluation means 2 downstream, which initially comprise an image processing means 3, in which the images taken with the image pickup means 1 are analyzed. In this case, on the one hand traffic signs 4a, 4b are detected, whether they are in recorded image are present. The image 4a is in shown embodiment to an entry prohibition sign, the shield 4b it is a sign indicating the beginning of a motorway. The Image processing means 3, which have a corresponding computing and Processing device are capable of, from the delivered Image data, the corresponding data indicating a traffic sign analyze and to determine the regulatory content with in one closer storage data stored known comparison data Road signs to compare. That in that case would not be in that shown storage area the data to the shields 4a, 4b deposited with which the analyzed data are compared to a definitive one Shield determination to make.

Furthermore, the image processing means for detecting the curvature of a trained roadway, as indicated by the symbol 5, the one after on the left and a curved lane to the right, is indicated. Therefor can by suitable data transformation a curvature of the Image data analyzed road descriptive value or the like be determined, which then describes the curvature. For example, can one Right curvature a value of "+1" and a left curvature a value of Be assigned "-1".

Furthermore, the image processing means 3 are for analyzing and recognizing a Ausfädelspur and a Einfädelspur trained on a highway, as indicated by the symbols 6. The symbol 6a shows the possibility the detection of a Ausfädelspur, the symbol 6b shows the possibility of Detection of a threading track based on the recorded images.

The information and analysis results are calculated by the Image processing means 3 to an evaluation device 7, which is also part the processing and evaluation means 2 is given, in which a Situation analysis and evaluation of the given determination and evaluation Analysis results done. Results in consideration of the delivered Parameter that a wrong course is present, so is about the Evaluation device 7 outputs a signal to trigger a for the driver recognizable action serves. This can, for example, a audible warning 8 or an optical warning 9 or both be issued together, which the driver unmistakably on the indicate a given or imminent danger situation. Besides there is a possibility that the signal to a control unit 10, the Operation of the motor vehicle engine 11 controls are given. The Control unit can, for example, the engine 11 completely when signaling switch off or throttle, so that only a slow ride with the Possibility of a vehicle application or the like is given.

Fig. 2 shows in the form of a schematic diagram of a typical roadway guide in Area of a motorway exit and exit. Shown is fragmentary one Highway 12, from which a Ausfädelspur 13 in a motorway exit 14 passes, which in turn opens into a federal highway 15 or the like. On the other side leads from the main road 15, a motorway entrance 16 in a threading 17, then retracted on the highway 12 can be. In the example shown, the uphill and downhill are over corresponding structural islands 18 at least partially separated.

Coming from the left, a motor vehicle 19 travels the federal highway 15. Am Motor vehicle 19 is the windshield side, the image pickup means 1, such as it is known from Fig. 1, is provided. It takes the ones lying in front of him Roadway up.

In the first example, the motor vehicle 1 erroneously moves to the left in FIG take the exit 14. This driving situation is indicated by the dashed Arrow I shown. This entrance is forbidden, as the prohibition sign 4a, which in the shown example is on the left lane edge, clearly shows. This Prohibition sign 4a is visible in the captured image and is about the Image processing means 3 detected and used by the motor vehicle 19 Roadway, so assigned to the motorway exit 14, so that the Evaluation 7 immediately detects a wrong-way and the or Signals for issuing the warning signals, etc. outputs.

In the second example, the motor vehicle 19 travels as shown by the dashed line Arrow II indicated correctly in the designated highway entrance 16 a. The beginning of the highway is over here on the right side of the road arranged road sign 4b displayed. This road sign 4b will also recorded and the image processing means 3 detected. Moves the motor vehicle 19 continues and takes the driver Lane, as indicated by the dashed arrow IIa, so describes the driveway 16 is a right turn. These Curve curvature is also detected via the image processing means 3 and together with the detected road sign and its assignment given to the evaluation 7, in the context of the situation analysis and evaluation recognizes that the correct lane is being used. to last safety is as soon as the Einfädelspur 17 in the taken picture is recognizable, this also recognized by the image processing means 3, so that definitely ensures that the correct driveway is chosen.

In the example shown, the lane runs after turning off the Bundesstraße 15 is essentially straightforward and then goes into the Right-hand bend of the motorway exit 16 via. But there is also the Possibility that the lane first describes a slight left turn and then turns into the right-hander. This left turn would over the processing means 3 also recognized and evaluated, with a Left turn but for a highway entrance in and of itself is unusual and may be an indication of a possible wrong-way drive, although before the Shield 4b has been broken. Either can already in this case a signal be issued to give a warning signal, which after detection the next right turn either canceled or by gift a release signal is changed, but at the latest when the Threading track 17 was definitely recognized. There is also the possibility of that To suppress signal upon detection of the slight left turn and first to wait for the further course of the drive until the possibility to detect the Ausfädelspur 13 or the Einfädelspur 17 is given, which then finally allows a situation assessment.

Another exemplary driving situation arises when the according to arrow II bent motorists leaves the motorway exit 16 and how with the Arrow IIb points to exit 14. This describes can be seen a left turn, which is detected by the image processing means 3 becomes. Either a signal output can already be made here which corresponds to the Driver at least the possibility of a wrong drive displays. Does that move? Vehicle then as indicated by the dashed arrow llc further, so the Ausfädelspur 13 is detected in the image from a certain point in time, which is a clear indication that there is a wrong-way drive. No later than now a warning must be given, so that the driver can act immediately, for example, by immediate braking or left-hand approach, then turning, etc. Also, e.g. an emergency stop or the like take place.

The lane detection according to the invention can also be supported in that the evaluation device 7 has actual position data, e.g. above GPS or DGPS are recorded, these being with of the Image processing means 3 given data can be compared and can thus enter into the analysis and evaluation result.

Fig. 3 shows a flowchart for illustrating the test procedure. about the image processing means 3 is an image processing such that First, it checks whether in the picture shown a "forbidden entrance" sign is available. If this is the case, then immediately the output of a visual or audible warning or a suitable one Control intervention in vehicle operation. Will not be "entrance forbidden" sign detected, it is checked if a "start highway" sign in the picture is shown. If this is not the case, the control loop returns and analyzes the next captured image in the manner described.

If a "start highway" sign has been recorded, it will be described below checked whether the road shown in the picture a curvature to the left describes what is unusual on most motorway ramps. Is this not the case, then the control loop also returns to the beginning back. If a lane curvature is detected to the left, then if necessary an advance warning in optical or acoustic form issued to sensitize the driver already now. Subsequently For example, a captured image is analyzed as to whether or not there is a threading trace is shown in the image, which detects the basis of the features described above can be. It should be noted that under the of course, several more below recorded images are analyzed to the progressive Detect movement of the vehicle. If this is the case, then is also directly a visual or audible warning or a Control intervention. If no Ausfädelspur recognized, then the picture analyzes whether a threading track is shown. If this is the case, then the returns Control loop also back to the beginning, since the motor vehicle then correctly ascends the highway. If no threading track is detected, then a timer checked whether or how long the image processing already in With regard to the analysis of the threading or threading track. Represents out that a given limit period is not yet has been exceeded, the control loop returns and detects again, whether a threading or threading track is detected. Has the system but the captured images after initiating the threading / threading trace analysis long enough in this regard the images analyzed, so is of it assume that the originally assumed danger situation either not present at all or the driver the highway entrance has left otherwise or that an original error analysis, the Initialization of the track analysis has existed. In this case, the returns Control loop also returns to its output, i. the analysis mode will be reset.

Claims (11)

Motor vehicle with an image pickup means (1) for receiving Pictures of the motor vehicle (19) traveled or to driving lane and with processing and evaluation means (2), the for determining any visible in the image road sign (4a, 4b), its contents and its assignment to the traveled or to be driven lane and output at least one recognizable to the driver action triggering signal (S) depending on the determination result are formed. Motor vehicle according to claim 1,
characterized in that the processing and evaluation means (2) for detecting a the entrance forbidding plate (4a) and to output a signal (S) upon detection of mapping of the shield (4) are formed to the traveled road. Motor vehicle according to claim 1 or 2,
characterized in that the processing and evaluation means (2) for recognizing the beginning of a highway or other lane with an excellent direction indicating sign (4b) and for outputting a signal (S) when recognized assignment of the sign (4b) to be driven or to be traveled lane and in the presence of at least one further information resulting from an evaluation of or subsequently recorded images on the traveled or to be traveled roadway are formed. Motor vehicle according to one of the preceding claims,
characterized in that the processing and evaluation means (2) for determining the curvature (5) and / or a two lanes delimiting and separating lane marking shown in the image or a subsequently recorded image roadway shown as further in the context of the possible signal output to be considered information are formed. Motor vehicle according to claim 4,
characterized in that processing and evaluation means (2) for determining the curvature (5) in the form of an optionally signed value and for comparing the value with a desired value is formed, the signaling being dependent on the result of the comparison. Motor vehicle according to claim 4 or 5,
characterized in that the processing and evaluation means (2) are adapted to determine a changing curvature of the curved lane (5), wherein the output of a signal in a determined change may be omitted. Motor vehicle according to one of the preceding claims,
characterized in that the processing and evaluation means (2) for determining a Ausfädelspur (6a, 13) leading off a highway and optionally a Einfädelspur leading to a highway (6b, 17) as further road-related information, depending on the or optionally one renewed signal output, are formed. Motor vehicle according to one of the preceding claims,
characterized in that the processing and evaluation means (2) for determining target road data based on the image acquisition and for comparing the target road data with actual data indicating the current position of the vehicle, in particular by means of a GP or DGP -Systems recorded data as well as to output a signal in case of mismatch of the data are formed. Motor vehicle according to one of the preceding claims,
characterized in that on the basis of the signal, an audible and / or visual warning signal (8, 9) can be output. Motor vehicle according to claim 9,
characterized in that means are provided for manually and / or acoustically to be made by the driver suppressing the warning signal. Motor vehicle according to one of the preceding claims,
characterized in that there is provided a controller (10) controlling operation of the vehicle engine (11) adapted to receive the signal and to vary the engine operation in response to the signal.

Images