JP4432730B2 - Road marking detection device for vehicles - Google Patents

Description

  The present invention relates to an apparatus and method for capturing a road environment ahead of a host vehicle with a camera, detecting the road marking in which the captured image is processed and road traffic regulations and instructions are written on a road surface.

  There is known a safe driving support device that captures a character string or a symbol drawn on a road surface and compares a reference character / symbol pattern including a character string such as “stop” stored in advance to detect a road sign ( For example, see Patent Document 1).

Prior art documents related to the invention of this application include the following.
JP 2002-219998 A

However, in the above-described conventional apparatus, road markings are collated in character string units, so a wide range of roads ahead of the vehicle must be imaged and image processing must be performed, and the computational load of image processing is heavy. There is.
In addition, if one character string of a road sign cannot be completely imaged, there is a problem that the matching result with the reference character string pattern is inconsistent and detection reliability is lowered.

In the present application, every time one character or one symbol of a road marking is detected by the road marking detection unit, the image processing area of the road marking detection unit on the captured image of the imaging unit is narrowed down based on the detection result. In the road marking transmission means that includes an area narrowing means and transmits an identification result obtained by identifying the road marking based on the characters or symbols continuously detected by the road sign detection means to the occupant, the area narrowing means moves the vehicle forward. When the captured image is processed to detect a road sign continuously drawn in the traveling direction on the road one character or one symbol at a time, each time one character or one symbol of the road sign is detected, the detection result is displayed. based refine image processing area on the captured image, as well as identify the road sign based on the detected character or symbol in succession, one character of the road sign Other depending on the position of one symbol, and sets the position of the image processing area of the road sign detection means for detecting a character or a symbol of the next road markings.

According to the present invention, it is possible to reduce the image processing load while improving the detection reliability of road markings. The area narrowing means partially detects road markings that are continuously painted on the road one character or one symbol at a time, and the search area (image) for the next road marking based on the detection result (detection position) of one road marking. Since the position and size of the (processing area) are set, the image processing area can be narrowed to the minimum necessary range, and the image processing load can be reduced.

<< First Embodiment of the Invention >>
FIG. 1 shows the configuration of the first embodiment. The scanning laser radar 1 irradiates a laser beam in front of the vehicle to receive reflected light from the object, and measures the distance and direction to the object. The radar processing device 2 extracts obstacles ahead of the vehicle based on the measurement result of the laser radar 1, and the position and size of the obstacles on the two-dimensional coordinates in the vehicle longitudinal direction and the vehicle width direction with the own vehicle as the origin ( Width). The progressive scan CCD camera 3 images the front of the vehicle at high speed. The image processing device 4 processes the image captured by the camera 3 and detects a road marking in which road traffic regulations and instructions are written on the road surface.

  The external environment recognition device 5 is based on the vehicle detection results of the radar processing device 2 and the image processing device 4, the vehicle speed detected by the vehicle speed detection device 6, the steering angle detected by the steering angle detection device 7, and the like. The vehicle is detected, it is determined whether or not the vehicle is an obstacle, and the automatic brake control device 8 is controlled as necessary to drive the negative pressure brake booster 9 to generate the braking force of the vehicle. The outside recognition device 5 calculates the braking force based on the distance to the obstacle detected by the radar processing device 2, and calculates the braking force based on the distance to the stop line detected by the image processing device 4. The larger one is output to the automatic brake control device 8 as a braking force command value, and the braking force command voltage is applied to the solenoid valve of the negative pressure brake booster 9 to brake the host vehicle.

  The radar processing device 2, the image processing device 4, and the external environment recognition device 5 are each provided with a microcomputer and its peripheral components, various actuator drive circuits, and the like, and exchange information with each other via a communication line.

  FIG. 2 shows a process of processing a captured image by the camera 3 to detect a paused road sign “stop” painted on the road. The external environment recognition device 5 repeats the road marking detection process shown in FIG. 2 every 50 msec. In step 100, an image captured by the camera 3 at this sampling and processed by the image processing apparatus 4 is read. In the subsequent step 101, the vehicle speed is read from the vehicle speed detection device 6 and the steering angle is read from the steering angle detection device 7. In step 102, it is confirmed whether or not the counter DetectsStep representing the number of times a mark such as a road marking character or symbol has been recognized is 0, that is, no road marking mark has been recognized yet.

If no mark such as a road marking character or symbol has been recognized yet, the process proceeds to step 103, and initial conditions for detecting the road marking are set as follows.
obj_Y = 20,
obj_X = 0,
ProcessWide = 1.5,
ProcessDist = 10.0
Here, obj_Y is the position in the front-rear direction (unit meter) of the image processing area on the road, and obj_X is the position in the left-right direction of the image processing area on the road (unit meter, the front of the host vehicle is 0). Further, ProcessWide is a half value (unit meter) of the horizontal range of the image processing area, and ProcessDist is a half value (unit meter) of the front-rear direction range of the image processing area.

In step 104, the image processing area of the image captured by the camera 3 is set as follows.
disp_obj_YA = y0 + focusV * CAM_h / (obj_Y + ProcessDist) (1)
disp_obj_YB = y0 + focusV * CAM_h / (obj_Y-ProcessDist) (2)
disp_obj_XL = x0 + focusH * obj_X / obj_Y-focusH * ProcessWide / obj_Y (3)
disp_obj_XR = x0 + focesH * obj_X / obj_Y + focusH * ProcessWide / obj_Y
... (4)
Here, disp_obj_YA is the upper coordinate of the image processing area, disp_obj_YB is the lower coordinate of the image processing area, disp_obj_XL is the left coordinate of the image processing area, and disp_obj_XR is the right coordinate of the image processing area. Further, y0 and x0 are the longitudinal coordinates [pix: number of pixels] of the vanishing point (a parameter determined by the mounting position and orientation of the camera 3 (direction of the photographing lens)). Similarly, focus V and focus H are values [pix] (parameters determined by the angle of view of the camera 3 and the resolution of the light receiving element) obtained by converting the focal length of the camera 3 in the vertical and horizontal directions into pixels. CAM_h is the mounting height (unit: meters) of the camera 3.

  In step 105, pre-processing for identifying marks such as road marking characters and symbols in the image processing area is performed. As this processing method, in this embodiment, differentiation and binarization processing are performed. The differential operation is performed by a sobel filter which is a general method for obtaining a differential image. The sobel filter is an operation for obtaining a luminance change between pixels adjacent to a certain pixel. In the binarization process, the average value Av and the variance value Vr are obtained for the luminance of the obtained differential image, and the binarization process is performed using these values as threshold values. Note that binarization is black when the luminance of each pixel (in this case, the luminance change for a differential image) is equal to or less than a threshold value, white when the luminance exceeds the threshold value, and white and shade of the differential image. And black.

In step 106, a reference mark is determined according to the value of the recognition count counter DetectStep of the road marking mark.
If DetectStep = 0, Ref_Mark = “Re” ... still not detected,
When DetectStep = 1, Ref_Mark = “MA” ... “RE” is detected,
When DetectStep = 2, Ref_Mark = “Stop” ... “Ma” is detected,
When DetectStep = 3, Ref_Mark = “−”... All are detected and a stop line is searched.
... (5)
Here, Ref_Mark is a mark that is referred to in order to collate with the detected road marking mark, and is a reference with a sense of perspective as viewed from diagonally above in the same manner as when looking down the road marking mark on the road surface from the camera 3. It is assumed that a mark for use is set in advance.

  In step 107, the road marking mark identified by the pre-processing described above is compared with the reference mark to recognize the road marking mark. As this recognition method, in this embodiment, an OCR (Optical Character Recognition) method between a reference mark and a binarized image is applied, and the road marking mark can be recognized when there is a predetermined similarity or more. And If the road marking mark is recognized in the subsequent step 108, the process proceeds to step 109. If the road marking mark is not recognized, the process proceeds to step 112.

If the road marking mark is recognized, the road marking mark recognition number counter DecetStep is incremented in step 109. If the counter DetectStep is 4, the process proceeds to step 113. If it is less than 4, the process proceeds to step 110. In step 110, based on the recognized position of the road marking mark on the screen, the position of the image processing area on the screen for searching for the road marking mark next is obtained by the following equation.
obj_Y = focusV * CAM_h / (Ref_Mark_Center_y−y0) +2.0,
obj_X = obj_Y (Ref_Mark_Center_x−x0) / focusH (6)
In Expression (6), Ref_Mark_Center_y is the vertical (up and down) position of the recognized mark on the screen, and Ref_Mark_Center_x is the horizontal (left and right) position of the recognized mark on the screen.

In step 111, the size of the image processing area to be searched for the next road marking mark is obtained by the following equation based on the recognized size of the road marking mark on the screen.
ProcessWide = (Rcg_Mark_SizeH * obj_Y / focusH) / 2
ProcessDist = (Rcg_Mark_SizeV * obj_Y / focusV) / 2 (7)
In equation (7), Rcg_Mark_SizeH and Rcg_Mark_SizeV are the horizontal size (unit is the number of pixels) and vertical size (unit is the number of pixels) of the recognized road marking mark.

  On the other hand, if the road marking mark recognition frequency counter DetectStep is 4, the process proceeds to step 113, and a pause mark “-” in front of the host vehicle is detected. At the same time, a warning is displayed on a display (not shown), and temporary stop information is provided to the driver. In step 112, the past values of the vehicle behavior data such as the vehicle speed and the steering angle are updated, and the process is terminated.

  In this way, a road sign continuously painted in the traveling direction on the road is partially detected one character or one symbol at a time, and the search area of the next road sign (image processing is determined based on the detection result of one road sign. Since the position and size of the (region) are set, the image processing region can be narrowed to the minimum necessary range, and the image processing load can be reduced.

<< Second Embodiment of the Invention >>
In the first embodiment described above, an example is shown in which road markings are partially detected in order of “re” → “ma” → “stop” character by character, but one character or one symbol at a time regardless of the character order. A second embodiment for partially detecting road marking will be described. The configuration of the second embodiment is the same as the configuration shown in FIG.

3 to 4 show processing for detecting a temporarily stopped road sign “stop” painted on the road by processing the image captured by the camera 3. The external environment recognition device 5 repeats the road marking detection process shown in FIGS. 3 to 4 every 50 msec. In step 200, an image captured by the camera 3 at this sampling and processed by the image processing device 4 is read. In the next step 201, the vehicle speed is read from the vehicle speed detection device 6 and the steering angle is read from the steering angle detection device 7. In step 202, the value of the counter DetectsStep representing the number of times a mark such as a character or symbol on the road marking has been recognized is confirmed, and branching is made as follows according to the value of the counter DetectsStep.
If DetectsStep = 0 (the state in which nothing is detected), go to step 203.
If DetectsStep = 1 (only “re” is detected), go to step 205.
If DetectsStep = 2 (only “ma” is detected), go to step 207.
If DetectsStep = 3 (detects “re” and “ma”), go to step 209.
If DetectsStep = 4 (only “stop” is detected), go to step 211.
If DetectsStep = 5 (“Re” and “Stop” are detected), go to Step 213.
If DetectsStep = 6 ("Ma" and "Stop" are detected), go to step 215.
If DetectsStep = 7 (“re”, “ma”, “stop” is detected), go to step 217.

When the counter DetectsStep = 0 and no mark such as a character or symbol of the road marking has been detected yet, the parameter for detecting the road marking is set as follows in step 203.
obj_Y1 = 20,
obj_X1 = 0,
ProcessWide = 1.5,
ProcessDist = 10.0,
If RcgState> 2 or RcgState <0, RcgState = 0
... (8)
Here, obj_Y1 is the position in the front-rear direction (unit meter) of the image processing area on the road, and obj_X1 is the position in the left-right direction of the image processing area on the road (unit meter, the front of the host vehicle is 0). Further, ProcessWide is a half value (unit meter) of the horizontal range of the image processing area, and ProcessDist is a half value (unit meter) of the front-rear direction range of the image processing area. RcgState is a variable that determines the type of road marking mark recognized by the image processing in the current sampling cycle, and the initial value is zero.
In the next step 204, a reference mark is set according to the value of a variable RcgState that determines the type of road marking mark recognized in the image processing of the current sampling period.
When RcgState = 0, Ref_Mark = “Re”,
When RcgState = 1, Ref_Mark = “MA”,
When RcgState = 2, Ref_Mark = “stop” (9)
Here, Ref_Mark is a mark that is referred to in order to collate with the detected road marking mark, and is a reference with a sense of perspective as viewed from diagonally above in the same manner as when looking down the road marking mark on the road surface from the camera 3. It is assumed that a mark for use is set in advance.

If the counter DetectsStep = 1 and only “le” is detected, parameters for detecting road markings are set as follows in step 205.
If RcgState> 2 or RcgState <0, RcgState = 0,
If RcgState = 0, obj_Y1 = obj_Y ... for detecting “ma”,
If RcgState = 1, obj_Y1 = obj_Y + 2.0 ... for "stop" detection
... (10)
In the next step 206, a reference mark is set according to the value of a variable RcgState that determines the type of road marking mark recognized in the image processing of the current sampling period.
When RcgState = 0, Ref_Mark = “MA”,
When RcgState = 1, Ref_Mark = “stop” (11)

If the counter DetectsStep = 2 and only “ma” is detected, parameters for detecting road markings are set as follows in step 207.
If RcgState> 2 or RcgState <0, RcgState = 0,
If RcgState = 0, then obj_Y1 = obj_Y ... for "stop" detection,
If RcgState = 1, obj_Y1 = obj_Y-4.0 ... for detecting “re”
(12)
In the next step 208, a reference mark is set according to the value of a variable RcgState that determines the type of road marking mark recognized in the image processing of the current sampling period.
When RcgState = 0, Ref_Mark = “stop”,
When RcgState = 1, Ref_Mark = “Re” (13)

In the case where the counter DetectsStep = 3 and “re” and “ma” are detected, parameters for detecting road markings are set as follows in step 209.
If RcgState> 2 or RcgState <0, RcgState = 0,
If RcgState = 0, then obj_Y1 = obj_Y ... for "stop" detection,
If RcgState = 1, obj_Y1 = obj_Y + 2.0 ... for "-" detection
(14)
In the subsequent step 210, a reference mark is set according to the value of a variable RcgState that determines the type of road marking mark recognized in the image processing of the current sampling period.
When RcgState = 0, Ref_Mark = “stop”,
When RcgState = 1, Ref_Mark = “−” (stop line) (15)

If the counter DetectsStep = 4 and only “stop” is detected, parameters for detecting road markings are set as follows in step 211.
If RcgState> 2 or RcgState <0, RcgState = 0,
If RcgState = 0, obj_Y1 = obj_Y-4.0 ... for detecting “re”,
If RcgState = 1, obj_Y1 = obj_Y-2.0 ... for detecting “ma”
... (16)
In the subsequent step 212, a reference mark is set in accordance with the value of a variable RcgState that determines the type of road marking mark recognized in the image processing of the current sampling period.
When RcgState = 0, Ref_Mark = “Re”,
When RcgState = 1, Ref_Mark = “MA” (17)

When the counter DetectsStep = 5 and “re” and “stop” are detected, parameters for detecting road markings are set as follows in step 213.
If RcgState> 2 or RcgState <0, RcgState = 0,
If RcgState = 0, obj_Y1 = obj_Y-4.0 ... for detecting “ma”,
If RcgState = 1, obj_Y1 = obj_Y ... "-" detection
... (18)
In the subsequent step 212, a reference mark is set in accordance with the value of a variable RcgState that determines the type of road marking mark recognized in the image processing of the current sampling period.
When RcgState = 0, Ref_Mark = “MA”,
When RcgState = 1, Ref_Mark = “−” (19)

In the case of DetectsStep = 6 (when “MA” and “STOP” are detected), the parameter for detecting the road display is set as follows in step 215.
If RcgState> 2 or RcgState <0, RcgState = 0,
If RcgState = 0, obj_Y1 = obj_Y-6.0 ... for detecting “re”,
If RcgState = 1, obj_Y1 = obj_Y ... "-" detection
... (20)
In the subsequent step 216, a reference mark is set according to the value of a variable RcgState that determines the type of road marking mark recognized in the image processing of the current sampling period.
When RcgState = 0, Ref_Mark = “Re”,
When RcgState = 1, Ref_Mark = “−” (21)

If the counter DetectsStep = 7 and “re”, “ma”, “stop” are detected, parameters for detecting road display are set as follows in step 217.
obj_Y1 = obj_Y (for detecting "-") (22)
In the subsequent step 218, the reference mark is set as follows regardless of the value of the variable RcgState that determines the type of road marking mark recognized in the image processing of the current sampling period.
Ref_Mark = “-”

In step 219, the image processing area on the captured image of the camera 3 is set as follows.
disp_obj_YA = y0 + focus V * CAM_h / (obj_Y1 + ProcessDist)
... (23)
disp_obj_YB = y0 + focusV * CAM_h / (obj_Y1-ProcessDist)
... (24)
disp_obj_XL = x0 + focusH * obj_X1 / obj_Y1
-FocusH * ProcessWide / obj_Y1 (25)
disp_obj_XR = x0 + focusH * obj_X1 / obj_Y1
+ FocusH * ProcessWide / obj_Y1 (26)
Here, disp_obj_YA is the upper coordinate of the image processing area, disp_obj_YB is the lower coordinate of the image processing area, disp_obj_XL is the left coordinate of the image processing area, and disp_obj_XR is the right coordinate of the image processing area. Further, y0 and x0 are the longitudinal coordinates [pix: number of pixels] of the vanishing point (a parameter determined by the mounting position and orientation of the camera 3 (direction of the photographing lens)). Similarly, focus V and focus H are values [pix] (parameters determined by the angle of view of the camera 3 and the resolution of the light receiving element) obtained by converting the focal length of the camera 3 in the vertical and horizontal directions into pixels. CAM_h is the mounting height (unit: meters) of the camera 3.

  In step 220, pre-processing is performed for identifying marks such as characters and symbols of road markings in the image processing area. As this processing method, in this embodiment, differentiation and binarization processing are performed. The differential operation is performed by a sobel filter which is a general method for obtaining a differential image. The sobel filter is an operation for obtaining a luminance change between pixels adjacent to a certain pixel. In the binarization process, the average value Av and the variance value Vr are obtained for the luminance of the obtained differential image, and the binarization process is performed using these values as threshold values. Note that binarization is black when the luminance of each pixel (in this case, the luminance change for a differential image) is equal to or less than a threshold value, white when the luminance exceeds the threshold value, and white and shade of the differential image. And black.

In step 221, the road marking mark identified by the pre-processing described above is compared with the reference mark to recognize the road marking mark. As this recognition method, in this embodiment, an OCR (Optical Character Recognition) method between a reference mark and a binarized image is applied, and the road marking mark can be recognized when there is a predetermined similarity or more. And A variable RcgResult representing the recognition result state is set as follows according to the recognized mark.
If Ref_Mark = “Re” is recognized, RcgResult = 1,
If Ref_Mark = “ma” is recognized, RcgResult = 2,
If Ref_Mark = “stop” is recognized, RcgResult = 4,
If Ref_Mark = “−” is recognized, RcgResult = 8,
When Ref_Mark cannot be recognized, RcgResult = 0 (27)

In step 222, the process branches according to the value of the counter DetectsStep representing the number of times the road marking mark has been recognized and the value of the variable RcgResult representing the recognition result state according to the recognized mark.
If RcgResult = 0 (cannot be detected), go to step 226.
If RcgResult = 8 (stop line detected), go to step 228.
If DetectStep <RcgResult, go to step 223.
If DetectStep> RcgResult, go to step 225.

In the case of DetectStep <RcgResult, the position of the area to be searched next is obtained by the following equation based on the position of the road marking mark recognized in step 223 on the screen.
obj_Y = focusV * CAM_h / (Ref_Mark_Center_y−y0) +2.0,
obj_X = obj_Y (Ref_Mark_Center_x−x0) / focusH (28)
Here, Ref_Mark_Center_y is the vertical direction (vertical direction) position of the recognized road marking mark on the screen, and Ref_Mark_Center_x is the horizontal direction (horizontal direction) position of the recognized road marking mark on the screen.

In the subsequent step 224, the size of the area to be searched next is obtained by the following equation based on the size of the recognized road marking mark on the screen.
ProcessWide = (Rcg_Mark_SizeH * obj_Y / focusH) / 2
ProcessDist = (Rcg_Mark_SizeV * obj_Y / focusV) / 2 (29)
Here, Rcg_Mark_SizeH and Rcg_Mark_SizeV are the horizontal size (unit: number of pixels) and vertical size (unit: number of pixels) of the recognized road marking mark. In step 225, a counter DetectsStep representing the number of times the road marking mark has been recognized is set by the following equation.
DectectsStep = DectectsStep + RcgResult (30)

In step 226, a variable RcgState that determines the type of road marking mark recognized in the image processing of the current sampling period is set by the following equation.
RcgState = RcgState + 1 (31)
In step 228, since the temporary stop mark “−” is detected in front of the host vehicle, the driver's seat belt is wound up by a seat belt retractor (not shown) to notify the driver that there is a temporary stop mark. In step 227, the past values of the vehicle behavior data such as the vehicle speed and the steering angle are updated, and the process is terminated.

  In this way, road markings that are continuously painted in the traveling direction on the road are partially detected one character or one symbol at a time regardless of the character recognition order, and the next detection result of one road marking is Since the position and size of the road marking search area (image processing area) are set, the image processing area can be narrowed to the minimum necessary range, and the image processing load can be reduced. Further, in the first embodiment described above, the road marking cannot be detected unless all the characters of the road marking character string can be detected. However, in the second embodiment, a part of the road marking character string is detected. If this character can be detected, the road marking can be detected.

<< Third Embodiment of the Invention >>
A third embodiment for detecting a crosswalk without a signal by partially recognizing a road display mark one symbol at a time in the order of “◇” → “◇” → “−” will be described. The configuration of the third embodiment is the same as the configuration shown in FIG.

  FIG. 5 shows a process of detecting the road marking “◇” of the pedestrian crossing painted on the road by processing the image captured by the camera 3. The external environment recognition device 5 repeats the road marking detection process shown in FIG. 5 every 50 msec. In step 300, an image picked up by the camera 3 in the current sampling and processed by the image processing device 4 is read. In the subsequent step 301, the vehicle speed is read from the vehicle speed detection device 6 and the steering angle is read from the steering angle detection device 7. In step 302, it is confirmed whether or not the counter DetectsStep representing the number of times a mark such as a road marking character or symbol has been recognized is 0, that is, no road marking mark has been recognized yet.

If no mark such as a road marking character or symbol has been recognized yet, the process proceeds to step 103, and initial conditions for detecting the road marking are set as follows.
obj_Y = 20,
obj_X = 0,
ProcessWide = 1.5,
ProcessDist = 10.0
Reference mark rotation allowable range; minimum value αs1 = −45 [deg], maximum value αs2 = + 45 [deg] (32)
In Expression (32), obj_Y is the front-rear direction position (unit meter) of the image processing area on the road, and obj_X is the left-right position of the image processing area on the road (unit meter, the front of the host vehicle is 0). Further, ProcessWide is a half value (unit meter) of the horizontal range of the image processing area, and ProcessDist is a half value (unit meter) of the front-rear direction range of the image processing area.

In step 304, the image processing area of the image captured by the camera 3 is set as follows.
disp_obj_YA = y0 + focusV * CAM_h / (obj_Y + ProcessDist) (33)
disp_obj_YB = y0 + focusV * CAM_h / (obj_Y-ProcessDist) (34)
disp_obj_XL = x0 + focusH * obj_X / obj_Y-focusH * ProcessWide / obj_Y (35)
disp_obj_XR = x0 + focusH * obj_X / obj_Y + focusH * ProcessWide / obj_Y
... (36)
Here, disp_obj_YA is the upper coordinate of the image processing area, disp_obj_YB is the lower coordinate of the image processing area, disp_obj_XL is the left coordinate of the image processing area, and disp_obj_XR is the right coordinate of the image processing area. Further, y0 and x0 are the longitudinal coordinates [pix: number of pixels] of the vanishing point (a parameter determined by the mounting position and orientation of the camera 3 (direction of the photographing lens)). Similarly, focus V and focus H are values [pix] (parameters determined by the angle of view of the camera 3 and the resolution of the light receiving element) obtained by converting the focal length of the camera 3 in the vertical and horizontal directions into pixels. CAM_h is the mounting height (unit: meters) of the camera 3.

  In step 305, pre-processing for identifying marks such as road marking characters and symbols in the image processing area is performed. As this processing method, in this embodiment, differentiation operation and binarization processing are performed. The differential operation is performed by a sobel filter which is a general method for obtaining a differential image. The sobel filter is an operation for obtaining a luminance change between pixels adjacent to a certain pixel. In the binarization process, the average value Av and the variance value Vr are obtained for the luminance of the obtained differential image, and the binarization process is performed using these values as threshold values. Note that binarization is black when the luminance of each pixel (in this case, the luminance change for a differential image) is equal to or less than a threshold value, white when the luminance exceeds the threshold value, and white and shade of the differential image. And black.

In step 306, a reference mark is set in accordance with the value of the counter DetectsStep representing the number of times the road marking mark has been recognized.
If DetectsStep = 0 and nothing is detected yet, Ref_Mark = “◇”,
If DetectsStep = 1 and the first “◇” is detected, Ref_Mark = “◇”,
If DetectsStep = 2 and two “◇” are detected, Ref_Mark = “−” (look for a horizontal line as a pedestrian crossing) (37)
Here, Ref_Mark is a mark that is referred to in order to collate with the detected road marking mark. Similar to the camera 3 looking down the road marking on the road surface, it is a reference for viewing with a perspective as seen from diagonally above. Assume that marks are set in advance.

  In step 307, the road marking mark identified by the pre-processing described above is compared with the reference mark to recognize the road marking mark. As this recognition method, in this embodiment, an OCR (Optical Character Recognition) method between a reference mark and a binarized image is applied, and the road marking mark can be recognized when there is a predetermined similarity or more. And Further, when this recognition is performed, the allowable rotation range of the reference mark is limited from the minimum αs1 to the maximum αs2. If the road marking mark is recognized in step 308, the process proceeds to step 309. If not recognized, the process proceeds to step 314.

If the road marking mark on the pedestrian crossing can be recognized, in step 309, the road marking mark recognition frequency counter DetectStep is incremented. If the counter DetectStep is 3, the process proceeds to step 315. If it is less than 3, the process proceeds to step 310. In step 310, based on the position of the recognized road marking mark on the screen, the position of the image processing area where the next road marking mark is searched is obtained by the following equation.
obj_Y = focusV * CAM_h / (Ref_Mark_Center_y−y0) +2.0,
obj_X = obj_Y (Ref_Mark_Center_x−x0) / focusH (38)
In equation (38), Ref_Mark_Center_y is the vertical (up and down) position of the recognized mark on the screen, and Ref_Mark_Center_x is the horizontal (left and right) position of the recognized mark on the screen.

In step 311, the size of the image processing area to be searched for the next road marking mark is obtained by the following equation based on the recognized size of the road marking mark on the screen.
ProcessWide = (Rcg_Mark_SizeH * obj_Y / focusH) / 2
ProcessDist = (Rcg_Mark_SizeV * obj_Y / focusV) / 2 (39)
In Expression (39), Rcg_Mark_SizeH and Rcg_Mark_SizeV are the horizontal size (unit is the number of pixels) and vertical size (unit is the number of pixels) of the recognized road marking mark.

In step 312, the inclination of the reference mark when searching for the next mark is limited based on the recognized direction of the road marking mark on the pedestrian crossing.
αs1 = αr−30 [deg],
αs2 = αr + 30 [deg] (40)
Here, αr is the rotation angle of the reference mark at the time of recognition, and αs1 and αs2 are the minimum angle and the maximum angle that limit the rotation angle of the reference mark when searching in the next sampling period.

  In this embodiment, an example in which the direction of the reference mark is tilted according to the detected direction of the road marking mark is shown. However, the search area (image processing area) is tilted according to the detected direction of the road marking mark. May be.

In step 313, the horizontal position of the area to be searched next is limited based on the orientation of the recognized road marking mark on the screen.

obj_X = obj_Y (Ref_Mark_Center_x−x0) / focus H + 2.0 tan αr
... (41)

  On the other hand, in step 315, since a pedestrian crossing without a signal is detected in front of the host vehicle, for example, the repulsive force of the accelerator pedal is strengthened to provide information to the driver. In step 314, the past values of the vehicle behavior data such as the vehicle speed and the steering angle are updated, and the process is terminated.

  In this way, the road markings of the pedestrian crossing that are continuously painted in the direction of travel on the road are partially detected one by one, and the search area for the next road marking (image processing) is detected based on the detection result of one road marking. Since the position and size of the (region) are set, the image processing region can be narrowed to the minimum necessary range, and the image processing load can be reduced.

<< Fourth Embodiment of the Invention >>
The first step is to detect a road marking of “slow down” that promptly decelerates by partially recognizing the mark one character or one symbol at a time in the order of “se” → “to” → “fall” → “degree” → “speed” Embodiment 4 will be described. The configuration of the fourth embodiment is the same as the configuration shown in FIG.

  FIG. 6 shows a process of detecting a “speed reduction” road marking painted on the road by processing an image captured by the camera 3. The external environment recognition device 5 repeats the road marking detection process shown in FIG. 6 every 50 msec. In step 400, an image picked up by the camera 3 in this sampling and processed by the image processing device 4 is read. In the subsequent step 401, the vehicle speed is read from the vehicle speed detection device 6 and the steering angle is read from the steering angle detection device 7. In step 402, it is confirmed whether or not the counter DetectsStep representing the number of times a mark such as a road marking character or symbol has been recognized is 0, that is, no road marking mark has been recognized yet.

If no mark such as a character or symbol of the road marking has been recognized yet, the process proceeds to step 403, and initial conditions for detecting the road marking are set as follows.
obj_Y = 20,
obj_X = 0,
ProcessWide = 1.5,
ProcessDist = 10.0
obj_Yf = obj_Y,
obj_Yn = obj_Y (42)
In equation (42), obj_Y is the position in the front-rear direction (unit meter) of the image processing area on the road, and obj_X is the position in the left-right direction of the image processing area on the road (unit meter, the front of the host vehicle is 0). Further, ProcessWide is a half value (unit meter) of the horizontal range of the image processing area, and ProcessDist is a half value (unit meter) of the front-rear direction range of the image processing area.

In step 404, the image processing area of the captured image of the camera 3 is set as follows using [obj_Yf, obj_Yn, obj_X1, obj_Xr] representing the position range.
disp_obj_YA = y0 + focus V * CAM_h / (obj_Yf + ProcessDist) (43)
disp_obj_YB = y0 + focus V * CAM_h / (obj_Yn-ProcessDist) (44)
disp_obj_XL = x0 + focusH * obj_Xl / obj_Y-focusH * ProcessWide / obj_Y (45)
disp_obj_XR = x0 + focusH * obj_Xr / obj_Y + focusH * ProcessWide / obj_Y
... (46)
Here, disp_obj_YA is the upper coordinate of the image processing area, disp_obj_YB is the lower coordinate of the image processing area, disp_obj_XL is the left coordinate of the image processing area, and disp_obj_XR is the right coordinate of the image processing area. Further, y0 and x0 are the longitudinal coordinates [pix: number of pixels] of the vanishing point (a parameter determined by the mounting position and orientation of the camera 3 (direction of the photographing lens)). Similarly, focus V and focus H are values [pix] (parameters determined by the angle of view of the camera 3 and the resolution of the light receiving element) obtained by converting the focal length of the camera 3 in the vertical and horizontal directions into pixels. CAM_h is the mounting height (unit: meters) of the camera 3.

  In step 405, preprocessing for identifying a mark such as a character or symbol of a road marking in the image processing area is performed. As this processing method, in this embodiment, differentiation operation and binarization processing are performed. The differential operation is performed by a sobel filter which is a general method for obtaining a differential image. The sobel filter is an operation for obtaining a luminance change between pixels adjacent to a certain pixel. In the binarization process, the average value Av and the variance value Vr are obtained for the luminance of the obtained differential image, and the binarization process is performed using these values as threshold values. Note that binarization is black when the luminance of each pixel (in this case, the luminance change for a differential image) is equal to or less than a threshold value, white when the luminance exceeds the threshold value, and white and shade of the differential image. And black.

In step 406, a reference mark is set according to the value of the counter DetectsStep representing the number of times the road marking mark has been recognized.
If DetectsStep = 0, Ref_Mark = “se”,
If DetectsStep = 1, Ref_Mark = “To”,
If DetectsStep = 2, Ref_Mark = “Fall”,
If DetectsStep = 3, Ref_Mark = “degrees”,
When DetectsStep = 4, Ref_Mark = “Speed” (47)
Here, Ref_Mark is a mark that is referred to in order to collate with the detected road marking mark. Similar to the camera 3 looking down the road marking on the road surface, it is a reference for viewing with a perspective as seen from diagonally above. Assume that marks are set in advance.

  In step 407, the road marking mark identified by the pre-processing described above is compared with the reference mark to recognize the road marking mark. As this recognition method, in this embodiment, an OCR (Optical Character Recognition) method between a reference mark and a binarized image is applied, and the road marking mark can be recognized when there is a predetermined similarity or more. And Further, when this recognition is performed, the allowable rotation range of the reference mark is limited from the minimum αs1 to the maximum αs2. If a road marking mark is recognized in step 408, the process proceeds to step 409. If not recognized, the process proceeds to step 414.

If the low speed road marking mark can be recognized, the road marking mark recognition number counter DecetStep is incremented in step 409. If the counter DetectStep is 5, the process proceeds to step 415. If it is less than 5, the process proceeds to step 410. In step 410, based on the position of the recognized road marking mark on the screen, the position of the image processing area where the next road marking mark is searched is obtained by the following equation.
obj_Y = focusV * CAM_h / (Ref_Mark_Center_y−y0) +2.0,
obj_Yn = obj_Y-func1 (DetectStep),
obj_Yf = obj_Y + func1 (DetectStep)
obj_Xl = obj_Y (Ref_Mark_Center_x-x0) / focus H-func1 (DetectStep)
obj_Xr = obj_Y (Ref_Mark_Center_x-x0) / focusH + func1 (DetectStep)
... (48)
Here, Ref_Mark_Center_y is the vertical (up and down) position of the recognized mark on the screen, and Ref_Mark_Center_x is the horizontal (left and right) position of the recognized mark on the screen. Also, obj_Yn and obj_Yf indicate the positions on the short distance side and the far distance side of the search position, and obj_Xl and obj_Xr indicate the positions on the left and right sides of the search position. func1 (A) is a function having the characteristics shown in FIG.

In step 411, the size of the image processing area to be searched for the next road marking mark is obtained based on the size of the recognized road marking mark on the screen by the following equation.
ProcessWide = (Rcg_Mark_SizeH * obj_Y / focusH) / func2 (DetectStep),
ProcessDist = (Rcg_Mark_SizeV * obj_Y / focusV) / func2 (DetectStep) (49)
In equation (49), Rcg_Mark_SizeH and Rcg_Mark_SizeV are the horizontal size (unit is the number of pixels) and vertical size (unit is the number of pixels) of the recognized road marking mark. Further, func2 (A) is a function having characteristics as shown in FIG.

In step 412, the inclination of the reference mark when searching for the next mark is limited based on the recognized direction of the reduced speed road marking mark.
αs1 = αr−func3 (DectectStep) [deg],
αs2 = αr + func3 (DectectStep) [deg] (50)
Here, αr is the rotation angle of the reference mark at the time of recognition, and αs1 and αs2 are the minimum angle and the maximum angle that limit the rotation angle of the reference mark when searching in the next sampling period. Further, func3 (A) is a function having characteristics as shown in FIG. In this embodiment, an example in which the direction of the reference mark is tilted is shown, but the search area may be tilted.

In step 413, the horizontal position of the area to be searched next is limited based on the orientation of the recognized road marking mark on the screen.
obj_Xl = obj_Y (Ref_Mark_Center_x-x0) / focus H + 2.0 tan ([alpha] r-func4 (DetectStep)),
obj_Xr = obj_Y (Ref_Mark_Center_x-x0) / focus H + 20. tan (αr-func4 (DectectStep)) (51)
Here, func4 (A) is a function having the characteristics shown in FIG.

  On the other hand, in step 415, since the road marking of “slow down” indicating that there is a situation in which the measurement is likely to be erroneous in front of the host vehicle is detected, a braking force of, for example, 0.07G is used as information provision to the driver. Drive shaft torque demand control is performed in addition to the target drive force, and the process proceeds to step 414. In step 414, the past values of the vehicle behavior data such as the vehicle speed and the steering angle are updated, and the process is terminated.

  As described above, the “speed reduction” road sign continuously painted in the traveling direction on the road is partially detected one character or one symbol at a time, and the next road sign is detected based on the detection result of one road sign. Since the position and size of the search area (image processing area) are set, the image processing area can be narrowed to the minimum necessary range, and the image processing load can be reduced.

<< Fifth Embodiment of the Invention >>
The initial search area (image processing area) is narrowed down from the behavior of the host vehicle, and the search area when partially detecting the road display one character or one symbol at a time is also narrowed down from the behavior of the host vehicle, reducing the computational load of image processing. A fifth embodiment configured as described above will be described. The configuration of the fifth embodiment is the same as the configuration shown in FIG.

  FIG. 7 shows a process of detecting a “speed reduction” road marking painted on the road by processing an image captured by the camera 3. The outside recognition device 5 repeats the road marking detection process shown in FIG. 7 every 50 msec. In step 500, an image picked up by the camera 3 in the current sampling and processed by the image processing device 4 is read.

In step 501, the vehicle speed is read from the vehicle speed detection device 6 and the steering angle is read from the steering angle detection device 7, and the curvature radius Row [m] and pitch angle Pitch_z0 [deg] of the future course of the host vehicle are estimated by the following equations.
Row = (1 + A * Vsp ² * LWB) / Steer,
Pitch_z0 = BPF_B1 * mc_z0 + BPF_B2 * mc_z1 + BPF_B3 * mc_z2
-BPF_A2 * Pitch_z1-BPF_A3 * Pitch_z2 (52)
Here, A is a stability factor (a value that can be regarded as a constant determined by the vehicle weight, wheelbase length, center of gravity position, and lateral force of the tire) that is unique to the vehicle, and Vsp is the vehicle speed [m / s], LWB represents the wheel base length [m], and Steer represents the steering angle [rad] (right turn is positive). BPF_B1, BPF_B2, BPF_B3, BPF_A2, and BPF_A3 are coefficients of a digital differential filter (bandpass filter) set to have a predetermined differential characteristic, mc_z0 is the current value of the master cylinder pressure of the brake, and mc_zn is its n Sampling past values. The same applies to Pitch_zn, and the downward direction (nose dive direction) is positive.

In step 502, it is confirmed whether or not the counter DetectsStep representing the number of times a mark such as a road marking character or symbol has been recognized is 0, that is, no road marking mark has been recognized yet. If no mark such as a character or symbol of the road marking has been recognized yet, the process proceeds to step 503, and initial conditions for detecting the road marking are set as follows.
obj_Y = 20,
obj_X = func5 (Row),
ProcessWide = 1
ProcessDist = 10.0
obj_Yf = obj_Y,
obj_Yn = obj_Y (53)
In Expression (53), obj_Y is a position in the front-rear direction (unit meter) of the image processing area on the road, and obj_X is a position in the left-right direction of the image processing area on the road (unit meter, the front of the host vehicle is 0). Further, ProcessWide is a half value (unit meter) of the horizontal range of the image processing area, and ProcessDist is a half value (unit meter) of the front-rear direction range of the image processing area. func5 (Row) is a function of characteristics as shown in FIG.
Compared to the above-described fourth embodiment, by setting obj_X = func5 (Row), the initial position of the search area is corrected by the predicted course of the host vehicle, and by further setting ProcessWide = 1, the initial position of the search area is set. The size of can be squeezed.

In step 504, disp_obj_YA and disp_obj_YB are corrected based on the estimated pitch angle of the host vehicle to obtain an image processing area.
disp_obj_YA = y0 + focusV * CAM_h / (obj_Y + ProcessDist)
+ Vp / va * Pitch_z0 (54)
disp_obj_YB = y0 + focusV * CAM_h / (obj_Y-ProcessDist)
+ Vp / va * Pitch_z0 (55)
disp_obj_XL = x0 + focusH * obj_X / obj_Y-focusH * ProcessWide / obj_Y (56)
disp_obj_XR = x0 + focesH * obj_X / obj_Y + focusH * ProcessWide / obj_Y
... (57)
Here, disp_obj_YA is the upper coordinate of the image processing area, disp_obj_YB is the lower coordinate of the image processing area, disp_obj_XL is the left coordinate of the image processing area, and disp_obj_XR is the right coordinate of the image processing area. Further, y0 and x0 are the longitudinal coordinates [pix: number of pixels] of the vanishing point (a parameter determined by the mounting position and orientation of the camera 3 (direction of the photographing lens)). Similarly, focus V and focus H are values [pix] (parameters determined by the angle of view of the camera 3 and the resolution of the light receiving element) obtained by converting the focal length of the camera 3 in the vertical and horizontal directions into pixels. CAM_h is the mounting height (unit: meters) of the camera 3. Further, vp is the number of pixels [piz] in the vertical direction of the CCD camera 3, and va is the angle of view [deg] in the vertical direction of the CCD camera 3.

  In step 505, preprocessing for identifying a mark such as a character or symbol of a road marking in the image processing area is performed. As this processing method, in this embodiment, differentiation and binarization processing are performed. The differential operation is performed by a sobel filter which is a general method for obtaining a differential image. The sobel filter is an operation for obtaining a luminance change between pixels adjacent to a certain pixel. In the binarization process, the average value Av and the variance value Vr are obtained for the luminance of the obtained differential image, and the binarization process is performed using these values as threshold values. Note that binarization is black when the luminance of each pixel (in this case, the luminance change for a differential image) is equal to or less than a threshold value, white when the luminance exceeds the threshold value, and white and shade of the differential image. And black.

In step 506, a reference mark is set in accordance with the value of the counter DetectsStep representing the number of times the road marking mark has been recognized.
If DetectsStep = 0, Ref_Mark = “se”,
If DetectsStep = 1, Ref_Mark = “To”,
If DetectsStep = 2, Ref_Mark = “Fall”,
If DetectsStep = 3, Ref_Mark = “degrees”,
When DetectsStep = 4, Ref_Mark = “Speed” (58)
Here, Ref_Mark is a mark that is referred to in order to collate with the detected road marking mark. Similar to the camera 3 looking down the road marking on the road surface, it is a reference for viewing with a perspective as seen from diagonally above. Assume that marks are set in advance.

  In step 507, the road marking mark identified by the pre-processing described above is compared with the reference mark to recognize the road marking mark. As this recognition method, in this embodiment, an OCR (Optical Character Recognition) method between a reference mark and a binarized image is applied, and the road marking mark can be recognized when there is a predetermined similarity or more. And Further, when this recognition is performed, the allowable rotation range of the reference mark is limited from the minimum αs1 to the maximum αs2. If the road marking mark is recognized in step 508, the process proceeds to step 509. If not recognized, the process proceeds to step 514.

  If the low speed road marking mark can be recognized, the road marking mark recognition number counter DetectStep is incremented in step 509. If the counter DetectStep is 5, the process proceeds to step 515. If it is less than 5, the process proceeds to step 510.

In step 510, based on the position of the recognized road marking mark on the screen, the position of the image processing area where the next road marking mark is searched is obtained by the following equation.
obj_Y = focusV * CAM_h / (Ref_Mark_Center_y−y0) +2.0,
obj_Yn = obj_Y-func1 (DetectStep),
obj_Yf = obj_Y + func1 (DetectStep)
obj_Xl = obj_Y (Ref_Mark_Center_x-x0) / focus H-func1 (DetectStep)
+ Func6 (Row),
obj_Xr = obj_Y (Ref_Mark_Center_x-x0) / focusH + func1 (DetectStep)
+ Func6 (Row) (59)
Here, Ref_Mark_Center_y is the vertical (up and down) position of the recognized mark on the screen, and Ref_Mark_Center_x is the horizontal (left and right) position of the recognized mark on the screen. Also, obj_Yn and obj_Yf indicate the positions on the short distance side and the far distance side of the search position, and obj_Xl and obj_Xr indicate the positions on the left and right sides of the search position. func1 (A) is a function having the characteristics shown in FIG. Further, func6 (Row) is a function having characteristics as shown in FIG.

In step 511, the size of the image processing area to be searched for the next road marking mark is obtained by the following equation based on the recognized size of the road marking mark on the screen.
ProcessWide = (Rcg_Mark_SizeH * obj_Y / focusH) / func2 (DetectStep),
ProcessDist = (Rcg_Mark_SizeV * obj_Y / focusV) / func2 (DetectStep) (60)
In the equation (60), Rcg_Mark_SizeH and Rcg_Mark_SizeV are the horizontal size (unit: number of pixels) and vertical size (unit: number of pixels) of the recognized road marking mark. Further, func2 (A) is a function having characteristics as shown in FIG.

In step 512, the inclination of the reference mark when searching for the next mark is limited based on the recognized direction of the reduced speed road marking mark.
αs1 = αr−func3 (DectectStep) [deg],
αs2 = αr + func3 (DetectStep) [deg] (61)
Here, αr is the rotation angle of the reference mark at the time of recognition, and αs1 and αs2 are the minimum angle and the maximum angle that limit the rotation angle of the reference mark when searching in the next sampling period. Further, func3 (A) is a function having characteristics as shown in FIG. In this embodiment, an example in which the direction of the reference mark is tilted is shown, but the search area may be tilted.

In step 513, the horizontal position of the area to be searched next is limited based on the orientation of the recognized road marking mark on the screen.
obj_Xl = obj_Y (Ref_Mark_Center_x-x0) / focus H + 2.0 tan ([alpha] r-func4 (DetectStep)),
obj_Xr = obj_Y (Ref_Mark_Center_x-x0) / focus H + 20. tan (αr-func4 (DectectStep)) (62)
Here, func4 (A) is a function having the characteristics shown in FIG.

  On the other hand, in step 515, since a road marking of “slow down” indicating that there is a situation in which the measurement is likely to be erroneous in front of the host vehicle is detected, a braking force of, for example, 0.07 G is used as information to the driver. Drive shaft torque demand control is performed in addition to the target drive force, and the process proceeds to step 514. In step 514, the past values of the vehicle behavior data such as the vehicle speed and the steering angle are updated, and the process is terminated.

  As described above, the “speed reduction” road marking continuously painted in the traveling direction on the road is partially detected one character or one symbol at a time, and the image processing area is narrowed according to the behavior of the vehicle. Therefore, the image processing load can be reduced.

  The correspondence between the constituent elements of the claims and the constituent elements of the embodiment is as follows. That is, the camera 3 is the imaging means, the image processing device 4 and the external recognition device 5 are the road marking detection means, the area narrowing means and the road marking identification means, the speaker and the display of the first embodiment, and the second embodiment. The seat belt retractor of the embodiment, the accelerator pedal repulsion force adding device of the third embodiment, and the drive shaft torque demand control device of the fourth and fifth embodiments respectively constitute road marking transmission means. In addition, as long as the characteristic function of this invention is not impaired, each component is not limited to the said structure.

  As described above, according to an embodiment, when a captured image obtained by capturing the front of the host vehicle is processed and road markings continuously drawn in the traveling direction on the road are detected one character or one symbol at a time, the road markings are detected. Each time one character or one symbol is detected, the image processing area on the captured image is narrowed down based on the detection result, and the road marking is identified based on the continuously detected character or symbol. The image processing load can be reduced while improving the detection reliability of the sign.

  According to one embodiment, the position of the image processing area for detecting one character or one symbol of the next road sign is set according to the position of one character or one symbol of the detected road sign. Therefore, every time one character or one symbol of a road sign is detected, the image processing load when detecting one character or one symbol of the next road sign can be reduced.

  According to one embodiment, each time one character or one symbol of a road sign is detected, the setting range of the position of the image processing area is narrowed, so that one character or one symbol of the road sign is detected. Every time, it is possible to reduce the image processing load when detecting one character or one symbol of the next road marking.

  According to one embodiment, the size of the image processing area for detecting one character or one symbol of the next road sign is set according to the size of one character or one symbol of the detected road sign. Therefore, every time one character or one symbol of a road sign is detected, the image processing load when detecting one character or one symbol of the next road sign can be reduced.

  According to one embodiment, the size of the image processing area is reduced every time one character or one symbol of a road sign is detected, so that every time one character or one symbol of a road sign is detected. The image processing load when detecting one character or one symbol of the next road marking can be reduced.

  According to one embodiment, the image processing area for detecting one character or one symbol of the next road sign is inclined according to the direction of the one character or one symbol of the detected road sign. Each time one character or one symbol of a sign is detected, the image processing load when detecting one character or one symbol of the next road sign can be reduced.

  According to one embodiment, the position of the image processing area for detecting one character or one symbol of the next road sign is set according to the direction of one character or one symbol of the detected road sign. Therefore, every time one character or one symbol of a road sign is detected, the image processing load when detecting one character or one symbol of the next road sign can be reduced.

  According to one embodiment, each time a character or symbol of a road sign is detected, the position in the left-right direction of the image processing area is narrowed, so that one character or symbol of the road sign is detected. Every time, it is possible to reduce the image processing load when detecting one character or one symbol of the next road marking.

  According to one embodiment, since one character or one symbol of the road sign to be detected next is specified based on one character or one symbol of the detected road sign, The image processing load when detecting one symbol can be reduced.

  According to one embodiment, since the pitch angle of the host vehicle is detected and the image processing area is corrected based on the detected pitch angle, an image is first detected when one character or one symbol of a road marking is detected. The processing load can be reduced, and when detecting one character or one symbol of the next road marking, it is possible to cope with a change in the road shape or the behavior of the host vehicle without increasing the image processing load.

  According to one embodiment, since the course of the host vehicle is predicted and the image processing area is corrected based on the predicted course of the host vehicle, when detecting one character or one symbol of a road sign for the first time, The image processing load can be reduced, and when one character or one symbol of the next road marking is detected, it is possible to cope with a change in the road shape or the behavior of the host vehicle without increasing the image processing load.

It is a figure which shows the structure of 1st Embodiment. It is a flowchart which shows the road marking detection process of 1st Embodiment. It is a flowchart which shows the road marking detection process of 2nd Embodiment. It is a flowchart which shows the road marking detection process of 2nd Embodiment following FIG. It is a flowchart which shows the road marking detection process of 3rd Embodiment. It is a flowchart which shows the road marking detection process of 4th Embodiment. It is a flowchart which shows the road marking detection process of 5th Embodiment. It is a figure which shows the characteristic of the function func1. It is a figure which shows the characteristic of the function func2. It is a figure which shows the characteristic of the function func3. It is a figure which shows the characteristic of the function func4. It is a figure which shows the characteristic of the function func5. It is a figure which shows the characteristic of the function func6.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Laser radar 2 Radar processing device 3 CCD camera 4 Image processing device 5 Outside world recognition device 6 Vehicle speed detection device 7 Steering angle detection device 8 Automatic brake control device 9 Negative pressure brake booster

Claims (10)

Imaging means for imaging the front of the vehicle;
Road sign detection means for processing a picked-up image of the image pickup means and detecting a road sign continuously drawn in a traveling direction on the road one character or one symbol at a time;
Area narrowing means for narrowing down the image processing area of the road sign detection means on the captured image of the imaging means based on the detection result each time one character or one symbol of the road sign is detected by the road sign detection means; ,
Road marking identifying means for identifying road markings based on characters or symbols continuously detected by the road marking detecting means;
Road marking transmission means for transmitting a road marking identification result by the road marking identification means to an occupant, and the area narrowing means is located at the position of one character or one symbol of the road marking detected by the road marking detection means. In response, the vehicular road marking detection apparatus , wherein the position of the image processing area of the road marking detection means for detecting one character or one symbol of the next road marking is set . In the vehicle road marking detection device according to claim 1,
The area narrowing means narrows the setting range of the position of the image processing area of the road marking detection means every time one character or one symbol of the road marking is detected by the road sign detection means. Road sign detection device. In the vehicle road marking detection apparatus according to any one of claims 1 and 2,
The area narrowing-down means detects the road sign detecting means for detecting one character or one symbol of the next road sign in accordance with the size of one character or one symbol of the road sign detected by the road sign detecting means. A vehicular road marking detection apparatus characterized in that the size of the image processing area is set . In the vehicle road marking detection apparatus according to claim 3,
The area narrowing means narrows the size of the image processing area of the road marking detection means each time one character or one symbol of the road marking is detected by the road sign detection means. Sign detection device. In the vehicle road marking detection device according to any one of claims 1 to 4,
The area narrowing-down means includes a road sign detection means for detecting one character or one symbol of the next road sign according to the direction of one character or one symbol of the road sign detected by the road sign detection means. A vehicle road marking detection apparatus characterized by tilting an image processing area . In the vehicle road marking detection apparatus according to any one of claims 1 to 5,
The area narrowing-down means includes a road sign detection means for detecting one character or one symbol of the next road sign according to the direction of one character or one symbol of the road sign detected by the road sign detection means. A road marking detection apparatus for a vehicle , wherein the position of an image processing area is set . In the vehicle road marking detection apparatus according to claim 6,
The area narrowing means narrows the position in the left-right direction of the image processing area of the road marking detection means each time one character or one symbol of the road marking is detected by the road marking detection means. Road sign detection device. In the vehicle road marking detection apparatus according to any one of claims 1 to 7,
The area narrowing-down means specifies one character or one symbol of a road sign to be detected next based on one character or one symbol of the detected road sign. In the vehicle road marking detection device according to any one of claims 1 to 8,
Pitch angle detection means for detecting the pitch angle of the host vehicle,
The vehicular road marking detection apparatus , wherein the area narrowing means corrects an image processing area of the road marking detection means based on the pitch angle detected by the pitch angle detection means . In the vehicle road marking detection device according to any one of claims 1 to 9,
A route prediction means for predicting the route of the host vehicle is provided,
The vehicular road marking detection apparatus , wherein the area narrowing means corrects an image processing area of the road marking detection means based on the course of the host vehicle predicted by the course prediction means .

Images