JP4305057B2 - Armpit detector - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an armpit detection device.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, there is known an armpit detection device that detects a driver's armpit action based on an allowable armpit time corresponding to the driver's face angle and a time during which the face angle is maintained (eg, Patent Document 1). , 2).
[0003]
In this device, the aside look allowed time is changed according to the direction of the driver's face, and the look aside action is determined. For example, when the driver's face direction angle is large, it is difficult for a scene in the forward direction or the like to enter the driver's field of view. Further, when the driver's face angle is small, since the scenery in the forward direction or the like is relatively easy to enter the driver's field of view, the act of looking aside is determined by setting a long look aside time.
[0004]
In this way, the driver's side-seeing action is suitably detected by changing the side-viewing allowable time.
[0005]
[Patent Document 1]
Japanese Patent Laid-Open No. 3-167698
[0006]
[Patent Document 2]
JP-A-7-57172
[0007]
[Problems to be solved by the invention]
However, since the above-mentioned side-by-side detection device does not take into account the left and right pillar portions that block the driver's field of view, it is insufficient to appropriately determine the side-by-side action. For example, even when the face orientation angle is small, if the driver's visual recognition position is in the vicinity of the pillar portion or the like, the driver's field of view is obstructed, so that the allowable time for looking aside is relatively short. There is a need.
[0008]
As described above, it cannot be said that the conventional looking-aside detection device appropriately detects the driver's looking-aside act.
[0009]
[Means for Solving the Problems]
  According to the present invention, the photographing means photographs the face of the driver of the vehicle, the tracking means tracks the position of the driver's eyes based on the captured image including the driver's face obtained by the imaging means, The learning means determines the movement amount and staying time of the eye position output in time series from the tracking means, learns the eye position on the captured image when the driver sees the predetermined position, and The area setting means sets a predetermined number of image areas with different looking-aside allowable times based on the eye positions on the captured image learned by the learning means, and the looking-ahead detection means is assigned to the image areas set by the image area setting means. Based on the allowed time for looking aside, aside detection is performed.
  More specifically, the learning means learns the positions of the eyes on the captured image when viewing the positions of the left and right door mirrors and the room mirror as the predetermined positions, and the image area setting means The right end of the first image area (A) adjacent to the right of the front gaze area is determined based on the position of the second, and the second adjacent to the left of the front gaze area is determined based on the position of the eye when viewing the right door mirror. The left end and the lower end of the image area (B) are determined, and the right end of the third image area (C) adjacent to the right of the first image area (A) is determined based on the position of the eye when the left door mirror is viewed. Determine the bottom edge.
[0010]
【The invention's effect】
  According to the present invention, a predetermined number of image regions having different looking-aside allowable times are set based on the position of the eye on the captured image. Usually the driver is a carBothSome structures are relatively visible.These structures include left and right door mirrors and room mirrors.For this reason,When these structures are visibleBy using the position of the eye on the captured image as a reference,BothAn image area based on the position of a structure or the like can be set.That is, the right end of the first image area (A) adjacent to the right of the front gaze area is determined based on the position of the eye when viewing the rear view mirror, and the front gaze area is determined based on the position of the eye when viewing the right door mirror. The third image adjacent to the right of the first image area (A) is determined based on the position of the eye when the left door mirror is visually recognized. The right end and bottom end of the region (C) can be determined.Also carBothSince it is based on the position of a structure or the like, it is also possible to assign a side-to-side permissible time that takes into account a portion that obstructs the driver's field of view into each of a predetermined number of image areas.
[0011]
Then, based on the allowable time for looking aside allocated to each image area and the eye position to be tracked, the looking aside is detected. For this reason, it is possible to perform a look-ahead detection adapted to the in-vehicle environment based on the look-ahead allowable time allocated to each image area.
[0012]
Therefore, it is possible to improve the accuracy in detecting the driver's act of looking aside.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described with reference to the drawings.
[0014]
FIG. 1 is a configuration diagram of an armpit detection device according to the first embodiment of the present invention. As shown in the figure, the armpit detection device 1 of the present embodiment includes an imaging unit (imaging means) 10 that captures the face of the driver of the vehicle. The imaging unit 10 is a CCD camera or the like for imaging visible light, for example, and is installed below the front of the driver.
[0015]
Further, the aside detection device 1 includes a processing device 20 that detects a driver's aside action based on a captured image captured by the imaging unit 10, and when the processing device 20 detects a driver's aside action, And an alarm 30 for informing the driver of the effect.
[0016]
Next, a detailed configuration of the processing device 20 will be described with reference to FIG. FIG. 2 is a functional block diagram of the armpit detection device 1 according to the first embodiment of the present invention.
[0017]
The processing device 20 includes a position detection unit 21 that detects the position of the driver's eyes based on data of a captured image from the imaging unit 10, and a tracking unit (tracking unit) 22 that tracks the position of the driver's eyes. And.
[0018]
The position detection unit 21 detects the position of the eye from the entire captured image input from the imaging unit 10. The tracking unit 22 defines a tracking region including the eye position on the image on the basis of the position of the driver's eye detected by the position detection unit 21, and determines the position of the eye from the subsequent captured image. In the case of detection, the driver's eyes are detected from the tracking area.
[0019]
Further, the processing device 20 includes a learning unit (learning unit) 23 that determines the movement amount and staying time of the eye position output from the tracking unit 22 in time series. The learning unit 23 learns the position of the eye on the captured image when the driver looks at the left and right door mirrors, the room mirror, and the like based on the determined movement amount and residence time.
[0020]
The processing device 20 includes an image region setting unit (image region setting unit) 24 that sets an image region on the basis of the eye position on the captured image learned by the learning unit 23. The image area setting unit 24 sets a predetermined number of image areas. Here, the image area is an area to which an allowable time for looking aside for detecting the looking aside is allocated. The image area setting unit 24 assigns a different allowance time for looking aside to each of the predetermined number of image areas.
[0021]
The processing device 20 includes an armpit detection unit (armpit detection means) 25 that performs armpit detection. This side-view detection unit 25 performs side-arm detection based on the above-mentioned allowable side-view time. Specifically, the aside detection unit 25 determines to which image region the eye position tracked by the tracking unit 22 belongs. Then, based on the dwell time of the eye position tracked by the tracking unit 22 and the allowable time for looking aside of the image area to which the eye position belongs, the aside detection is performed.
[0022]
Next, the outline of the operation of the aside detection device 1 will be described. First, the imaging unit 10 images a region including the driver's face, and sends the obtained captured image data to the processing device 20.
[0023]
The position detection unit 21 of the processing device 20 receives captured image data and detects the position of the driver's eyes. The eye position is detected as follows, for example. FIG. 3 is an explanatory diagram of an initial process performed when the position detection unit 21 detects an eye position. In FIG. 3, a captured image of 480 pixels vertically and 512 pixels horizontally will be described as an example.
[0024]
  First, the position detection unit 21 acquires density value data for all pixels in the vertical direction of the image. That is, the density value data is acquired from the coordinates (0, 0) to (0, 479) shown in FIG. 3, and then the density value data is acquired from the coordinates (1, 0) to (1, 479). To do. Then, through Xa and Xb shown in FIG. 3, finally, density value data is obtained for the lines from coordinates (511, 0) to (511, 479). Thereafter, the position detection unit 21 extracts pixels in which the change in density value satisfies a predetermined condition, and obtains a pixel group as shown in FIG. FIG. 4 is an explanatory diagram showing a state when the position detection unit 21 extracts a predetermined pixel. As shown in the figure, the extracted pixels correspond to the positions of the driver's eyebrows, eyes, nose and mouth. More specifically, two pixels A1 and A2 are extracted for the Xc line. Also,XdFor the line, four pixels A1 to A4 are extracted. These pixels A1 to A4 are distinguished, for example, by the amount of change in density value. After the pixels are extracted, the position detection unit 21 groups the pixels that are close to each other in the horizontal direction of the image.
[0025]
FIG. 5 is an explanatory diagram showing a state when pixels adjacent in the horizontal direction of the image are grouped. As shown in the figure, by grouping, the position detection unit 21 forms continuous data G1 to G6 corresponding to the right eyebrow, left eyebrow, right eye, left eye, nose and mouth of the driver.
[0026]
Thereafter, the position detection unit 21 performs zoning processing. FIG. 6 is an explanatory diagram showing a state after zoning by the position detection unit 21. The position detection unit 21 zones the positions where the continuous data G1 to G6 exist in the vertical direction of the image. At this time, the position detection unit 21 forms three zones (ZONE: L, ZONE: C, and ZONE: R). Then, the position detection unit 21 detects the position of the eye by determining the relative positional relationship.
[0027]
After detecting the eye position, the tracking unit 22 stores the coordinate value of the detected eye position, and sets a tracking area smaller than the entire image with the stored position as a reference. Thereafter, the tracking unit 22 detects the position of the eye in the tracking region every time a captured image is input.
[0028]
7A and 7B are explanatory diagrams showing how the tracking unit 22 tracks the eye position. FIG. 7A shows the initial tracking area, and FIG. 7B shows the detected eye after setting the initial tracking area. (C) shows the newly set tracking area based on the detected eye position, and (d) shows the detected eye position from within the newly set tracking area. Yes. In FIG. 7, it is assumed that the eye indicated by the broken line has been detected last time, and the eye indicated by the solid line has been detected this time.
[0029]
When the position of the eye is detected by the position detection unit 21, the tracking unit 22 sets a tracking region around the detected eye position (FIG. 7A). The coordinate position of the eye at this time is (xk1, yk1). Thereafter, when a captured image is input, the tracking unit 22 detects the position of the eye from the tracking region centered at (xk1, yk1) (FIG. 7B). The detected coordinate position of the eye is (xk2, yk2).
[0030]
Then, the tracking unit 22 sets a tracking region centered on the newly detected eye coordinate position (xk2, yk2) (FIG. 7C). Thereafter, when the captured image is input again, the tracking unit 22 detects the position of the eye from the tracking region centered at (xk2, yk2). The coordinate position of the eye detected at this time is assumed to be (xk3, yk3).
[0031]
Then, the tracking unit 22 sets again a tracking region centered on the new eye coordinate position (xk3, yk3) (FIG. 7D). Thereafter, similarly, the tracking unit 22 detects the position of the eye from the tracking region.
[0032]
The tracking unit 22 sends the detected eye position data to the learning unit 23 and the aside detection unit 25 as shown in FIG. The learning unit 23 learns eye position data input in time series. Usually, a driver who is driving a vehicle visually recognizes a room mirror, left and right door mirrors, etc. in order to perform a confirmation operation. For this reason, the learning unit 23 can know the position of the eyes when viewing the positions of the room mirror and the left and right door mirrors by determining the amount of eye movement on the captured image.
[0033]
FIG. 8 is an explanatory diagram showing the situation when the driver visually recognizes the front, the rear view mirror, and the left and right door mirrors, and (a) shows the position of the driver's eyes when the driver is visually checking the front. Show. Further, (b) shows the position of the driver's eyes when viewing the rearview mirror, (c) shows the right door mirror, and (d) shows the driver's eye when viewing the left door mirror. The position of the eye is shown. Furthermore, (e) has shown the coordinate position of the eye according to a driver | operator's visual recognition location. In FIG. 8, an example of an image having 640 pixels in the X direction and 480 pixels in the Y direction will be described.
[0034]
As shown in FIGS. 8A to 8D, when the driver visually recognizes the front, the rear view mirror, and the left and right door mirrors, as shown in FIG. 8E, the coordinate position of the eye obtained from the image is Except when looking ahead, it is almost constant. Specifically, when the driver visually recognizes the front, the coordinate positions (x, y) of the eyes are dispersed in (260, 220) to (285, 240). Here, the center position of both eyes is the coordinate position of the eyes.
[0035]
On the other hand, when the driver is viewing the right door mirror, it is concentrated near (175,245), and when the driver is viewing the left door mirror, it is concentrated near (395,245). Further, when the driver is viewing the room mirror, the driver concentrates near (380, 220).
[0036]
In this way, the position of the driver's eyes at the time of viewing the mirror is concentrated in the same place to some extent, so that the amount of eye movement between the front viewing and the mirror viewing can be obtained by collecting data in advance. It can be obtained in a state that includes individual differences. That is, when the eye moves by a predetermined amount of movement in a predetermined direction, it can be used to estimate that one of the three mirrors is going to be visually recognized.
[0037]
Here, it is known that the residence time of the eyes of the driver when the mirror is viewed is an average of 0.7 seconds. FIG. 9 is an explanatory diagram showing the relationship between the staying time of the driver's eyes and the vehicle speed when the mirror is viewed.
[0038]
As shown in the figure, it can be seen that the residence time of the driver's eyes is concentrated at a location of about 0.7 seconds. It can also be seen that this time is not affected by the vehicle speed. Therefore, it is possible to specify which mirror is viewed based on the moving direction and moving amount and the eye retention time.
[0039]
As described above, the learning unit 23 learns the position of the eye on the captured image when the driver looks at the mirror, based on the moving direction and moving amount of the eye and the staying time of the eye. Then, the learning unit 23 sends the learned eye position data to the image region setting unit 24.
[0040]
The image region setting unit 24 sets a predetermined number of image regions having different looking-aside allowable times based on the eye position data learned by the learning unit 23. That is, an image area is set on the captured image based on the position of the eyes when viewing the left and right door mirrors and the room mirror.
[0041]
FIG. 10 is an explanatory diagram showing an example of a predetermined number of image regions with different look-ahead allowable times, (a) showing an example of a predetermined number of image regions, and (b) showing a look-ahead allowable time for each image region. An example is shown. In addition, the numerical value of the vertical direction and horizontal direction shown to Fig.10 (a) has shown the driver | operator's face direction angle. Further, although the image area is originally set on the captured image, in FIG. 10, for convenience of explanation, the image area based on the driver's field of view will be described. Furthermore, in the following description, a front area that the driver visually recognizes most during driving is defined as a front gaze area.
[0042]
The image area setting unit 24 sets, for example, five image areas A to E as shown in FIG. First, the image area setting unit 24 determines the right end of the image area A based on the position of the eye when viewing the room mirror. The image area setting unit 24 determines the left end and the lower end of the image area B based on the position of the eye when the right door mirror is visually recognized, and determines the right end and the image area C based on the position of the eye when the left door mirror is viewed. Determine the bottom edge. Further, the upper end of the image area E is determined based on the eye position when the left and right door mirrors are viewed.
[0043]
The image area setting unit 24 determines the left end of the image area D based on the tracking limit position in the right direction. FIG. 11 is an explanatory diagram of a method for setting the image area D by the image area setting unit 24. FIG. 11A shows the face orientation when the driver looks at the left door mirror, and FIG. It shows the face orientation when the driver looks at the right door mirror.
[0044]
As shown in FIG. 11A, the position of the driver's left eye when the driver looks at the left door mirror has almost reached the tracking limit. In other words, if the driver turns his face further to the left from the state of looking at the left door mirror, the left eye will not be imaged. Thus, when the left door mirror is viewed, the limit has been reached that allows the driver's left eye to be imaged.
[0045]
Further, as shown in FIG. 11B, the right eye may reach the tracking limit when the driver is viewing the right side. For example, the driver may be viewing the right further than the right door mirror when confirming an intersection. In this case, if the driver turns his face further to the right, the right eye will not be imaged. This position is the tracking limit position in the right direction. The image area setting unit 24 specifies a right limit position from the captured image, determines the left end of the image area D based on the tracking limit position, and sets the image area D.
[0046]
The tracking limit position in the right direction may be obtained from the captured image as described above, but may be obtained by another method. That is, the tracking limit position in the right direction may be obtained by calculation based on the tracking limit position when the left door mirror is viewed. In this case, if the tracking limit position in the left direction and the target position are obtained, the position of the tracking limit in the right direction is obtained.
[0047]
FIG. 12 is an explanatory diagram of a specific example of the image areas A to E set on the captured image. When the image region shown in FIG. 10A is replaced with a captured image, the result is as shown in FIG. As described above, these areas A to E are set based on the coordinate position of the eye when the rear view mirror is viewed and the coordinate position of the eye when the left and right door mirrors are viewed. The left and right tracking limit positions are also used as a reference.
[0048]
Refer to FIG. 10 again. The image region setting unit 24 sets different aside look allowance times for the image regions A to E allocated as described above. For example, as shown in FIG. 10B, the look-ahead allowable time for the image area D is smaller than the allowable look-ahead time for the image area B. This is because the face angle is larger in the image area D than in the image area B. Similarly, the allowable time for looking aside in the image area C is smaller than the allowable time for looking aside in the image area A.
[0049]
Further, the allowable time for looking aside in the image area B is smaller than the allowable time for looking aside in the image area A. When both areas A and B are visually recognized, the driver's face angle is substantially the same. However, a pillar is provided between the image area B and the forward gaze area. When the driver visually recognizes the image area B with this pillar, the front gaze area becomes difficult to see. For this reason, even if the face direction angles of both areas A and B are substantially the same, the allowable time for looking aside in image area B is shorter than the allowable time for looking aside in image area A.
[0050]
In addition, the allowable time for looking aside in the image area D is shorter than the allowable time for looking aside in the image area C. The face angle is substantially the same for both regions C and D. A pillar is installed between the image area B and the forward gaze area via the image area B. Similarly, pillars are also provided between the image area C and the forward gaze area. However, the left pillar is located farther from the driver than the right pillar. For this reason, the apparent pillar thickness from the driver is different. For the driver, the apparent pillar thickness is an obstacle to visual recognition of the forward gaze area. Therefore, the look-ahead allowable time of the image area D is smaller than the allowable look-ahead time of the image area C.
[0051]
Furthermore, the allowable time for looking aside in the image area E is substantially the same as the allowable time for looking aside in the image area D. For example, when the driver visually recognizes the installation position of the audio device or the like, the front gaze area becomes almost invisible because the heel is lowered. For this reason, the image area E has a shorter look-ahead allowable time, and is almost equivalent to the image area D that has the shortest look-ahead allowable time so far.
[0052]
The look-ahead detection unit 25 detects the driver's look-aside action based on the image areas A to E and the look-ahead allowable time set as described above. In other words, when the allowed look-ahead time elapses while the eye position remains in any of the image areas A to E, the look-ahead detection unit 25 detects the driver's look-aside action.
[0053]
Next, the operation of the armpit detection device 1 according to this embodiment will be described in detail. First, the imaging unit 10 of the armpit detection device 1 captures an area including the driver's face and sends the obtained captured image data to the processing device 20.
[0054]
And the processing apparatus 20 performs the process according to FIG. FIG. 13 is a flowchart showing a detailed operation of the processing apparatus 20 shown in FIG.
[0055]
As shown in the figure, first, the position detector 21 inputs captured image data (ST10). Thereafter, the position detection unit 21 determines whether or not an eye tracking region is set (ST11). If it is determined that the eye tracking area is set (ST11: YES), the process proceeds to step ST14.
[0056]
On the other hand, when it is determined that the eye tracking area is not set (ST11: NO), the position detection unit 21 detects the eye position from the entire image (ST12). Here, as described with reference to FIGS. 3 to 6, the position of the eye is detected. Then, the position detection unit 21 sends the eye position data to the tracking unit 22.
[0057]
Thereafter, the tracking unit 22 sets a tracking region based on the eye position data from the position detection unit 21 (ST13). Then, the tracking unit 22 detects eyes from within the tracking region based on newly input captured image data (ST14). Thereafter, the tracking unit 22 determines whether or not eye tracking is correctly performed (ST15).
[0058]
When it is determined that eye tracking is not performed correctly (ST15: NO), the tracking unit 22 clears the eye tracking region (ST16), and the process returns to step ST10. In this case, the tracking area is set again.
[0059]
On the other hand, when it is determined that eye tracking is being performed correctly (ST15: YES), the tracking unit 22 updates the eye tracking area (ST17). In other words, as described with reference to FIG. 7, the tracking area is updated based on the position of the eye.
[0060]
Thereafter, the learning unit 23 determines whether or not the learning of the eye position at the time of visually recognizing the predetermined position has ended (ST18). That is, it is determined whether or not all learning of the room mirror, the left and right door mirrors, and the tracking limit position in the right direction has been completed. When it is determined that the learning of the eye position at the time of viewing the predetermined position is not completed (ST18: NO), the learning unit 23 performs the eye position learning process (ST19). Then, the process returns to step ST10.
[0061]
When the above process is repeated and learning of the eye position is completed, “YES” is determined in step ST18. Thereafter, the image area setting unit 24 determines whether an image area is set (ST20). If it is determined that an image area is set (ST20: YES), the process proceeds to step ST22.
[0062]
On the other hand, when it is determined that no image area is set (ST20: NO), the image area setting unit 24 sets a predetermined number of image areas and sets different aside allowance times for these image areas (ST21). .
[0063]
After that, the side-by-side detection unit 25 performs a side-by-side detection process based on the eye position data from the tracking unit 22 and the data related to the image region from the image region setting unit 24 (ST22).
[0064]
If no side effect is detected by this process, the process returns to step ST10. On the other hand, when a look-aside is detected, the look-ahead detecting unit 25 sends data indicating that there is a look-aside to the alarm device 30. And the alerting | reporting device 30 performs alerting | reporting operation | movement (ST23). Thereafter, the process ends.
[0065]
In this way, detection of the driver's aside action is performed. As described above, in step ST21, a predetermined number of image regions having different look-ahead allowable times are set, and in step ST22, look-ahead detection is performed. As described with reference to FIG. 10, the side-viewing allowable time is determined in consideration of the installation position of the structure in the vehicle, for example, the pillar, and is more appropriate than the conventional one. It has become. Then, since the side-by-side detection is performed based on the appropriate side-by-side allowable time, an apparatus with high detection accuracy can be obtained.
[0066]
Next, details of the learning process (ST19) shown in FIG. 13 will be described with reference to FIGS. FIG. 14 is a flowchart showing details of the eye position learning process (ST19) shown in FIG. 13, and shows the learning process when the room mirror is viewed.
[0067]
As shown in the figure, first, the learning unit 23 determines whether or not the eye position has moved to the right by a predetermined amount M1 (ST30). If it is determined that the eye position has not moved to the right by the predetermined amount M1 (ST30: NO), the process proceeds to step ST32.
[0068]
On the other hand, when it is determined that the eye position has moved to the right by the predetermined amount M1 (ST30: YES), the learning unit 23 turns on the eye movement flag FL1 and performs other operations described with reference to FIGS. The movement flags FL2 to FL4 are turned off (ST31).
[0069]
Thereafter, the learning unit 23 determines whether or not the eye movement flag FL1 is on and the eye position stays at the position moved by the predetermined amount M1 for a predetermined time (ST32). When it is determined that the eye movement flag FL1 is not on or the eye position has not stayed at the position moved by the predetermined amount M1 (ST32: NO), the process shown in FIG. 14 ends.
[0070]
Here, if the position of the eye has not moved to the right by the predetermined amount M1 (when “NO” in step ST30), the movement flag FL1 is not turned on, so the processing shown in FIG. Will end. Further, even when the rightward movement is performed by the predetermined amount M1, if the staying time is short, the processing shown in FIG. 14 ends. The predetermined time in step ST32 is set to “0.7 seconds”, for example.
[0071]
In this way, in the learning process, a process for excluding data that should not be learned for eye positions is performed, and an image region is not set inadvertently in the subsequent process.
[0072]
By the way, when it is determined that the eye movement flag FL1 is on and the eye position stays at the position moved by the predetermined amount M1 for a predetermined time (ST32: YES), the learning unit 23 sets the eye movement flag FL1. Turn off (ST33). Then, the learning unit 23 determines whether or not the rectangular small area “a” is set at the position where the eye stays.
[0073]
When it is determined that the rectangular small area a is not set at the position where the eye stays (ST34: NO), the rectangular small area a is set at the staying position (ST35), and the processing shown in FIG. finish.
[0074]
On the other hand, when it is determined that the rectangular small area a is set (ST34: YES), the value of the counter α is counted up (ST36). Here, when the rectangular small area a is set, but the staying position of the eye is not within the rectangular small area a, the small area a is set again in step ST35 or the processing shown in FIG. 14 is ended. Etc. (not shown).
[0075]
Thereafter, the learning unit 23 determines whether or not the value of the counter α has reached a predetermined value (ST37). When it is determined that the value of the counter α has not reached the predetermined value (ST37: NO), the processing shown in FIG. 14 ends.
[0076]
On the other hand, when it is determined that the value of the counter α has reached a predetermined value (ST37: YES), the learning unit 23 sets, for example, the position of the small region a as the eye position when viewing the room mirror (ST38). Thereby, the learning unit 23 completes the learning of the eye position. Thereafter, the process shown in FIG. 14 ends.
[0077]
Incidentally, in parallel with the processing shown in FIG. 14, the processing shown in FIGS. 15 to 17 is also executed. 15 to 17 are flowcharts showing details of the eye position learning process (ST19) shown in FIG. 15 shows a learning process when the left door mirror is visually recognized, FIG. 16 shows a learning process when the right door mirror is visually recognized, and FIG.
[0078]
Steps ST40 to ST48 shown in FIG. 15 are the same as steps ST30 to ST38 shown in FIG. Similarly, Step ST50 to Step ST58 shown in FIG. 16 and Step ST60 to Step ST68 shown in FIG. 15 to 17, the moving direction and the moving amounts M2 to M4 are different in steps ST40, 50, and 60, respectively, and the predetermined time at the time of right rear confirmation is different in FIG.
[0079]
Naturally, the small areas b to d and the counters β to θ for counting up are also different.
[0080]
Next, the side look determination (ST22) shown in FIG. 13 will be described in detail. FIG. 18 is a detailed flowchart of the look-aside determination (ST22) shown in FIG. In FIG. 18, a process for determining whether or not an eye exists in the image area A will be described. Further, FIG. 18 shows a case where the eye position is stored in the image area A, but the same applies to the other image areas B to E.
[0081]
First, the armpit detection unit 25 determines whether or not an eye is present in the image area A (ST70). When it is determined that the eye is present in the image area A (ST70: YES), the armpit detection unit 25 determines whether the timer is operating (ST71).
[0082]
If it is determined that the timer is operating (ST71: YES), the process proceeds to step ST73. On the other hand, when it is determined that the timer is not in operation (ST71: NO), the armpit detection unit 25 starts the timer (ST72).
[0083]
Thereafter, the look-ahead detection unit 25 determines whether or not the time measured by the timer (that is, the staying time) exceeds the allowable look-ahead time set in the image area A (ST73). When it is determined that the allowable time for looking aside has not been exceeded (ST73: NO), the process shown in FIG. 18 ends.
[0084]
On the other hand, when it is determined that the allowable time for looking aside has been exceeded (ST73: YES), the looking-aside detection unit 25 outputs a warning and resets the timer (ST74). Then, the process shown in FIG. 18 ends.
[0085]
In step ST70, when it is determined that the eye is not present in the image area A (ST70: NO), the armpit detection unit 25 resets the timer without outputting an alarm (ST75), and the processing ends thereafter. To do.
[0086]
By the way, in this embodiment, the setting process (ST21) of the image areas A to E is performed by learning about all of the rearview mirror, the left and right door mirrors, and the tracking limit position in the right direction (including calculation by calculation). Is supposed to be executed. However, if at least one learning of the visual recognition position is completed, one of the image areas may be set based on the position, and the side-by-side determination may be started. In this case, the image region setting unit 24 sequentially sets the image region every time learning of the eye position on the captured image is completed.
[0087]
FIG. 19 is an explanatory diagram showing processing in the case where image areas are set step by step. As shown in the figure, first, the learning unit 23 determines whether or not the image area A has not been set and the learning of the eye position at the time of viewing the rearview mirror has ended (ST80). When it is determined that the image area A is not set and learning of the eye position at the time of viewing the rearview mirror has ended (ST80: YES), the image area setting unit 24 sets the image area F, and further An allowable time for looking aside is set (ST81). Then, the process shown in FIG. 19 ends.
[0088]
This image area F is, for example, as shown in FIG. FIG. 20 is an explanatory diagram illustrating an example of an image area that is set in stages. The image area F is an area including the image areas A and C and a part of the image area E. The reason why the image area F is set in step ST81 is that the boundary between the forward gaze area and the image area A can be set according to the position of the eye when the room mirror is viewed. In addition, since the boundary between the image area A and the image area C cannot be set only by the eye position when the room mirror is viewed, the area on the side having a larger face orientation angle than the image area A is included. Furthermore, since the lower end of the area cannot be set only by the eye position when viewing the room mirror, a part of the image area E is included.
[0089]
Again, a description will be given with reference to FIG. If it is determined that either the image area A is not set or the learning of the eye position at the time of viewing the rearview mirror is not satisfied (ST80: NO), the process proceeds to step ST82. To do. Then, the learning unit 23 determines whether or not the image area C is not set and the learning of the eye position at the time of viewing the left door mirror is finished (ST82).
[0090]
If it is determined that the image area C has not been set and the learning of the eye position at the time of viewing the left door mirror has ended (ST82: YES), the image area setting unit 24 sets the image area C, and further An allowable time to look aside at C is set (ST83).
[0091]
The image area C here is as shown in FIG. 20, and is the same as that described with reference to FIG. When the position of the eye at the time of viewing the left door mirror is determined, the right end and the lower end of the image area C are determined as described above. Therefore, the image area setting unit 24 determines the image area C and further sets an allowable time for looking aside.
[0092]
Thereafter, the image area setting unit 24 determines whether or not the image area E has been set (ST84). If it is determined that the image area E has been set (ST84: YES), the processing shown in FIG. 19 ends.
[0093]
On the other hand, when it is determined that the image area E has not been set (ST84: NO), the image area setting unit 24 sets the image area E and the allowable look-aside time for the area E (ST85). Then, the process shown in FIG. 19 ends.
[0094]
The image region E here is as shown in FIG. 20, and is the same as the person described with reference to FIG. When the position of the eye when viewing the left door mirror is determined, the upper end of the image area E can be determined as described above. For this reason, the image area setting unit 24 determines the upper end of the image area G based on the position of the eye when visually recognizing the left door mirror, and further sets the allowable look-aside time for the area G.
[0095]
Again, a description will be given with reference to FIG. When it is determined that either the image area C is not set or the learning of the eye position at the time of viewing the left door mirror is not satisfied (ST82: NO), the process proceeds to step ST86. To do. Then, the learning unit 23 determines whether or not the image region B is not set and learning of the eye position at the time of viewing the right door mirror is finished (ST86).
[0096]
When it is determined that the image area B has not been set and the learning of the eye position when viewing the right door mirror has ended (ST86: YES), the image area setting unit 24 sets the image area G, and further An allowable time aside for G is set (ST87).
[0097]
Here, the image area G is as shown in FIG. 20 and is an area having a size obtained by adding the image area B and the image area D. The reason why the image region G is set in step ST87 is that the boundary between the front gaze region and the image region B can be set according to the position of the eye when the right door mirror is viewed. Moreover, it is because the boundary between the image area B and the image area D cannot be set only by the position of the eye when viewing the right door mirror.
[0098]
Reference is again made to FIG. After setting the image region G and the sidewatch allowable time for the region G, the process proceeds to step ST84. Then, through the same processing as described above, the processing shown in FIG.
[0099]
If it is determined that either the image area B has not been set or the learning of the eye position at the time of viewing the right door mirror has not been satisfied (ST86: NO), the process proceeds to step ST88. Migrate to Then, the learning unit 23 determines whether or not the image region D is not set and learning of the eye position at the time of right rear viewing is finished (ST88).
[0100]
When it is determined that the image area D has not been set and the learning of the eye position during right rear viewing has been completed (ST88: YES), the image area setting unit 24 sets the image area D, and further An allowable time aside for D is set (ST89).
[0101]
The image region D here is as shown in FIG. 20, and is the same as that described with reference to FIG. When the position of the eye at the time of right rear viewing is determined, the boundary between the image region D and the image region B is determined as described above, so the image region setting unit 24 determines the image region D and further looks aside from the region D. Set the allowable time. Then, the process shown in FIG. 19 ends.
[0102]
If it is determined that either the image area D is not set or the learning of the eye position at the time of right rear viewing is not satisfied (ST88: NO), the process shown in FIG. Ends.
[0103]
In this way, according to the aside detection device 1 according to the present embodiment, a predetermined number of image regions having different aside allowance times are set with reference to the eye position on the captured image. Usually, a driver visually recognizes a relatively large amount of room mirror or the like. For this reason, the image region based on the position of the room mirror or the like can be set by using the position of the eye on the captured image as a reference. In addition, since it is based on the position of the room mirror or the like, it is also possible to assign a side-viewing allowable time taking into account a pillar or the like that blocks the driver's field of view to each of a predetermined number of image areas.
[0104]
Then, based on the allowable time for looking aside allocated to each image area and the eye position to be tracked, the looking aside is detected. For this reason, it is possible to perform a look-ahead detection based on a look-ahead allowable time allocated to each image area in conformity with the in-vehicle environment.
[0105]
Therefore, it is possible to improve the accuracy in detecting the driver's act of looking aside.
[0106]
The learning unit 23 learns the position of the eye on the captured image when the positions of the left and right door mirrors and the room mirror are viewed. Normally, during driving, the driver visually recognizes the positions of the left and right door mirrors and the room mirror. For this reason, the position of the detected eye increases when looking at the left and right door mirrors and the rearview mirror, and the amount of data obtained by detection is compared to when looking at other structures inside the vehicle. Become more. Therefore, the time until the image area is determined is shorter than that when other structures inside the vehicle are viewed. Accordingly, it is possible to shorten the time until the image area is set.
[0107]
The learning unit 23 learns the position of the eye on the captured image when the driver's eye reaches the tracking limit position. Usually, during driving, the driver checks the right rear side relatively frequently. For this reason, the amount of data obtained by detection is larger than when looking at other structures inside the vehicle. Therefore, the time until the image area is determined is shorter than that when other structures inside the vehicle are viewed. Accordingly, it is possible to shorten the time until the image area is set.
[0108]
Further, the learning unit 23 learns the position of the eye on the captured image when the driver's eyes have reached either the left or right tracking limit position. Then, based on the learned eye position, the eye position on the captured image when the driver's eye reaches the other tracking limit position is estimated. For this reason, even if the position of the eye on the captured image when the other tracking limit position is reached is not detected, the image area can be set based on the position obtained by estimation. Accordingly, it is possible to shorten the time until the image area is set.
[0109]
The image area setting unit 24 sequentially sets the image area every time the position of the eye on the captured image is learned. For this reason, since the image area is set without waiting for learning of the positions of all eyes, it is possible to shorten the time until the side-eye detection is performed.
[0110]
In the above embodiment, the process shown in FIG. 19 may be executed in place of steps ST19, ST20, ST21 shown in FIG.
[0111]
Next, a second embodiment of the present invention will be described. The armpit detection device 2 according to the second embodiment is the same as that of the first embodiment, but differs from that of the first embodiment in that an instruction unit (instruction means) 40 is provided.
[0112]
Hereinafter, the armpit detection device 2 according to the second embodiment will be described. FIG. 21 is a configuration diagram of the armpit detection device 2 according to the second embodiment of the present invention. The instruction unit 40 is operated to cause the learning unit 23 to learn the eye position. Specifically, the instruction unit 40 is configured by a switch or the like installed below the front of the vehicle.
[0113]
FIG. 22 is a functional block diagram of the armpit detection device 2 according to the second embodiment of the present invention. As shown at the same time, the instruction unit 40 is connected to the learning unit 23 in the processing device 20. The instruction unit 40 is operated to send a predetermined signal to the learning unit 23. And the learning part 23 which received this starts learning of the position of an eye.
[0114]
FIG. 23 is a flowchart showing detailed operations of the processing apparatus 20 shown in FIG. Note that steps ST90 to ST96 and steps ST100 to T103 in the figure are the same as steps ST10 to ST19 and steps ST20 to T23 shown in FIG.
[0115]
In step ST96, the learning unit 23 performs eye position learning processing. At this time, when the driver operates the instruction unit 40, a predetermined signal is transmitted.
[0116]
In step ST97, the learning unit 23 determines whether or not the instruction unit 40 has been operated by determining whether or not a predetermined signal has been transmitted. If it is determined that instruction unit 40 has not been operated (ST97: NO), the process returns to step ST90.
[0117]
On the other hand, when it is determined that the instruction unit 40 has been operated (ST97: YES), the learning unit 23 starts the learning process according to the instruction (ST98). The learning process performed at this time is the same as that shown in FIGS. 14 to 17, but unlike the process of step ST <b> 96, the position currently viewed by the driver is forcibly learned. That is, when the instruction unit 40 is operated while the driver is viewing the room mirror, the position is forcibly learned.
[0118]
Then, the learning unit 23 determines whether or not learning has been completed for the learning process performed in response to the operation of the instruction unit 40 (ST99). If it is determined that learning has not been completed (ST99: NO), the process returns to step ST90. On the other hand, when it is determined that learning is completed (ST99: YES), the image area setting unit 24 determines whether an image area is set (ST100). Thereafter, the same processing as steps ST20 to T23 in FIG. 13 is performed.
[0119]
As described above, according to the side-by-side detection device 2 according to the present embodiment, as in the first embodiment, it is possible to improve the accuracy in detecting the driver's side-by-side action, and the time until the image area is set. Can be shortened. In addition, the time required to detect a side look can be shortened.
[0120]
In addition, an instruction unit 40 is provided that causes the learning unit 23 to learn the eye position when operated. For this reason, the position of the eye is easily learned just by operating the instruction unit 40 by the driver, and the time until the setting of the image region is shortened. Therefore, it is possible to detect a side look early from the start of operation.
[0121]
Next, a third embodiment of the present invention will be described. The armpit detection device 3 according to the third embodiment is the same as that of the first embodiment, but includes a travel state detection unit (travel state detection unit) 50 and an allowable time change unit (allowable time change unit) 26. Thus, it is different from that of the first embodiment.
[0122]
Hereinafter, the aside detection device 3 according to the third embodiment will be described. FIG. 24 is a configuration diagram of the armpit detection device 3 according to the third embodiment of the present invention. The traveling state detection unit 50 detects the traveling state of the vehicle, and includes a vehicle speed sensor 51, an inter-vehicle distance sensor 52 (distance sensor), and a navigation device 53.
[0123]
The vehicle speed sensor 51 detects the vehicle speed of the vehicle, and includes, for example, a sensor provided on the output side of the automatic transmission, a wheel speed sensor provided to detect wheel rotation, and the like. The inter-vehicle distance sensor 52 detects the distance between the preceding vehicle and the host vehicle, and includes, for example, image processing using a CCD camera, an infrared camera, an ultrasonic sensor, a millimeter wave radar, a laser radar, and the like.
[0124]
Further, the navigation device 53 guides the vehicle to the destination by presenting the vehicle position, the destination, the route to the destination, and the like to the driver. The navigation device 53 has a function of presenting information related to the type of road on the map to the driver.
[0125]
FIG. 25 is a functional block diagram of the armpit detection device 3 according to the third embodiment of the present invention. As shown at the same time, the armpit detection device 3 according to the third embodiment has an allowable time changing unit 26 in the processing device 20.
[0126]
The permissible time changing unit 26 changes the permissible time for looking aside according to the output signal from the traveling state detecting unit 50. Specifically, the permissible time changing unit 26 shortens the aside allowance time as the vehicle speed increases based on the vehicle speed signal from the vehicle speed sensor 51. Further, the permissible time changing unit 26 shortens the aside allowance time as the host vehicle approaches the preceding vehicle based on the distance signal from the inter-vehicle distance sensor 52. Further, the permissible time changing unit 26 is based on the information regarding the type of road from the navigation device 53, when the host vehicle is traveling on a general road, rather than the permissible time for looking aside when the host vehicle is traveling on an automobile-only road. Shorten the allowable time for looking aside.
[0127]
Next, the operation in the case of changing the allowable time for looking aside based on the vehicle speed will be described. FIG. 26 is a flowchart showing a detailed operation of the processing device 20 shown in FIG. 24, and shows an example in which the aside allowance time is changed according to the vehicle speed. Further, FIG. 26 illustrates a case where the look-ahead allowable time of the image area A is changed.
[0128]
First, the allowable time changing unit 26 receives a vehicle speed signal from the vehicle speed sensor 51 and detects the vehicle speed (ST110). And the permissible time change part 26 calculates | requires the numerical value etc. according to the vehicle speed. Further, the allowable time changing unit 26 reads information on the allowable time for looking aside of the image area A set by the image area setting unit 24 from the aside detection unit 25. Then, based on the information such as numerical values and the read information on the allowable time for looking aside, the allowable time for looking aside is changed and set (ST111).
[0129]
In this change, the allowable time changing unit 26 changes the allowable time for looking aside using a predetermined arithmetic expression, map data, and the like. FIG. 27 is an explanatory diagram illustrating an example of a side allowance time according to the vehicle speed. Specifically, as shown in FIG. 27, the allowable time changing unit 26 shortens the sidewatch allowable time as the vehicle speed increases. Note that the areas A to E in FIG. 27 indicate the image areas A to E described with reference to FIG.
[0130]
Refer to FIG. 26 again. After the change of the time required for looking aside, the process proceeds to step ST112 and ends after the process after step ST112. The processing of step ST112 to step ST117 is the same as the processing of step ST70 to step ST75 shown in FIG.
[0131]
Next, the operation in the case of changing the sidewatch allowable time based on the inter-vehicle distance will be described. FIG. 28 is a flowchart showing the detailed operation of the processing device 20 shown in FIG. 24, and shows an example in which the aside look-ahead time is changed according to the inter-vehicle distance. Further, FIG. 26 illustrates a case where the look-ahead allowable time of the image area A is changed.
[0132]
First, the permissible time changing unit 26 inputs a distance signal from the inter-vehicle distance sensor 52 and detects the distance to the preceding vehicle (ST120). And the permissible time change part 26 calculates | requires the numerical value etc. according to the distance between vehicles. Further, the allowable time changing unit 26 reads information on the allowable time for looking aside of the image area A set by the image area setting unit 24 from the aside detection unit 25. Then, based on the information such as numerical values and the read information on the allowable time for looking aside, the allowable time for looking aside is changed and set (ST121).
[0133]
In this change, the allowable time changing unit 26 changes the allowable time for looking aside using a predetermined arithmetic expression, map data, and the like. FIG. 29 is an explanatory diagram showing an example of a side allowance time according to the vehicle speed. Specifically, the permissible time changing unit 26 shortens the permissible time for looking aside as the host vehicle approaches the preceding vehicle as shown in FIG. Note that the areas A to E in FIG. 27 indicate the image areas A to E described with reference to FIG.
[0134]
Refer to FIG. 28 again. After changing the sidewatch allowable time, the process proceeds to step ST122, and ends after the process after step ST122. The processing of step ST122 to step ST127 is the same as the processing of step ST70 to step ST75 shown in FIG.
[0135]
Next, the operation in the case of changing the aside allowance time based on the information regarding the type of road will be described. FIG. 30 is a flowchart showing the detailed operation of the processing device 20 shown in FIG. 24, and shows an example in the case where the look-ahead allowable time is changed based on the road type information. In addition, FIG. 30 illustrates a case where the look-ahead allowable time of the image area A is changed.
[0136]
First, the permissible time changing unit 26 inputs road type information from the navigation device 53, and determines whether or not the own vehicle has entered an automobile-only road from a general road (ST130).
[0137]
If it is determined that the host vehicle does not enter the automobile road from a general road (ST130: NO), the process proceeds to step ST132. On the other hand, when it is determined that the host vehicle has entered an automobile-only road from a general road (ST130: YES), the allowable time changing unit 26 provides information on the allowable time for looking aside in the image area A set by the image area setting unit 24. Read from the armpit detector 25. Then, the read sideward allowable time is set long. Specifically, the permissible time changing unit 26 sets “aside-looking allowable time = aside-looking allowable time × 1.2” and sets the obtained new aside-looking allowable time (ST131).
[0138]
Thereafter, the permissible time changing unit 26 inputs the road type information from the navigation device 53, and determines whether or not the own vehicle has entered a general road from the car-only road (ST132).
[0139]
When it is determined that the host vehicle does not enter the general road from the automobile exclusive road (ST132: NO), the process proceeds to step ST134. On the other hand, when it is determined that the host vehicle has entered a general road from an automobile-only road (ST132: YES), the permissible time changing unit 26 obtains information on the permissible time to look aside in the image area A set by the image area setting unit 24. Read from the armpit detector 25. Then, the read aside allowance time is set short. Specifically, the permissible time changing unit 26 sets “aside-looking allowable time = aside-looking allowable time × 0.83”, and sets the obtained new aside-looking allowable time (ST133).
[0140]
After changing the aside allowance time, the process proceeds to step ST134 and ends after the process after step ST134. The processes in steps ST134 to ST139 are the same as the processes in steps ST70 to ST75 shown in FIG.
[0141]
In this way, according to the armpit detection device 3 according to the present embodiment, as in the first embodiment, it is possible to improve accuracy in detecting the driver's armpit act, and the time until the setting of the image area Can be shortened. In addition, the time required to detect a side look can be shortened.
[0142]
Further, the traveling state of the vehicle is detected, and the allowable time for looking aside is changed according to the detection result. For this reason, not only the vehicle interior environment but also the environment around the vehicle and the like can be taken into consideration to change the allowable time for looking aside. Therefore, it is possible to further improve the accuracy in detecting the driver's aside action.
[0143]
Further, based on the vehicle speed signal from the vehicle speed sensor 51, the aside allowance time is shortened as the vehicle speed increases. For this reason, for example, when the vehicle speed is high, even if the driver is visually recognizing the vicinity of the forward gaze area, the allowable time for looking aside can be shortened, and the looking aside can be detected more appropriately.
[0144]
In addition, based on the distance signal from the inter-vehicle distance sensor 52, the aside allowance time is shortened as the host vehicle becomes closer to the preceding vehicle. For this reason, for example, when the distance to the preceding vehicle is short, even if the driver visually recognizes the vicinity of the front gaze area, the allowable time for looking aside can be shortened, and the looking aside can be detected more appropriately. it can.
[0145]
In general, ordinary roads have pedestrians and signals, and automobile roads have no pedestrians and signals. Thus, the actual situation is that ordinary roads have more opportunities to stop vehicles than motorways. In addition, automobile roads often have wider roadside belts. For this reason, it is possible to more appropriately detect a side effect by shortening the time required for looking aside when traveling on a general road than when traveling on an automobile-only road.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of an armpit detection device according to a first embodiment of the present invention.
FIG. 2 is a functional block diagram of the armpit detection device according to the first embodiment of the present invention.
FIG. 3 is an explanatory diagram of an initial process performed when a position detection unit detects an eye position.
FIG. 4 is an explanatory diagram showing a state when a position detection unit extracts a predetermined pixel.
FIG. 5 is an explanatory diagram showing a state when pixels adjacent in the horizontal direction of the image are grouped.
FIG. 6 is an explanatory diagram showing a state after zoning by a position detection unit.
FIGS. 7A and 7B are explanatory diagrams showing the tracking of the eye position by the tracking unit, where FIG. 7A shows an initial tracking area, and FIG. 7B shows the detected eye after setting the initial tracking area; (C) shows the tracking area set based on the detected eye position, and (d) shows the eye position detected from within the newly set tracking area.
FIG. 8 is an explanatory diagram showing a situation when the driver visually recognizes the front, the rear view mirror, and the left and right door mirrors, and FIG. (B) shows the position of the driver's eye when viewing the rearview mirror, (c) shows the position of the driver's eye when viewing the right door mirror, (d) The position of the driver's eyes when viewing the left door mirror is shown, and (e) shows the coordinate position of the eyes according to the driver's viewing location.
FIG. 9 is an explanatory diagram showing the relationship between the staying time of the driver's eyes and the vehicle speed when the mirror is viewed.
FIGS. 10A and 10B are explanatory diagrams showing examples of a predetermined number of image areas with different looking-aside allowable times; FIG. 10A shows an example of a predetermined number of image areas, and FIG. An example is shown.
11A and 11B are explanatory diagrams of a method for setting an image area by an image area setting unit, in which FIG. 11A shows the state of the face when the driver looks at the left door mirror, and FIG. Shows the face orientation when looking at the right door mirror.
FIG. 12 is an explanatory diagram of a specific example of an image area set on a captured image.
13 is a flowchart showing a detailed operation of the processing apparatus shown in FIG.
14 is a flowchart showing details of the eye position learning process (ST19) shown in FIG. 13, and shows the learning process when the rearview mirror is viewed. FIG.
FIG. 15 is a flowchart showing details of the eye position learning process (ST19) shown in FIG. 13, and shows the learning process when the left door mirror is viewed;
16 is a flowchart showing details of eye position learning processing (ST19) shown in FIG. 13, and shows learning processing when the right door mirror is viewed.
FIG. 17 is a flowchart showing details of the eye position learning process (ST19) shown in FIG. 13, showing the learning process at the time of right rear viewing;
18 is a detailed flowchart of aside look determination (ST22) shown in FIG.
FIG. 19 is an explanatory diagram showing processing when image regions are set in stages.
FIG. 20 is an explanatory diagram illustrating an example of an image region that is set in stages.
FIG. 21 is a configuration diagram of an armpit detection device according to a second embodiment of the present invention.
FIG. 22 is a functional block diagram of an armpit detection device according to a second embodiment of the present invention.
FIG. 23 is a flowchart showing detailed operations of the processing apparatus shown in FIG. 21;
FIG. 24 is a configuration diagram of an armpit detection device according to a third embodiment of the present invention.
FIG. 25 is a functional block diagram of an armpit detection device according to a third embodiment of the present invention.
FIG. 26 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 24, and shows an example in the case where the look-ahead allowable time is changed according to the vehicle speed.
FIG. 27 is an explanatory diagram showing an example of a side allowance time according to the vehicle speed.
FIG. 28 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 24, and shows an example in the case where the look-ahead allowable time is changed according to the inter-vehicle distance.
FIG. 29 is an explanatory diagram showing an example of a side allowance time according to the vehicle speed.
30 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 24, and shows an example in the case of changing the allowable time for looking aside based on road type information.
[Explanation of symbols]
1-3 ... armpit detection device
10: Imaging unit (imaging means)
20 ... Processing device
21: Position detection unit
22 ... Tracking unit (tracking means)
23. Learning part (learning means)
24: Image area setting unit (image area setting means)
25 .. Aside detection unit (side detection means)
26 ... Allowable time changing section (allowable time changing means)
30 ... Alarm
40. Instruction unit (instruction means)
50 ... Traveling state detection unit (running state detection means)
51. Vehicle speed sensor
52. Inter-vehicle distance sensor
53. Navigation device
AG ... Image area

Claims (11)

Photographing means for photographing the face of the driver of the vehicle;
Tracking means for tracking the position of the driver's eyes based on the captured image including the driver's face obtained by the imaging means;
Learning means for determining the movement amount and the staying time of the eye position output in time series from the tracking means, and learning the eye position on the captured image when the driver sees the predetermined position;
Image region setting means for setting a predetermined number of image regions with different aside allowance times based on the position of the eye on the captured image learned by the learning means;
Based on the allowable time for looking aside allocated to the image area set by the image area setting means,
Equipped with a,
The learning means learns the position of the eye on the captured image when viewing the positions of the left and right door mirrors and the room mirror as the predetermined position,
The image area setting means determines the right end of the first image area (A) adjacent to the right of the front gaze area based on the position of the eye when the rear view mirror is viewed, and sets the right position of the eye when the right door mirror is viewed. Based on the left end and the lower end of the second image area (B) adjacent to the left of the front gaze area based on the position of the eye when viewing the left door mirror, to the right of the first image area (A) An armpit detection device that determines a right end and a bottom end of an adjacent third image region (C) . The armpit detection device according to claim 1, further comprising an instruction unit that is operated to cause the learning unit to learn an eye position. Traveling state detecting means for detecting the traveling state of the vehicle;
In accordance with an output signal from the traveling state detection means, an allowable time changing means for changing the allowable time for looking aside;
The armpit detection device according to claim 1, further comprising: The traveling state detection means includes a vehicle speed sensor that detects a vehicle speed of the vehicle,
4. The armpit detection device according to claim 3, wherein the permissible time changing means shortens the armpit allowable time as the vehicle speed increases based on a vehicle speed signal from a vehicle speed sensor. The traveling state detection means includes a distance sensor that detects a distance between the preceding vehicle and the host vehicle,
5. The armpit look according to claim 3, wherein the permissible time changing means shortens the armpit allowance time as the host vehicle approaches the preceding vehicle based on a distance signal from a distance sensor. Detection device. The travel state detection means includes a navigation device that presents information about the type of road on the map to the driver,
The permissible time changing means is based on the information on the type of road from the navigation device, and the allowable time for looking aside when traveling on a general road rather than the allowable time for looking aside when the vehicle is traveling on an automobile-only road. 6. The armpit detection device according to claim 3, wherein the armpit detection device according to claim 3 is shortened. The length of the sideward allowable time is set such that the sideward allowable time of the second image area (B) is set shorter than that of the first image area (A), and the sideward allowable time of the third image area (C) is set. The aside detection device according to any one of claims 1 to 6, wherein the time is set to be shorter than the allowable side aside time of the first image area (A). The said learning means learns the position of the eye on a captured image when a driver | operator's eyes have reached the tracking limit position when the driver looks at the predetermined position. Item 8. The armpit detection device according to any one of Item 7. The learning means learns the position of the eye on the captured image when the driver's eyes have reached one of the left and right tracking limit positions when the driver sees the predetermined position, and The position of the eye on the captured image when the driver's eye reaches the other tracking limit position is estimated based on the position, according to any one of claims 1 to 8. Aside detection device. 10. The aside look according to claim 1, wherein the image area setting unit sequentially sets the image area every time the position of the eye on the captured image is learned. Detection device. Based on the captured image including the driver's face obtained by photographing the driver's face, the driver's eye position is tracked, and the movement amount and retention of the driver's eye position specified by the tracking A predetermined number of images with different time-tolerance allowances based on the positions of the eyes on the captured image obtained by determining the time and learning the position of the eye when the driver views the predetermined position An armpit detection device that sets a region and performs armpit detection based on the armpit allowable time of the image region ,
The learning learns the position of the eye on the captured image when viewing the positions of the left and right door mirrors and the room mirror as the predetermined position,
The image area is set by determining the right end of the first image area (A) adjacent to the right of the front gaze area based on the position of the eye when viewing the rearview mirror, and setting the eye position when viewing the right door mirror. Based on the left end and the lower end of the second image area (B) adjacent to the left of the front gaze area based on the position of the eye when viewing the left door mirror, to the right of the first image area (A) An armpit detection device that determines a right end and a bottom end of an adjacent third image region (C) .

Images