JPH08313632A - Alarm generator, alarm generating method and vehicle mounting alarm generator - Google Patents

Abstract

PURPOSE: To alarm an operator in early stage by discriminating an object detected by means of a laser radar and delivering an alarm in early stage when the detected object is a pedestrian. CONSTITUTION: A two-dimensional scanner 14 scans a pulse laser light projected from an LD 13 in response to a signal received from a control circuit 11 and delivers the distance, horizontal angle and vertical angle of an object to a signal processor 20 every time when the object is detected. The signal processor 20 decides whether the detected object is a pedestrian or an obstacle and a different alarm must be delivered. When it is decided that the detected object is a person, the risk of collision is compared with a predetermined threshold value and when the threshold value is not exceeded, an alarm for person is delivered from an alarm 19 otherwise no alarm is delivered.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention detects a person (pedestrian) existing in front of a vehicle and informs a driver of the person of the presence of the person, and a vehicle equipped with the alarm generating apparatus. Regarding

[0002]

BACKGROUND ART There is an inter-vehicle distance detecting device using a laser radar as a device for detecting an obstacle such as a vehicle existing in front of a vehicle. The inter-vehicle distance detecting device detects obstacles by projecting laser light and detecting reflected light of the laser light from a reflector provided at a rear portion of a vehicle or the like. Therefore, there are the following problems with regard to pedestrians, people riding bicycles, etc., to which the driver should be careful.

Since a reflector is provided at the rear of the bicycle, the inter-vehicle distance detecting device can detect a bicycle (a person riding a bicycle) by detecting the reflected light from the reflector. .

Further, for a pedestrian, the inter-vehicle distance detecting device cannot usually detect, and the laser light may be reflected depending on the clothing of the pedestrian. Therefore, the reflected light should be detected by the inter-vehicle distance detecting device. May be able to detect pedestrians. However, when the resolution of the inter-vehicle distance detection device is about ± 1 m and the operation cycle is about 0.1 seconds, the moving speed (walking speed) of the person is sufficiently small and it is detected as a stationary object. Therefore, the pedestrian cannot be distinguished from other stationary objects such as roadside reflectors and signboards.

[0005]

DISCLOSURE OF THE INVENTION It is an object of the present invention to detect a pedestrian (person) existing in front of a vehicle and to notify the driver of the vehicle of the presence of the pedestrian.

The first aspect of the present invention provides an alarm generating device and method for generating an alarm based on an obstacle such as a vehicle or a two-wheeled vehicle and a relative position to a pedestrian. A second aspect of the present invention provides an alarm generation device and method for detecting a pedestrian based on an image obtained by imaging and issuing an alarm when the pedestrian is detected. The alarm generating device according to the first invention and the alarm generating device according to the second invention are mounted on a vehicle.

The alarm generation device according to the first aspect of the present invention is an object position detecting means for detecting a detected object existing in front of a vehicle equipped with the alarm generation device and measuring a relative position of the detected detected object. Person judging means for judging whether the detected object is related to a person or an obstacle based on the relative position of the detected object detected by the object position detecting means, and detected by the object position detecting means It is determined whether or not an alarm is to be issued based on the detected relative position of the detected object, and when the person determination means determines that the detected object is related to a person, the detected object is related to an obstacle. It is determined that the alarm generation determination means that determines to generate an alarm from a position farther than when it is determined to be present, and that the above alarm generation determination means should generate an alarm. And a warning device for generating an alarm in response to the fact.

The alarm generation method according to the first aspect of the present invention detects an object to be detected existing in front of the vehicle, measures the relative position of the detected object to be detected, and based on the detected relative position of the object to be detected. It is determined whether the detected object is related to a person or an obstacle, and whether or not an alarm is issued is determined based on the relative position of the detected object. When it is determined that it is related to a person, it is determined that the detected object is to be issued a warning from a position farther than when it is determined to be related to an obstacle. An alarm is generated.

According to the first aspect of the present invention, the detected object existing in front of the vehicle is detected, and the relative position of the detected object is measured. For example, a vehicle, an obstacle such as a motorcycle, an obstacle provided on the roadside, a person (pedestrian), or the like is detected as the detected object. For detecting the object to be detected, for example, a detection device such as a laser radar using laser light is used.
When the detected object is detected, it is determined whether the detected object is a person or another obstacle based on the relative position of the detected object. Next, based on the relative position of the detected object, it is determined whether or not to issue an alarm. If the object to be detected is a person, an alarm is issued from a position farther than that of other obstacles.

Therefore, when a pedestrian is detected, an alarm is sounded from a position farther than when other obstacles are detected, and this alarm causes the driver of the vehicle to detect the presence of a pedestrian from an early stage. I can know. This allows the driver to pay attention to the pedestrian and easily take an action (operation) to protect the pedestrian.

Another alarm generating device according to the first aspect of the invention changes the type of alarm depending on whether the detected object is a person or another obstacle. This alarm generating device detects an object to be detected existing in front of a vehicle equipped with the alarm generating device and detects the relative position of the detected object to be detected by the object position detecting means and the object position detecting means. A person judging means for judging whether the detected object is related to a person or an obstacle based on the relative position of the detected object. When it is determined that the first alarm is to be issued, when the person determining means determines that the detected object is related to an obstacle, whether to issue the second alarm is detected. And an alarm device for issuing a first or second alarm in response to the alarm generation determining means determining that the alarm should be issued. It is equipped with a.

An alarm generating method corresponding to this alarm generating device detects an object to be detected existing in front of the vehicle, measures a relative position of the detected object to be detected, and detects the relative position of the detected object to be detected. Based on this, it is determined whether the detected object is related to a person or an obstacle, and if it is determined that the detected object is related to a person, whether to issue the first alarm, When it is determined that the detected object is related to an obstacle, whether or not to issue the second alarm is determined based on the relative position of the detected object, and in response to the determination that the alarm should be issued. To generate the first or second alarm.

According to the present invention, it is determined based on the detected relative position of the detected object whether the detected object is a person or an obstacle. As a result, whether the first alarm is issued when the detected object is related to a person or not is issued when the detected object is related to other obstacles. Whether or not it is determined based on the relative position of the detected object. The first alarm is, for example, louder than the second alarm.

Therefore, by changing the first alarm (for pedestrians) and the second alarm (for other obstacles),
The driver can easily determine whether the object (object to be detected) present in front is a pedestrian or another obstacle. As a result, the driver can quickly know the existence of the pedestrian, and can easily take measures for avoiding danger to the pedestrian.

In one embodiment of the first aspect of the invention, a vehicle speed sensor for detecting the speed of a vehicle equipped with an alarm generation device, and the vehicle speed detected by the vehicle speed sensor and the relative position of the object to be detected. Relative speed calculation means for calculating the relative speed of the detected object based on the above is further provided, and the alarm generation judgment means makes a judgment about the alarm using the relative speed in addition to the relative position of the detected object. It is a thing.

According to this embodiment, the speed of the vehicle equipped with the alarm generation device is measured, and the relative speed of the detected object is detected based on the speed of the vehicle and the relative position of each detected object. . The relative speed of the detected object is used to determine whether to issue an alarm. In determining whether to issue an alarm, an alarm is issued from a distant position when the detected object is approaching, and an alarm is issued from a position closer than when the detected object is approaching. To occur. Therefore, when there is a high risk of colliding with the detected object, such as when the detected object approaches, an alarm will be issued from a position farther away than when the object moves away. Therefore, the alarm can be more appropriately generated.

Next, an alarm generating device and method according to the second invention will be described.

An alarm generating device according to a second aspect of the present invention includes an image pickup means for picking up an image of a subject including a detected object and outputting image data representing the subject, the first image data output from the image pickup means, and the first image data. An image memory for storing the second image data output from the image pickup means after a lapse of a predetermined time from the image pickup of one image data, and a detected object based on the first image data and the second image data stored in the image memory Target image creating means for creating target image data representing an object,
A person judgment means for judging whether or not the detected object represented by the target image data created by the target image creation means is a person, and an alarm is issued in response to the person judgment means being a person. Equipped with a warning device.

In the alarm issuing method according to the second aspect of the present invention, an image of a subject including an object to be detected is imaged to obtain image data representing the object, the first image data obtained by the image pickup, and the first image data. The second image data obtained by the imaging after a lapse of a predetermined time from the image capturing is stored in the image memory, and the detected object is detected based on the first image data and the second image data stored in the image memory. Create target image data to represent, determine whether the detected object represented by the created target image data is a person,
An alarm is generated in response to the determination that the detected object is a person.

According to the second aspect of the present invention, the subject including the object to be detected is imaged by the image pickup means, and the image data representing the subject is output. The first image data captured first and the second image data captured after a lapse of a predetermined time from the imaging of the first image data are stored in the image memory. Target image data representing the detected object is created based on the first image data and the second image data stored in the image memory. Based on the target image data, it is determined whether the detected object represented by the target image data is a person. When it is determined that the detected object is a person (pedestrian), an alarm is sounded.

Therefore, when a pedestrian is detected, an alarm is sounded, so that the driver can know the existence of the pedestrian, so that the pedestrian can easily take measures to avoid danger.

In an embodiment of the second invention, the target image creating means stores the first image stored in the image memory.
Perspective conversion means for generating perspective image data by performing two-step perspective conversion on the image surface of the second image data, perspective image data generated by the perspective conversion means,
Binary image creating means for creating difference image data with the second image data stored in the image memory, and creating binary image data by binarizing the difference image data with a predetermined threshold value, binary image creating Of the second image data stored in the image memory, window determination means for extracting target images by subjecting the binary image data created by the means to labeling processing, and determining windows surrounding each target image, This is realized by an image creating unit that outputs the second image data included in each window as the target image data.

According to this embodiment, perspective image data obtained by perspectively converting the first image data onto the image surface of the second image data is created, and the difference image data between the perspective image data and the second image data is set to a predetermined threshold value. Binarized image data is created by. Labeling processing is performed on the binary image data, target images representing the detected object are extracted, and a window surrounding each target image is determined. Of the second image data, the second image data included in each window is output as target image data. In the difference image between the perspective image subjected to the two-stage perspective transformation and the second image, an image representing a plane such as a road almost disappears, and only an image representing a solid such as a vehicle or a pedestrian remains. Therefore, by binarizing the difference image and applying labeling processing,
It is possible to extract images that respectively represent three-dimensional objects (detected objects) such as pedestrians. The second image data in the window including the image of each detected object includes target image data representing the detected object.

According to another embodiment of the second aspect of the invention, the person judging means extracts the horizontal edge and the vertical edge of the detected object based on the target image data created by the target image creating means. The height and width of the detected object are calculated based on the extraction means and the horizontal and vertical edges of the detected object extracted by the edge extraction means, and the detected object is a person based on the height and width. It is realized by a target determining unit that determines whether or not there is.

According to this embodiment, the horizontal edge and the vertical edge of the detected object are extracted based on the target image data representing the detected object. The height and width of the detected object are detected based on the horizontal edge and the vertical edge of the detected object. Whether the detected object is a person is determined based on the height and the width. Since a person (pedestrian) and an obstacle such as a vehicle have different heights and widths, it is determined whether the detected object is a pedestrian or another obstacle based on these heights and widths. be able to.

[0026]

【Example】

First Embodiment The alarm generation device in the first embodiment detects an obstacle including a pedestrian using a two-dimensional scanning laser radar and issues an alarm according to the distance between them.

FIG. 1 is a block diagram showing the electrical construction of an alarm generator mounted on a vehicle. A vehicle equipped with this alarm generation device is called a host vehicle.

The laser radar 10 is mounted on a vehicle so as to project a laser beam toward the front of the vehicle. The laser radar 10 includes a portion that projects the projected light and a portion that receives the reflected light. As shown in FIG. 2, the laser radar 10 is mounted on the front portion of the vehicle 1, for example, at or near the bumper 1A. It is not necessary to expose the entire laser radar 10 to the outside of the vehicle body of the vehicle 1, as long as at least the laser light emission window and the reflected light incidence window are opened. Various signal processing circuits will generally be provided inside the vehicle body.

Since obstacles such as vehicles, two-wheeled vehicles, and bicycles are provided with reflectors at their rear portions, the obstacles can be detected by receiving the laser light reflected by the reflectors. On the other hand, for pedestrians, in order to detect pedestrians, as shown in FIG. 3, materials having retroreflective characteristics (referred to as reflective materials) 5H and 5L are respectively placed on a part close to a person's head and feet. Install it in two places close to each other. The parts near the head are, for example, hats, clothes collars, etc., and the parts near the legs are, for example, shoes,
For example, the hem of trousers or the hem of a skirt. In addition to the parts near the head and the legs, the reflective material may be applied to other parts, for example, the reflective material may be applied to the arm as an armband.

The overall operation of the laser radar 10 is controlled by the control circuit 11. The control circuit 11 generates a series of light-emission pulses having a constant period, and the laser diode (L
D) It is given to the drive circuit 12. The period of the light emission pulse is a time longer than the time required for light to make a round trip over the maximum measurable distance.

The control circuit 11 also generates a horizontal scanning angle signal and a vertical scanning angle signal for scanning the laser beam in a predetermined scanning angle range (detection area) in the horizontal direction and the vertical direction, respectively. Give to the scanner 14. The scanning angle range is, for example, 400 [mrad] in the horizontal direction and 70 [mrad] in the vertical direction as shown in FIG.
The scanning angle range is divided into 80 and 14 areas in each direction, that is, into 80 × 14 areas. One area is 5 × 5 [mrad ² ], and laser light is projected onto each area.

The LD drive circuit 12 responds to the light emission pulse given from the control circuit 11 by a laser diode (L
D) Drive 13 As a result, the pulsed laser light is projected from the LD 13.

The two-dimensional scanner 14 scans the pulsed laser light projected by the LD 13 in response to the horizontal scanning angle signal and the vertical scanning angle signal supplied from the control circuit 11, and emits the pulsed laser light in the above-mentioned scanning angle range. . FIG. 5 is a block diagram showing the detailed structure of the two-dimensional scanner.

The pulsed laser light from the LD 13 is reflected by the horizontal scanning mirror 41 to be scanned in the horizontal direction, and further reflected by the vertical scanning mirror 43 to be scanned in the vertical direction. Horizontal scanning mirror 41
Is rotated by the horizontal scanning motor 42 at an angle based on the horizontal scanning angle signal given from the control circuit 11, and the vertical scanning mirror 41 is moved at an angle based on the vertical scanning angle signal given from the control circuit 11 by the vertical scanning motor 42. Is rotated.
The mirrors 41 and 43 are galvanometer mirrors, for example.

For the two-dimensional scanning of the pulsed laser light, a polygon mirror may be used in addition to the two-dimensional scanner shown in FIG. 5, and a collimator direct-acting type for rotating a collimator for projecting the laser light may be used. You may use it. Other materials may also be used.

The rotation angles of the horizontal scanning mirror 41 and the vertical scanning mirror 43 are detected by the scan angle detection circuit, and the horizontal angle detection signal and the vertical angle detection signal representing those angles are given to the control circuit 11.

The reflected light of the laser light is reflected by the photodiode (P
D) The light received by the 16 and the light receiving signal of the PD 16 is received by the light receiving circuit.
The level is discriminated by 17 at a predetermined threshold. When the level of the light receiving signal exceeds the threshold value, the light receiving timing signal is output from the light receiving circuit 17 to the control circuit 11.

The PD 16 may be a multi-division PD (PD divided in the horizontal direction or in a matrix). Regarding detection of reflected light using this multi-division PD, Japanese Patent Application No.
No. 196217.

In the control circuit 11, the delay time from the projection timing of the laser light based on the light emission pulse to the reception timing of the reflected light based on the light reception timing signal is measured, and based on this delay time, the own vehicle (this laser The distance r from the vehicle 1) equipped with the radar 10 to the detected object is calculated. When the detected object is detected (when the distance r to the detected object is calculated), the horizontal angle θ and the vertical angle are determined based on the horizontal angle detection signal and the vertical angle detection signal provided from the scan angle detection circuit 15. φ is calculated respectively. Every time a detected object is detected, the distance r, horizontal angle θ and vertical angle φ of the detected object are signal processing device.
Given to 20.

The traveling speed (vehicle speed) v of the host vehicle is a vehicle speed sensor.
It is detected by 18 and provided to the signal processor 20.

The signal processing device 20 detects the detected object based on the distance r, the horizontal angle θ and the vertical angle φ of the detected object detected by the laser radar 10, and the vehicle speed v detected by the vehicle speed sensor 18. It is determined whether it is a pedestrian or another obstacle, and whether a different alarm is issued depending on whether it is a pedestrian or another obstacle.

The signal processing device 20 is realized by a programmed computer. The signal processing device 20 may be entirely realized by hardware. Further, part of the signal processing device 20 can be realized by hardware and the other part can be realized by software.

FIG. 6 is a flow chart showing a processing procedure for alarm generation in the signal processing device 20.

FIG. 7 shows the structure of data stored in the signal processing device 20, which is data relating to the detected object detected when the laser beam is scanned once within the scanning angle range. In this figure, N detected objects are shown.

In the signal processing device 20, the polar coordinate system represented by the distance r of the object to be detected, the horizontal angle θ and the vertical angle φ is converted into an orthogonal coordinate system (step 31). As shown in FIG. 7, the position of the laser radar 10 is the origin O, the horizontal direction is the X axis, the vertical direction is the Z axis, and the traveling direction of the vehicle 1 is Y.
The axis. The Y axis is parallel to the road surface at the height position of the laser radar 10. When the height reference Z = 0 on the road surface (ground), the laser radar 10 uses the bumper 1A of the vehicle 1.
Since it is attached to, the height ΔZ corresponding to the height position where it is attached may be used for correction.

The position (x, y, z) of the detected object represented by the orthogonal coordinate system is expressed by the following equation by the distance r of the detected object, the horizontal angle θ and the vertical angle φ.

X = r · cos θ (1) y = r · sin θ · sin φ (2) z = r · cos φ (3)

Since one object may be detected as two or more detected objects, the processing of combining these detected objects into one may be performed before or after the coordinate conversion.

When this summarizing process is performed before coordinate conversion, the distance r, the horizontal angle θ, and the vertical angle φ are each within a predetermined range with respect to one detected object as a reference. Then, it is determined that they are detected from one target, and a plurality of detected objects detected from one target are detected based on their distance r, horizontal angle θ, and vertical angle φ. The distance r, horizontal angle θ and vertical angle φ of the detected objects are calculated.
For example, the average is calculated.

When the summarizing process is performed after coordinate conversion, based on the coordinates (x, y, z) of the detected object,
When another detected object is within a predetermined range with respect to one detected object, they are determined to be one target, and the detected objects detected from the same target are as described above. They are combined into one detected object.

The relative speed of the detected object with respect to the vehicle is calculated based on the position (x, y, z) of the detected object and the vehicle speed v (step 32). The relative velocity is calculated, for example, based on the x coordinate and the y coordinate of the position (x, y, z), that is, based on the position information of the detected object in the horizontal plane. Calculation of the relative speed of the detected object based on the position (x, y) and the vehicle speed V is disclosed in, for example, Japanese Patent Application No. 4-305019.

Next, based on the relative distance to the detected object (relative distance before conversion of the coordinate system or z coordinate after conversion of the coordinate system) and the relative speed of the detected object for which the relative velocity is calculated. Then, the collision risk level representing the risk level of collision with the detected object is calculated (step 33). This collision risk is, for example, when the detected object approaches the vehicle 1 on which the laser radar 10 is mounted at the relative speed V of the detected object from the current distance D, the detected object and the vehicle Is the time until they collide. The collision risk t is calculated by the following equation.

T = D / V (V> 0) (4) t = ∞ (V ≦ 0) (5)

When the relative speed V is 0 or negative, it means that the vehicle is moving away from the vehicle and the risk of the vehicle colliding with the object to be detected is low. Therefore, the collision risk t is set to ∞. .

Next, whether or not the detected object to be detected is a person (pedestrian) is judged based on the position information (step 34). The positions of the detected objects are rearranged in ascending order of the Y coordinate, and the position P1 (x1,
y1, z1) and the position P2 (x2, y2, z2) of the other detected object, these two detected objects are human (pedestrian) depending on whether or not the condition expressed by the following equation is satisfied. Is determined.

| X1 −x2 | <Thx AND | y1 −y2 | <Thy AND Thz1 <| z1 −z2 | <Thz2 (6)

Equation (6) is based on the premise that one detected object is a reflective material worn near the head and the other detected object is a reflective material worn near the foot. Therefore, Thx and Thy are thresholds determined by the resolution of the laser radar 10, and Thz1 and Thz2 are thresholds determined by the height of the person. These thresholds are, for example,
Thx = 0.5 [m], Thy = 1.0 [m], Thz1 = 1.0 [m] and Thz2 = 1.8 [m].

By using the conditional expression of the person judgment represented by the equation (6), it is possible to distinguish obstacles existing on and on the road, such as roadside reflectors (delineators), walls, signboards, etc., from people. You can Since the roadside reflector is detected as one detected object and the wall and the signboard detect the detected object continuously in a wide area, the above-mentioned person judgment condition is not satisfied.

When it is determined that the detected object is a person (YES in step 34), the collision risk t is compared with a predetermined alarm generation threshold Thm (step 35), and the collision risk t is alarm generation threshold Thm. If less (YES at step 35), a human alarm is generated by the alarm device 19 (step 36), otherwise (NO at step 35) no alarm is generated.

If it is determined that the detected object is not a person, that is, an obstacle other than a person (NO in step 34),
The collision risk t is compared with a predetermined alarm generation threshold Tho (step 37), and when the collision risk t is less than the alarm generation threshold hTo (YES in step 37), another alarm is generated by the alarm device 19. If not (step 38), otherwise (NO at step 37), no alarm is issued.

The relative velocity of the detected object may be taken into consideration as the condition for determining the detected object as a person. Further, it may be possible to calculate the human-likeness by fuzzy inference and determine whether or not the detected object is a human based on the human-likeness.

The alarm generation threshold is Thm> Tho, for example, Thm = 10 [seconds] and Tho = 5 [seconds]. Since people (pedestrians) are particularly dangerous for obstacles such as vehicles,
When a person is detected, an alarm is issued earlier than other obstacles (vehicles, etc.) to inform the driver of the presence of the person.

It is preferable to use different alarms for humans and other alarms. For example, the volume of sound (volume) is changed, the frequency of sound is changed, or the volume and frequency of sound are changed. Further, the synthesized voice may be used as an alarm.

The vehicle speed v may be taken into consideration when determining whether the alarm is issued. For example, when the vehicle speed v is high, the alarm generation threshold Th
When m and Tho are set large, when the vehicle speed v is small, the alarm generation thresholds Thm and Tho are set small.

In this way, it is judged whether or not the detected object detected by the laser radar 10 is a pedestrian, and when it is a pedestrian, the pedestrian is ahead of the vehicle from an early stage. The driver can be alerted by sounding an alarm. If the driver pays attention to the pedestrian, accidents involving the pedestrian can be reduced.

Second Embodiment An alarm generating device according to a second embodiment uses an image obtained by capturing a scene in front of a vehicle with a camera,
It detects pedestrians by moving stereo vision and issues an alarm.

FIG. 9 is a block diagram showing the structure of the alarm generator in the second embodiment. In this figure, the vehicle speed sensor 18 and the alarm device 19 are the same as those shown in FIG.

Each memory is realized by a storage element such as a RAM or an EEPROM. The first image memory 53, the second image memory 54, the perspective image memory 56, and the difference image memory 57 can be configured by one memory, or some of them can be combined, for example, the first image memory 53. The second image memory 54 may be configured as one image memory.

The video camera 51, as shown in FIG.
On the back side of the room mirror 1B (the side opposite to the mirror surface), pass the windshield 1C of the vehicle 1 through the vehicle 1
It is installed so that the front of can be imaged. The video camera 51 is installed so that its visual axis (optical axis) is substantially parallel to the road surface in the straight traveling direction of the vehicle. Video camera 51 For example, a CCD camera or the like, preferably a video camera with a shutter. The video camera 51 captures a scene in front of the vehicle 1 in response to a first image pickup signal or a second image pickup signal provided from an image processing device 55 described later, and outputs an analog video signal representing the landscape to A / A. Output to the D converter 52.

The signal processing device 52 includes an A / D conversion circuit,
An analog video signal representing a landscape captured by the video camera 51 is converted into gray level digital image data by an A / D conversion circuit. Image data is a
Pixels (vertical) × b pixels (horizontal), each pixel being, for example, 8
It is represented by bit gray level data (gradation data). Image data is stored in the first image memory 53 based on the first image pickup signal, and image data is stored in the second image memory 54 based on the second image pickup signal.

The image processing device 55 detects an object to be detected based on the first image data and the second image data stored in the first image memory 53 and the second image memory 54, respectively, and the vehicle speed detected by the vehicle speed sensor 18. In addition to extracting, a different alarm is sounded depending on whether the detected object is a person. 11 and 12 are flow charts showing the overall processing procedure in the image processing apparatus 55.

The first image pickup signal is output to the video camera 51, the scene in front of the vehicle 1 is picked up by the video camera 51, and image data representing the scene is stored in the first image memory 53.
(Step 61). The image data stored in the first image memory 53 is called first image data, and the image represented by the first image data is called first image.

When a predetermined time τ (for example, 1/30 seconds) elapses after the output of the first image pickup signal, the second image pickup signal is output from the image processing device 55 to the video camera 1, and the second image pickup is performed. In response to the signal, the scene in front of the vehicle 1 is imaged again by the video camera 1 and the image data is stored in the second image memory (step 62). The image data stored in the second image memory 4 is referred to as second image data, and the image represented by this second image data is referred to as the second image data.
It is called an image. .

The second image is the time τ from the time when the first image was captured.
Since the image is taken after only the time elapses, the vehicle 1 has a distance d.
= V · τ (v represents the vehicle speed detected by the vehicle speed sensor 18). Therefore, the image pickup position of the second image is moved forward by the distance d from the image pickup position of the first image.

The first image data stored in the first image memory is subjected to two-step perspective conversion on the image surface of the second image by the image processing device 55 to produce perspective image data. It is stored in the image memory 56 (step 63). An image represented by the fluoroscopic image data is called a fluoroscopic image. The two-stage perspective transformation process in step 63 may be performed after capturing the first image and before capturing the second image.

The two-stage perspective transformation creates an image captured from a different second viewpoint based on an image captured from a certain first viewpoint. First, the image captured from the first viewpoint is perspective-transformed (perspective projection) on a horizontal plane serving as a reference. Secondly, the image perspectively transformed on the horizontal plane is perspectively transformed into an image picked up from the second viewpoint. In this embodiment, the first viewpoint and the second viewpoint are the positions of the video camera 51 that has captured the first image and the second image, respectively, and the horizontal plane is the road (road surface). First
An image (a perspective image) viewed from the same viewpoint as the viewpoint at which the second image is captured is created based on the image.

Next, the difference image data between the second image data stored in the second image memory 54 and the perspective image data stored in the perspective image memory 56 is the difference between the image data of the corresponding pixels. It is created by calculating the absolute value, and the difference image data is stored in the difference image memory 57 (step 64). In this difference processing, if there is a coordinate stored in the internal memory of the image processing device 55, the value (gray level) of the pixel data corresponding to this coordinate is set to zero. An image represented by the difference image data is called a difference image.

Generally, an image which has been subjected to a two-step perspective transformation, and 2
Comparing the images taken from the perspective of stepwise perspective transformation,
The same position (coordinates) on the plane such as the road surface (including the speed limit, the center line, etc. drawn on the road surface)
Value of the image data representing the road surface (gray
Level) becomes zero or a value close to zero, images representing solid bodies such as vehicles, pedestrians, and streetlights other than this plane are not at the same position, and the value of the image data does not become zero or a value close to zero. Therefore, the image representing the road surface is almost deleted from the perspective image, and the image representing the solid other than the road surface remains in the perspective image.

The differential image data stored in the differential image memory 57 is binarized by a predetermined threshold value by the image processing device 55, and the binary image data is created (step 65).
The threshold used for this binarization is set to a value suitable for emphasizing the image representing the obstacle.

By binarizing the difference image data, the image data representing the road surface becomes 0, and the image data representing a solid other than the road surface, that is, an obstacle becomes 1. Therefore, in the binary image, only the image showing the obstacle is extracted.

A labeling process is performed on the binary image data to extract a set of pixels (images) representing obstacles (step 66). The window is determined to include all the labeled images (step 6).
7).

For details of the processing in steps 61 to 67,
It is disclosed in Japanese Patent Application No. 6-153048.

A window surrounding an image representing each obstacle is
Horizontal edges and vertical edges are extracted from the second image data (target image data) set in the second image data stored in the second image memory (step 68) and included in each window (step 69). . By setting the window in the second image data, only the target image data representing the obstacle in the second image can be cut out.

The window may be set to the perspective image data stored in the perspective image memory 66 instead of or in addition to the second image data.

The minimum and maximum values in the Y direction of the horizontal edge extracted in each window are extracted, the difference between the maximum value and the minimum value is calculated as the vertical width of the obstacle, and the minimum value in the X direction of the vertical edge is calculated. The horizontal width is calculated based on the maximum value in the same manner as the vertical width (step 69). The aspect ratio R of the obstacle is calculated based on these vertical width and horizontal width (step 70). The aspect ratio R is, for example, R = vertical width / horizontal width. Further, it is determined based on this aspect ratio R whether the obstacle is a person or another obstacle such as a vehicle (step 71). The conditions for human judgment are Thr1 and Thr.
It is expressed by the following equation with 2 as the threshold value.

THR1 <r <Thr2 (7)

The thresholds Thr1 and Thr2 are determined in consideration of the human body type, and are, for example, Thr1 = 2 and Thr2 = 4.

In step 71, if the obstacle is a person (pedestrian) (YES in step 71), an alarm for a person is issued by the alarm device 19 (step 72), and if not (NO in step 71). ), The other alarm is issued (step 73).

Different alarms are sounded according to the obstacles thus detected.

It may be used to determine whether or not an alarm sounds, for example, depending on whether each obstacle is on the lane in which the host vehicle is traveling or on another lane or sidewalk.

Further, the laser radar 10 of the first embodiment is further provided in the alarm generation device of the second embodiment, and it is determined whether or not the alarm is issued based on the distance to the obstacle (object to be detected). You may.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an alarm generation device in a first embodiment.

FIG. 2 shows how a laser radar included in an alarm generation device is attached to a vehicle.

FIG. 3 shows how a reflective material is worn by a person.

FIG. 4 shows a scanning angle range in which a laser beam is projected.

FIG. 5 shows a detailed configuration of a two-dimensional scanner.

FIG. 6 is a flow chart showing a processing procedure of alarm generation in the signal processing device of the first embodiment.

FIG. 7 shows a structure of data stored in the signal processing device.

FIG. 8 is a diagram illustrating conversion of a coordinate system.

FIG. 9 is a block diagram showing a configuration of an alarm generation device in a second embodiment.

FIG. 10 shows a state in which a camera included in the alarm generation device is attached to a vehicle.

FIG. 11 is a flow chart showing an overall processing procedure of the alarm generation device.

FIG. 12 is a flow chart showing an overall processing procedure of the alarm generation device.

[Explanation of symbols] 10 Laser radar 11 Control circuit 12 LD drive circuit 13 LD 14 Two-dimensional scanner 15 Scan angle detection circuit 16 PD 17 Light receiving circuit 18 Vehicle speed sensor 19 Alarm device 20 Signal processing device 41, 43 Mirror 42, 44 Motor 51 video camera 52 signal processing device 53 first image memory 54 second image memory 55 image processing device 56 perspective image memory 57 differential image memory

Claims (12)

[Claims] 1. An object position detecting means for detecting a detected object existing in front of a vehicle equipped with an alarm generation device, and measuring a relative position of the detected detected object, the object position detecting means detecting the relative position. A person judging means for judging whether the detected object is related to a person or an obstacle based on the relative position of the detected object, and the relative position of the detected object detected by the object position detecting means. When the person determining means determines that the detected object is related to a person, the position farther than when it is determined that the detected object is related to an obstacle is determined. From the alarm generation determining means that determines to generate an alarm from the above, and generates an alarm in response to the alarm generation determining means determining that an alarm should be generated. An alarm generator equipped with an alarm device. 2. An object position detecting means for detecting a detected object existing in front of a vehicle equipped with an alarm generation device, and measuring a relative position of the detected detected object, the object position detecting means detecting the relative position. Person judging means for judging whether the detected object is related to a person or an obstacle based on the relative position of the detected object, and the person judgment means is related to the detected object. When it is determined that the first alarm is issued, whether the second alarm is issued when the person determining means determines that the detected object is related to an obstacle, An alarm generation determination means for determining based on a relative position, and an alarm device for generating a first or second alarm in response to determination that the alarm generation determination means determines to generate an alarm Alarm generator. 3. The alarm generation device according to claim 2, wherein the first alarm has a volume higher than that of the second alarm. 4. The person judging means judges whether or not the two detected objects are related to one person based on the positional relationship between the two detected objects.
The alarm generation device according to any one of claims 1 to 3. 5. A vehicle speed sensor for detecting a speed of a vehicle equipped with an alarm generation device, and a relative speed of the detected object based on a vehicle speed detected by the vehicle speed sensor and a relative position of the detected object. The relative speed calculation means for calculating a speed is further provided, and the alarm generation judgment means judges the warning by using the relative speed of the detected object in addition to the relative position thereof. The alarm generation device according to any one of claims. 6. An image pickup means for picking up an image of a subject including an object to be detected and outputting image data representing the subject, first image data output from the image pickup means, and a predetermined time from the image pickup of the first image data. An image memory for storing the second image data output from the image pickup means after a lapse of time, and target image data representing the detected object based on the first image data and the second image data stored in the image memory A target image creating means, a person determining means for determining whether or not the detected object represented by the target image data created by the target image creating means is a person, and the person determining means is a person An alarm generation device equipped with an alarm device that generates an alarm in response to. 7. The perspective conversion means for generating perspective image data in which the target image creating means performs two-step perspective conversion of the first image data stored in the image memory on the image surface of the second image data, Difference image data between the perspective image data created by the perspective conversion means and the second image data stored in the image memory is created, and the binary image data obtained by binarizing the difference image data with a predetermined threshold is generated. Binary image creating means to be created, window determining means for extracting target images by performing labeling processing on the binary image data created by the binary image creating means, and determining a window surrounding each target image, and the above image Image creating means for outputting the second image data contained in each window among the second image data stored in the memory as the target image data, The alarm generation device according to claim 6, further comprising: 8. The person judging means extracts the horizontal edge and the vertical edge of the detected object based on the target image data created by the target image creating means, and the edge extracting means. Object determining means for calculating the height and width of the detected object based on the horizontal and vertical edges of the detected object and determining whether the detected object is a person based on these heights and widths, The alarm generation device according to claim 6, further comprising: 9. A vehicle equipped with the alarm generation device according to any one of claims 1 to 8. 10. A detected object existing in front of a vehicle is detected, a relative position of the detected detected object is measured, and the detected object is a person based on the detected relative position of the detected object. It is determined whether or not an alarm is issued based on the relative position of the detected object, and it is determined that the detected object is related to a person. Occasionally, it is determined that an alarm should be issued from a position farther than when the detected object is related to an obstacle, and an alarm is issued in response to the determination that an alarm should be issued. . 11. A detected object existing in front of a vehicle is detected, a relative position of the detected detected object is measured, and the detected object relates to a person based on the detected relative position of the detected object. It is determined whether the detected object is a person or not, and when the detected object is determined to be a person, whether the first alarm is issued is related to the detected object is an obstacle. When it is determined that the second alarm is to be issued, it is determined based on the relative position of the detected object, and in response to the determination that the alarm is to be issued, the first or second alarm is issued. appear,
Alarm generation method. 12. A subject including a detected object is imaged to obtain image data representing the subject, the first image data obtained by the image pickup, and an image pickup after a predetermined time has elapsed from the image pickup of the first image data. The second image data obtained by the above is stored in the image memory, and the target image data representing the detected object is created based on the first image data and the second image data stored in the image memory, An alarm generation method for determining whether or not the detected object represented by the created target image data is a person and issuing an alarm in response to the determination that the detected object is a person.