JP6635428B2 - Car peripheral information display system - Google Patents

Description

  The present invention relates to a vehicle periphery information display system that displays a situation around the vehicle in a vehicle such as an automobile.

  2. Description of the Related Art Conventionally, as an automobile peripheral information display system, a navigation device 2 mounted on a vehicle such as an automobile and displaying map information to a driver to guide the driver to a destination is known. The navigation device 2 stores map information in a map database, specifies the current position of the vehicle using GPS information received from GPS satellites, reads a map up to the destination, and accompanies the travel of the vehicle. Is displayed every moment.

  However, the map information is information mainly based on roads, and an inefficient screen is obtained if there are many portions having a very small amount of information on the screen. Therefore, a navigation device that displays additional information on a map display screen in addition to road information in a superimposed manner is known (for example, see Patent Literatures 1 and 2).

JP 2001-99666 A JP 2001-42026 A

  However, the prior art according to the above-mentioned patent document displays a map display screen by displaying information such as an animation or a general moving image, or a still image or text data in an area where information is small in displaying a map. The present invention provides an in-vehicle navigation device in which the user is less likely to feel bored by making changes.

  Therefore, since the information displayed by such a navigation device is additional information indicating that there is a displayable area if the user can drive the vehicle or in an autonomous vehicle, the information is displayed. The information is not always necessary for the user to check the automatic driving status.

  In recent years, research and development of an automatic driving system for further improving the safety of road traffic have been promoted. In this system, an automobile recognizes a surrounding environment and runs automatically. In Japan, an automatic driving system for a vehicle such as an automobile is defined in four levels from level 1 to level 4. In Level 1, the vehicle performs any of acceleration, steering, and braking, and is called a safe driving support system. Level 2 is called a semi-automatic driving system (semi-automatic driving), in which the vehicle performs a plurality of operations of acceleration, steering, and braking simultaneously. Level 3 is a state in which the vehicle performs all acceleration, steering, and braking, and the driver responds only in an emergency. This is also called a semi-automatic driving system. Level 4 is called a fully automatic driving system in a state where all of the acceleration, steering and braking are performed by a person other than the driver, and the driver is not involved at all. In addition, Level 2 to Level 4 are also referred to as an autonomous driving system (“Step SIP (Strategic Innovation Creation Program) Automated Driving System R & D Plan”, November 13, 2014, Cabinet Office, Policy Supervisor (Science) Technology and Innovation))). It should be noted that "automatic driving" in the present specification basically refers to automatic driving at the fully automated level from level 1 to level 4 described above.

  In such an automatic driving system, there is a demand for a passenger of an automobile to confirm the reliability of the automatic driving state by grasping the surrounding situation recognized by the own vehicle.

  The present invention has been made in view of the above points, and an object of the present invention is to provide an automobile peripheral information display system capable of displaying a real situation around an own vehicle in a superimposed manner on a map.

  The vehicle surrounding information display system of the present invention includes a sensor device for detecting an external situation from the own vehicle, and a current position and a traveling direction of the own vehicle, and map data of a surrounding area including the current position from a map database. A navigation device to be read, a target determination unit that detects the presence of the detection target object from the sensor information detected by the sensor device and determines the type of the detection target object, and a position of the detection target object based on the own vehicle from the sensor information. Image generation for generating a map image in which an icon corresponding to the detection target is superimposed on a position on the map data indicated by the position information on the detection target detected by the target determination unit, and a position detection unit for obtaining information. And a display device for displaying the map image.

  When the sensor device includes a camera that captures an image outside the host vehicle, the target determination unit detects the presence of the detection target by pattern matching from the captured image of the camera. I do. Further, if the camera is constituted by a stereo camera, the position detection unit may calculate position information of the detection target from an image of the stereo camera.

  In addition, the target determination unit may detect a lane line including a white line or a yellow line based on luminance information of the image. By doing so, when the object discriminating unit detects, as the detection target, an oncoming other vehicle that is in front of the own vehicle and an oncoming other vehicle that is opposed to the own vehicle, the image generation unit performs an autonomous search on the map data. The map image that displays the lane line schematically on a road on which the vehicle travels and displays the icons of the preceding other vehicle and the oncoming other vehicle while distinguishing the lane can be generated.

  If the object discriminating unit removes an object other than the detection target part detected from the image, the position detection unit can obtain only the position information of the detection target in a short time.

  On the other hand, if the sensor device has a configuration including a radar sensor composed of any one of or a combination of a millimeter-wave radar, a microwave radar, a laser radar, an infrared sensor, and an ultrasonic sensor, the position detection unit includes: Position information of the detection target can be obtained based on distance information between the host vehicle and the detection target detected by the radar sensor.

  Further, a communication device that performs inter-vehicle communication with another vehicle and obtains position information and vehicle attribute information on the other vehicle may be provided. By doing so, the image generation unit can generate a map image in which an icon corresponding to the vehicle attribute information is superimposed on a location corresponding to the position information on the map data. This inter-vehicle communication is conceivable in any case in which communication is performed indirectly between the own vehicle and another vehicle directly or through a communication line via a relay center.

  According to the vehicle surrounding information display system of the present invention, the surrounding state recognized by the sensor device of the own vehicle is superimposed on the map data around the current position of the own vehicle output by the navigation device, thereby displaying the vehicle. Crew members can judge the surrounding situation from a bird's-eye view.

1 shows a block diagram of the entire configuration of a vehicle periphery information display system according to a preferred embodiment of the present invention. FIG. 2 is a plan view illustrating an example of an arrangement configuration of a sensor device. 3 is a flowchart illustrating the operation of the vehicle periphery information display system according to the present invention. (A) is a schematic explanatory diagram of an image based on the map data shown by the navigation device, and (b) is an explanatory diagram of an image in which a surrounding situation is superimposed on the image by icons. The surrounding situation in front of the own vehicle is illustratively shown. FIG. 2 is a system configuration diagram illustrating an operation of inter-vehicle communication. FIG. 3 is a schematic explanatory diagram showing a configuration of a data frame of a message transmitted by inter-vehicle communication. 4 is a flowchart illustrating an operation of displaying the status of another vehicle obtained through inter-vehicle communication on map data.

  Hereinafter, an embodiment of a vehicle periphery information display system according to the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram conceptually showing the entire configuration of the vehicle periphery information display system 1. The vehicle image display system 1 includes a navigation device 2, a sensor device 3, a communication device 4, a display device 5, and a control unit 6.

  The navigation device 2 specifies the position and traveling direction of the own vehicle mounted on the basis of GPS information received from a GPS (global positioning system) 11. The navigation device 2 includes a map database 2A, and reads map data of an area including the position of the identified vehicle from the map database 2A.

  The sensor device 3 detects a situation around the own vehicle, and in this example, as shown in FIG. 2, is disposed on the front windshield 15 in order to capture a view ahead of the vehicle 14 in the traveling direction. The stereo camera 13 includes a pair of front cameras 16 and 17, and a millimeter-wave radar sensor 18 disposed on a rear windshield 19 for detecting a vehicle following the automobile 14.

  By using the stereo camera 13 as the sensor device 3, the object to be detected in front of the automobile 14 can be recognized three-dimensionally, and by widening the angle of view of the cameras 16 and 17, the camera 16 jumps out from the front side. The object to be detected can be reliably detected. Further, a rear camera may be arranged instead of or in addition to the radar sensor 18.

  The communication device 4 receives the position, course, and traveling state information of the other vehicle by communicating with the other vehicle traveling or parked around the own vehicle. In addition, the communication device 4 is configured to be able to receive infrastructure data such as traffic signal information, regulation information, pedestrian information, and accident vehicle information from a traffic signal or a communication device (roadside device) installed along the road. Although not shown, such information from the infrastructure can also be obtained by wireless communication with a traffic information center through the Internet or public broadcasting.

  The control unit 6 is configured by a computer including a CPU, a ROM, and a RAM. The CPU executes a control program stored in the ROM to control each of the above-described devices and to determine an object. It functions as the unit 7, the position detection unit 8, and the image generation unit 9, respectively.

  The object discriminating unit 7 analyzes the images sent from the cameras 16 and 17 of the stereo camera 13, respectively, and analyzes a vehicle (various automobiles, motorcycles, bicycles, etc.), a person, etc. (hereinafter, referred to as a detection target). ) And the lane are detected.

  In this case, for the detection of the detection target, the target determination unit 7 performs a pattern matching process to detect the outline of the detection target from the image and determine the presence of the detection target. Therefore, in the ROM, a pattern data file in which image features of detection targets such as various vehicles and people are registered in advance, and by determining whether the image includes the image features, The presence and type of the detection target are determined from the captured images of the cameras 16 and 17. Further, the target determining unit 7 performs a process of removing an object other than the detected target.

  Further, the target discriminating unit 7 detects the lane by detecting the lane line 24 of a white line or a yellow line based on the luminance information of each image captured by each of the cameras 16 and 17 for the detection of the lane. I do.

  The position detection unit 8 calculates the position of the detection target from each of the images of the cameras 16 and 17 from which unnecessary objects have been removed by the target determination unit 7. In addition, the position detection unit 8 determines the position of the following vehicle from the distance information between the vehicle and the following vehicle measured by the radar sensor 18.

  The image generation unit 9 generates an image that displays the surrounding situation of the vehicle based on the map data from the navigation device 2 and the position information of the detection target from the position detection unit 8. At this time, the image generation unit 9 superimposes an icon which is a symbolized picture of the person or the vehicle identified by the object identification unit 7 on the image of the map data from the navigation device 2 and surrounds the vehicle around the own vehicle. Create an image showing the situation. Therefore, a plurality of icons according to the type of the detection target are set in the image generation unit 9 in advance.

  The display device 5 is mounted on the dashboard of the automobile 14 so that the passenger can easily view the image, and displays the image generated by the image generation unit 8.

  The operation of the vehicle periphery information display system 1 having the above configuration will be described with reference to the flowchart shown in FIG.

  The navigation device 2 receives the GPS information from the GPS 11, calculates the current position of the own vehicle, and stores the history of the received GPS information to specify the traveling direction of the own vehicle (step S01). Then, the navigation device 2 reads out the map data including the current position of the own vehicle from the map database 2A and outputs the map data to the target determination unit 7 (Step S02). FIG. 4A illustrates a map image represented by the map data at this time, and the position and the traveling direction of the own vehicle are indicated by arrows. This map image is a general image displayed by the navigation device.

  Then, each of the cameras 16 and 17 captures an image of an external situation in front of and behind the own vehicle, and the object determination unit 7 determines that a detection target such as a person or a vehicle exists around the own vehicle by image recognition processing. Is determined.

  FIG. 5 exemplifies an image of a sight in front of the host vehicle. The target determination unit 7 performs pattern matching processing on each image captured by the cameras 16 and 17 to determine the presence of a detection target object. The lane line 24 is detected (step S03). Thereby, the preceding vehicle 20 traveling in the same lane L1 as the own vehicle, the oncoming vehicle 21 traveling in the right lane L2, the automobile 22 from the left side, and the bicycle 23 from the right side are respectively detected. .

  Then, the target determination unit 7 performs a process of removing objects such as the buildings 25 and 26 and the traffic signals 27 and 28 other than the automobiles 20, 21, 22 and the bicycle 23, which are detection targets (step S04). Then, the position detection unit 8 determines the position on the XY coordinate axis from the own vehicle to each of the automobiles 20, 21, 22, and the bicycle 23 based on the images of the stereo cameras 16 and 17 from which the object determination unit 7 has performed the removal processing. Information is obtained by calculation (step S05).

  On the other hand, the position detection unit 8 obtains, from the sensor device 3, information on the distance to the following vehicle measured by the radar sensor 18. Therefore, the position detection unit 8 calculates the position information on the XY coordinate axes of the relative position of the succeeding vehicle, which is the rear detection target object, with respect to the own vehicle based on the own vehicle (step S06). In this case, the object detected by the radar sensor 18 is a following vehicle, but if a rear camera is arranged together with the radar sensor 18 and an image captured behind is sent to the object determination unit 7, the type of the object detected behind is determined. Can be detected every time.

  When the type information of each detection target is indicated from the target determination unit 7, the image generation unit 9 selects an icon corresponding to the type and detects the selected icon 20 i, 21 i, 22 i, or 23 i from the position detection unit 8. A map image superimposed on the map data indicated by the navigation device 2 is generated and displayed according to the indicated position information of each detection target (step S07). At this time, in the image generated by the image generation unit 9, since the lane line 24 from the object determination unit 7 is detected, as shown in FIG. 24 are displayed to divide the lanes L1 and L2, and the icons of the preceding vehicle 20i and the oncoming vehicle 21i are distinguished and displayed. In this case, the radar sensor 18 detects the following vehicle, and the image generation unit 9 displays the icon 24i on the map data in accordance with the position information of the following vehicle.

  When a bus or a truck is recognized by the pattern matching processing of the target determination unit 7, these icons are displayed so as to distinguish them from passenger cars. Alternatively, a common icon representing all types of automobiles may be used. When a pedestrian is recognized, an icon that symbolizes a person is displayed.

  Some types of map data output from the navigation device 2 are displayed as three-dimensional images including surrounding buildings and the like. In this case, actual images captured by the cameras 16 and 17 are displayed on a map. It can also be displayed over the data.

  According to the vehicle image display system 1 having the above-described configuration, the occupant of the automobile 14 can check the situation around the own vehicle, which is detected by the sensor device 3, on the screen of the display device 5. Therefore, in particular, when the automobile 14 is an autonomous vehicle, it is possible to compare the surrounding situation observed by the occupant with the surrounding situation recognized by the sensor device 3, so that the reliability of the autonomous driving system can be reduced. It functions as an auxiliary system that can confirm the performance.

  The vehicle image display system 1 can display not only the information from the sensor device 3 but also the surrounding information obtained by the inter-vehicle communication by the communication device 4 on the display device 5.

  Here, the inter-vehicle communication will be described. As shown in FIG. 6, message data indicating the position of the own vehicle and the vehicle state is transmitted and received between the communication devices 4 of the vehicles 31 and 32 by unidirectional broadcast communication. Things. In this case, the vehicles 31 and 32 may directly communicate with each other, or may pass through a relay center 34 via a communication line 36 such as the Internet from an antenna 33 or a communication chip (not shown) installed along the road. And may be indirectly communicated. In the case shown in FIG. 6, vehicles 31 and 32 traveling toward the same intersection cannot be visually confirmed because the building 35 blocks each other's view, but both vehicles 31 and 32 communicate with each other by their positions and vehicle speeds. By transmitting the information, etc., it becomes safe driving support to prevent collision accidents.

  FIG. 7 schematically shows a configuration example of a data frame DF of a message exchanged between vehicles, a time information data frame DF1 in which hour, minute, and second data elements are written, and a latitude acquired by GPS or the like. A position information data frame DF2 in which each longitude data element is written, and a vehicle state information data frame DF3 and a vehicle in which vehicle speed information, vehicle azimuth information, automatic driving information, shift position information, and steering angle information are written. It is composed of a vehicle attribute information data frame DF4 in which each data element of size-specific (large, normal, motorcycle, etc.) information, vehicle width information and vehicle length information is written.

  The operation of displaying the information obtained by the inter-vehicle communication will be described with reference to the flowchart of FIG. First, the communication device 4 inputs the position and the vehicle speed of the own vehicle acquired by the navigation device 2 (Step S11), and acquires the position and the vehicle speed of another vehicle through the communication device 4 (Step S12). In this case, from another vehicle equipped with a communication device that is the same as or compatible with the communication device 4, vehicle status information such as position information, vehicle attribute information, speed and azimuth type of the other vehicle is provided as described above. Is output in the data frame.

  Therefore, the image generation unit 8 collates with the map data indicated by the navigation device 2 (step S13), and determines whether there is another vehicle around the own vehicle from the collation result (step S14). Then, when another vehicle with which communication is possible is present in the vicinity (“YES” in step S14), the image generation unit 8 selects and selects an icon according to the type of the other vehicle based on the vehicle attribute information. A map image superimposed on the map data is generated and displayed according to the position information of the message having received the icon (step S15). On the other hand, the communication device 4 transmits a message including time information, position information, vehicle state information, and vehicle attribute information relating to the own vehicle (step S16).

  By performing such vehicle-to-vehicle communication, even when the vehicle 22 shown in FIG. The presence can be detected and an icon can be displayed on the map data. In addition to the inter-vehicle communication, the communication device 4 obtains an accident vehicle, construction information, and the like by wireless communication with a traffic information center through the Internet or public broadcasting, and displays an icon representing these conditions in map data. It can also be displayed above.

  As described above, the preferred embodiments of the present invention have been described in detail. However, the present invention is not limited to the above embodiments, and can be implemented with various modifications or changes within the technical scope thereof. . For example, a pair of cameras (stereo camera) or a radar sensor may be arranged on both sides of the rear windshield 19 in order to widen the detection field of view behind the automobile. The left and right cameras and radar sensors may be arranged on each of the pair of door mirrors without being limited to the rear windshield 19.

DESCRIPTION OF SYMBOLS 1 Car periphery information display system 2 Navigation device 3 Sensor device 4 Communication device 5 Display device 7 Target discriminating unit 8 Position detecting unit 9 Image generating unit 13 Stereo camera 18 Radar sensors 20i, 21i, 22i, 23i, 24i Icon 24 Lane line 34 Relay center 36 Communication line

Claims (8)

A sensor device for detecting a situation around the vehicle,
A navigation device that specifies a current position and a traveling direction of the vehicle and reads map data of a surrounding area including the current position from a map database;
A target determining unit that detects the presence of the detection target from the sensor information detected by the sensor device and determines the type of the detection target;
A position detection unit that obtains position information of the detection target based on the vehicle from the sensor information;
An image generation unit that generates a map image in which an icon corresponding to the detection target is superimposed on a position on the map data indicated by the position information regarding the detection target detected by the target determination unit;
A display device for displaying the map image;
A communication device that performs inter-vehicle communication for directly or indirectly communicating with another vehicle to obtain position information and vehicle attribute information on the other vehicle;
With
The image generation unit, the communication apparatus receives a result of its location information and car輌属of information from other vehicles by communication between the vehicle and the position information of the other vehicle received and the map data from the navigation device Collation, as a result, when the other vehicle is present around the own vehicle, a map in which the icon corresponding to the vehicle attribute information of the other vehicle is superimposed on a location corresponding to the position information of the other vehicle on the map data. An automobile peripheral information display system for generating an image.   The sensor device includes a camera that captures an image outside the host vehicle, and the target determination unit detects the presence of the detection target from a captured image of the camera by pattern matching processing. Item 4. An automobile peripheral information display system according to item 1.   The vehicle surrounding information display system according to claim 2, wherein the camera is a stereo camera, and the position detection unit calculates position information of the detection target from an image of the stereo camera.   The vehicle surrounding information display system according to claim 2, wherein the target determination unit detects a lane line based on luminance information of the image.   The target determination unit detects, as the detection target object, an oncoming other vehicle that is ahead of the own vehicle and an oncoming other vehicle that faces the own vehicle, and the image generation unit determines whether or not a road on which the own vehicle travels in the map data. 5. The vehicle periphery according to claim 4, wherein the lane line is schematically displayed, and the map image is displayed to display the respective icons of the preceding vehicle and the oncoming vehicle while distinguishing the lanes. 6. Information display system.   The target determination unit removes an object other than the detection target unit detected from the image, and the position detection unit determines that the position detection unit obtains position information of the detection target object from the removed image. The vehicle periphery information display system according to claim 2 or 3, wherein:   The sensor device includes a millimeter-wave radar, a microwave radar, a laser radar, an infrared sensor, a radar sensor composed of any one of an ultrasonic sensor or a combination thereof, and the position detection unit detects the radar sensor. The vehicle surrounding information display system according to claim 1, wherein position information of the detection target is obtained based on distance information between the vehicle and the detection target. The communication device, the information of the accident vehicle and construction can be obtained by the vehicle between wireless communication with an external non-communication, the image generation unit, the information of the accident vehicle and construction wherein the communication device is obtained The vehicle peripheral information display system according to any one of claims 1 to 7 , wherein a map image in which the icon to be displayed is superimposed on the map data is generated.

Images