JP7210208B2 - Providing device - Google Patents

Description

The present invention relates to a provision device and a provision method.

2. Description of the Related Art Conventionally, for example, there are techniques for detecting surrounding conditions such as a target with a sensor, executing vehicle control based on the detection result, and notifying the driver of the detected target. Examples of such sensors include cameras, radars and lidars (LiDAR: Laser Imaging Detection and Ranging).

JP 2017-159789 A

By the way, the above-described sensor may be affected by the weather and the accuracy of the detection result may be deteriorated. In addition, it is not easy for the driver of a vehicle not equipped with the above-described sensor to immediately respond to a sudden change in weather.

SUMMARY OF THE INVENTION The present invention has been made in view of the above, and an object of the present invention is to provide a provision device and a provision method capable of detecting the weather with high precision and alerting the surroundings.

In order to solve the above-described problems and achieve the object, the provision device according to the present invention includes a sensor detection result acquisition section, a weather condition determination section, and a provision section. The sensor detection result acquisition unit acquires detection results of a plurality of sensors that detect surrounding conditions. The weather condition determination unit determines weather conditions based on the detection results of the plurality of sensors acquired by the sensor detection result acquisition unit. The providing unit provides the weather condition information, which is information about the weather condition determined by the weather condition determining unit, to the outside.

ADVANTAGE OF THE INVENTION According to this invention, while detecting a weather with high precision, it can call attention to surroundings.

FIG. 1 is a diagram showing an overview of a provision method according to an embodiment. FIG. 2 is a block diagram showing the configuration of a vehicle provided with the provision device according to the embodiment. FIG. 3 is a diagram showing an example of weight information. FIG. 4 is a diagram showing details of processing of weather detection using white line detection by the determination unit. FIG. 5 is a flowchart illustrating a processing procedure of processing executed by the providing apparatus according to the embodiment;

Hereinafter, embodiments of the provision device and the provision method disclosed in the present application will be described in detail with reference to the accompanying drawings. It should be noted that the present invention is not limited by this embodiment.

First, with reference to FIG. 1, an overview of the provision method according to the embodiment will be described. FIG. 1 is a diagram showing an overview of a provision method according to an embodiment. In FIG. 1, the provision device 1 according to the embodiment is mounted on a vehicle C and executes the provision method according to the embodiment. It should be noted that the provision device 1 is not limited to vehicles, and may be other moving objects such as motorcycles, bicycles, ships, and aircraft. Alternatively, the providing device 1 may be mounted on a stationary object such as a streetlight or a roadside object (guardrail, traffic light, etc.), not limited to a mobile object.

The vehicle C is also equipped with a target detection device (not shown) that integrates the detection results of a plurality of sensors for detecting surrounding conditions and detects targets existing in the surroundings. Note that the plurality of sensors include, for example, cameras, radar, and LiDAR (Laser Imaging Detection and Ranging).

In addition, in FIG. 1, the providing device 1 according to the embodiment detects dense fog that has occurred in front of the vehicle C, and provides a providing method for providing weather condition information, which is information calling attention to the detected dense fog, to the outside. .

Specifically, the provision device 1 according to the embodiment first acquires the detection results of the plurality of sensors described above (step S1). Subsequently, the providing device 1 determines weather conditions occurring around the vehicle C based on the detection results of the plurality of acquired sensors (step S2).

The weather here refers to bad weather such as heavy fog, rain, and snow. In other words, the weather detected by the provision device 1 is weather that lowers the accuracy of the detection result of the sensor described above. Note that the weather may be distinguished between daytime and nighttime. That is, the providing apparatus 1 may determine the weather conditions by subdividing the rain weather into day and night, such as daytime rain and nighttime rain.

In this way, the provision device 1 according to the embodiment originally determines the weather conditions using the detection results of the sensors used for target detection. A detailed weather condition determination method will be described later.

Subsequently, the provision device 1 according to the embodiment provides the determined weather condition information about the weather condition to the road management terminal (road management server SV) handled by the road administrator or the in-vehicle device mounted on the peripheral vehicle OC1 (step S3-1, S3-2).

Specifically, the provision device 1 according to the embodiment sends information indicating weather conditions such as weather codes, the position of the vehicle C, the current time, etc. to the road management terminal that manages the road on which the vehicle C travels. (step S2-1).

As a result, for example, the road administrator can display a warning such as "Watch out for heavy fog ahead!" Then, the driver of the surrounding vehicle OC2 behind who sees such a display can prepare for the dense fog in advance, so that he/she can deal with the dense fog that has occurred without hesitation.

Further, the provision device 1 according to the embodiment changes the weight given to the detection result of each of the plurality of sensors used by the above-described target object detection device according to the determined weather, and Weather condition information (deleted at the time of application *The red underline is omitted below.) including information on the weight (hereinafter sometimes referred to as weight information) to the target detection device of the vehicle OC1. provided (step S2-2).

Thereby, for example, in the target object detection device mounted on the surrounding vehicle OC1, the weight of each detection result of the plurality of sensors can be changed according to the weather. For example, in the case of dense fog, the weight of the detection result of the camera is lightened, and the weight of the detection result of the radar device is weighted. Therefore, even if the surrounding vehicle OC1 enters into thick fog, it is possible to detect the target object by assigning an optimum weight according to the thick fog. be able to.

Note that if the surrounding vehicle OC1 does not have a target object detection device, the providing device 1 according to the embodiment provides weather condition information indicating the occurrence of dense fog to the navigation device or the like of the surrounding vehicle OC1 instead of the weight information. can also As a result, the driver of the surrounding vehicle OC1 can prepare for the dense fog in advance, and can deal with the dense fog that has occurred without being confused.

As described above, in the provision method according to the embodiment, the weather conditions are determined using the detection results of a plurality of sensors, and the weather condition information is provided to the outside. Therefore, according to the provision method according to the embodiment, it is possible to detect the weather with high precision and to call attention to the surroundings.

Next, the configuration of the provision device 1 according to the embodiment will be described in detail with reference to FIG. FIG. 2 is a block diagram showing the configuration of a vehicle C equipped with the provision device 1 according to the embodiment. As shown in FIG. 2, the vehicle C according to the embodiment includes a providing device 1 and a target detection device 100. As shown in FIG. The providing device 1 and the target detection device 100 are communicably connected via a predetermined in-vehicle network, and can transmit and receive information to and from each other. Note that FIG. 2 shows a case where the target detection device 100 and the provision device 1 are configured separately, but the target detection device 100 and the provision device 1 are integrally configured as one device. good too.

The target detection device 100 includes a camera 10, a radar device 11 and a LiDAR 12, which are multiple sensors. The camera 10 is an imaging device that captures an image of the external situation of the vehicle C. As shown in FIG. The camera 10 is provided in the windshield of the vehicle C, for example, and images the front of the vehicle C. As shown in FIG. In addition, the camera 10 may be provided at a position for imaging the left and right sides of the vehicle C and a position for imaging the rear of the vehicle C. As shown in FIG. The target detection device 100 detects targets existing around the vehicle C by analyzing the image of the camera 10 .

The radar device 11 detects targets around the vehicle C using radio waves such as millimeter waves. Specifically, the radar device 11 detects a target by transmitting radio waves to the vicinity of the vehicle C and receiving reflected waves reflected by the target.

The LiDAR 12 detects targets around the vehicle C using laser light. Specifically, the LiDAR 12 detects the target by transmitting laser light around the vehicle C and receiving reflected light reflected by the target.

The target detection device 100 integrates detection results of the camera 10, the radar device 11 and the LiDAR 12 to finally detect the target. Specifically, the target detection device 100 detects the final target based on the result of assigning a predetermined weight to the detection result of each sensor. Such weights are provided by the providing device 1, and the details will be described later.

A provision device 1 according to the embodiment includes a control unit 2 and a storage unit 3 . The control unit 2 includes an acquisition unit 20 (an example of a sensor detection result acquisition unit), a determination unit 21 (an example of a weather condition determination unit), a determination unit 22 and a provision unit 23 . The storage unit 3 stores weight information 31 .

Here, the providing device 1 includes, for example, a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a data flash, an input/output port, and various circuits.

The CPU of the computer functions as an acquisition unit 20, a determination unit 21, a determination unit 22, and a provision unit 23 of the control unit 2 by reading and executing programs stored in the ROM, for example.

Further, at least one or all of the acquisition unit 20, the determination unit 21, the determination unit 22, and the provision unit 23 of the control unit 2 are implemented by hardware such as ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array). Can also be configured.

Also, the storage unit 3 corresponds to, for example, a RAM or a data flash. The RAM and data flash can store the weight information 31, various program information, and the like. Note that the providing apparatus 1 may acquire the above-described programs and various types of information via another computer or portable recording medium connected via a wired or wireless network.

The weight information 31 is information about weights used in target detection processing of the target detection device 100 . The weight information 31 includes information on weights assigned to each of a plurality of sensors. FIG. 3 is a diagram showing an example of the weight information 31. As shown in FIG.

As shown in FIG. 3, the weight information 31 is associated with items such as "sensor ID", "sensor name", and "weight". “Sensor ID” is identification information for identifying each sensor. "Sensor name" is the name of the sensor. "Weight" is information indicating the weight given to the detection result of the sensor during target detection processing. As for the "weight", the weight determined according to the weather is input by the provision unit 23, which will be described later.

Note that FIG. 3 shows the case where each sensor is one, but when multiple sensors with the same sensor name are mounted, for example, a sensor ID may be assigned to each sensor. That is, when multiple cameras are mounted, each of the multiple cameras may be assigned a sensor ID and a weight.

The control unit 2 determines the weather conditions based on the detection results of a plurality of sensors (for example, the camera 10, the radar device 11, and the LiDAR 12) that detect the surrounding conditions, and outputs weather condition information, which is information about the determined weather conditions. offer.

Acquisition unit 20 acquires detection results of a plurality of sensors that detect surrounding conditions. For example, the acquisition unit 20 acquires detection results from the camera 10, the radar device 11, and the LiDAR 12, respectively. The detection result may be, for example, information on the target (information on the presence or absence of the target, relative speed, position, etc.), or original data used for target detection, such as the image of the camera 10. may

The determination unit 21 (deleted at the time of filing * red underlines are omitted below) determines the weather conditions based on the detection results of the plurality of sensors acquired by the acquisition unit 20 . Specifically, the determination unit 21 determines the weather conditions based on the target detection results detected by each of the plurality of sensors.

For example, for the same target, if the target is not detected by the image of the camera 10 and the LiDAR 12 but is detected by the radar device 11, the determination unit 21 determines that the weather conditions are bad. do. In such a case, the determination unit 21 further determines the type of bad weather based on, for example, deposits adhering to the lens of the camera 10 .

That is, when raindrops adhere to the lens based on the image of the camera 10, the determination unit 21 determines that the bad weather is rain. Further, when the determination unit 21 detects that the lens of the camera 10 is buried in snow based on the image of the camera 10, the determination unit 21 determines that the bad weather is snow. Then, the determining unit 21 determines that the bad weather is dense fog when there is no attached matter on the lens or when fog-like water droplets are attached to the lens.

Note that the determination unit 21 determines that the weather condition is fine when the same target is detected by the camera 10, the radar device 11, and the LiDAR 12 respectively. Thus, by determining the weather conditions based on the target detection results, it is possible to accurately detect the weather.

In addition, the determination unit 21 can determine the weather conditions based on the detection result of marking lines, such as white lines, that mark the road on which the vehicle C travels as targets. This point will be described with reference to FIG.

FIG. 4 is a diagram showing details of weather condition determination processing by the determination unit 21 using white line detection. In FIG. 4, a solid line marking line L that divides the left and right sides of the road on which the vehicle C travels straight (the steering angle is substantially zero) is shown. Note that the demarcation line L is not limited to a solid line, and may be an intermittent demarcation line such as a broken line. Also, the demarcation line L may be a white line or a yellow line. Also, in FIG. 4, it is assumed that the degree of deterioration of the white lines (or yellow lines) of the demarcation lines L is the same, and that the demarcation lines L are the same in blurring and staining.

The existence of the lane marking L is determined by image recognition processing based on the detection data of each sensor, and lane marking position determination processing based on the map data of the navigation device (the lane marking data included in the map data) and the vehicle position data. can.

As shown in FIG. 4, the determination unit 21 first acquires the detection results of the lane marking L detected by each of the plurality of sensors. Subsequently, the determination unit 21 determines weather conditions based on the lane markings L detected by each of the plurality of sensors.

Specifically, as shown in FIG. 4 , the determination unit 21 determines a lane marking L that exists at a distance of at least a predetermined distance from the vehicle C and a lane line L that exists at a distance of less than a predetermined distance from the vehicle C for each of the plurality of sensors. The weather is detected based on the result of comparing the detection result with the division line L existing in the distance.

For example, for the camera 10 and the LiDAR 12, the determination unit 21 detects only the marking line L at a short distance, or the detection accuracy of the marking line L at a long distance is a predetermined threshold or less and is extremely low. When both the near-distance and long-distance lane markings L are detected, it is determined that the weather condition is heavy fog. This is a determination method that utilizes the fact that long-distance visibility is reduced when dense fog occurs. With such a determination method, dense fog can be determined with high accuracy.

Note that the detection accuracy of the marking line L is determined by the continuity of the edge (the part where the contrast suddenly changes (the part where the image is separated at the object boundary) obtained by the image processing of the marking line (the length of the edge straight line at the position where the marking line exists) : In the case of straight lane markings, if the accuracy is high, the straight line will be long, and if the accuracy is low, the straight line will be finely divided) and changes in edge strength (contrast intensity) (In the case of straight lane markings, the accuracy is high If the accuracy is low, the edge strength becomes a stable straight line, and if the accuracy is low, the edge strength becomes an unstable straight line).

Further, for example, the determination unit 21 has a lower recognition accuracy for the long-distance lane marking L than for the short-range lane marking L for the camera 10 and the LiDAR 12, and on the other hand, the radar device 11 has a low recognition accuracy for both the short-range and long-range lane markings. If the recognition accuracies of the lane markings L are the same, it is determined that the weather condition is rainy. This is a determination method that utilizes the fact that when it is raining, even the long-distance lane L can be detected to some extent, unlike the situation where the long-distance lane L can not be detected, such as heavy fog. With such a determination method, rain can be detected with high accuracy.

Further, for example, when both the near-distance and long-distance lane markings L are not detected for the camera 10 and both the near-distance and long-distance lane markings L are detected for the radar device 11 and the LiDAR 12, the determination unit 21 determines that the weather It is determined that the situation is sunny and backlit, or that it is night. It should be noted that whether the backlight is sunny or night can be determined by acquiring the time information. This is a determination method that utilizes the fact that the image captured by the camera 10 becomes a white or black image under backlight or at night, making it impossible to detect a target. As a result, it is possible to accurately detect sunny backlight and night.

Such weather discrimination can be realized by processing using a table showing the relationship between the output state of each sensor and the weather. should be set.

In addition, since the lights of the vehicle C are turned on at night, for example, the determination unit 21 preferably executes the above determination method taking into consideration the influence of such lights. In this manner, the weather conditions can be accurately determined by comparing the detection results of the near-distance marking line L and the long-distance marking line L with respect to the detection results of each of the plurality of sensors.

In FIG. 4, the detection results of the marking line L are divided into two types (short range and long range) and compared. good.

Returning to FIG. 2, the determination unit 22 will be described. The determination unit 22 determines weights to be assigned to each of the plurality of sensors in the target detection process of the target detection device 100 based on the weather conditions determined by the determination unit 21 .

For example, when the determination unit 21 determines that there is dense fog, the determination unit 22 weights the detection results of the camera 10 and the LiDAR 12 less than the weight of the detection results of the radar device 11 in the target detection process. As a result, when dense fog occurs, the detection results of the radar device 11 are more important than the detection results of the camera 10 and the LiDAR 12, so target detection accuracy can be prevented from deteriorating.

Further, when the determination unit 21 determines rain or snow, the determination unit 22 weights the detection result of the camera 10 and the LiDAR 12 less than the weight of the detection result of the radar device 11 . As a result, when it rains or snows, target detection processing is performed with more emphasis placed on the detection results of the radar device 11 than the detection results of the camera 10 and the LiDAR 12, so that the target detection accuracy can be prevented from deteriorating.

The determination unit 22 stores the determined information regarding the weight of each sensor as the weight information 31 in the storage unit 3 and outputs the information regarding the weight to the providing unit 23 .

The provision unit 23 provides the weather condition information weather condition information regarding the weather conditions detected by the determination unit 21 to the outside.

For example, the providing unit 23 provides the surrounding vehicle OC1 existing around the vehicle C with the weather condition information. Specifically, the providing unit 23 provides the weather condition information to the surrounding vehicle OC1 that is scheduled to pass through the position where the vehicle C is present (position where bad weather occurs).

In addition, it is preferable that the content of the weather condition information in such a case be changed depending on whether the surrounding vehicle OC1 is equipped with the target object detection device 101 or not. Specifically, when the surrounding vehicle OC1 is equipped with the target detection device 101, the providing unit 23 provides weather condition information including the weight information 31 determined by the determining unit 22 to the surrounding vehicle OC1.

As a result, the target detection device 101 of the surrounding vehicle OC1 can be changed in advance to a weight corresponding to dense fog, so that it is possible to prevent the target detection accuracy from deteriorating when the vehicle enters dense fog.

The providing unit 23 is not limited to providing the weight information 31 to the surrounding vehicle OC1. For example, the detection result of each sensor when bad weather is detected may be provided as the weather condition information. In such a case, the target detection device 101 of the surrounding vehicle OC1 can determine the weight to be assigned to each of its own sensors based on the detection results of each sensor included in the weather condition information.

On the other hand, when the surrounding vehicle OC1 is not equipped with the target object detection device 101, the provision unit 23 provides weather condition information including the type of weather (for example, heavy fog, rain, snow, etc.), the position of the vehicle C, the current time, and the like. I will provide a. As a result, the driver of the surrounding vehicle OC1 that has received the weather condition information can prepare for bad weather in advance, so that he/she can cope with the occurrence of bad weather without being confused.

In addition to the surrounding vehicle OC1, the providing unit 23 may provide the weather condition information to, for example, a road management terminal (road management server SV) managed by a road administrator who manages the road on which the vehicle C travels. Specifically, the providing unit 23 provides weather condition information including types of weather (for example, heavy fog, rain, snow, etc.), the position of the vehicle C, the current time, and the like. As a result, for example, the road administrator can display a warning against bad weather on an electric bulletin board installed behind the vehicle C or the like.

Next, a processing procedure of processing executed by the providing apparatus 1 according to the embodiment will be described with reference to FIG. FIG. 5 is a flowchart showing a processing procedure of processing executed by the providing device 1 according to the embodiment.

As shown in FIG. 5, first, the acquisition unit 20 acquires the detection results of a plurality of sensors for detecting surrounding conditions from the target detection device 100 (step S101). Subsequently, the determination unit 21 determines weather conditions based on the acquired detection results (step S102).

Subsequently, the provision unit 23 determines whether the weather condition determined by the determination unit 21 is bad weather (step S103). When the weather condition determined by the determination unit 21 is bad weather (step S103, Yes), the provision unit 23 determines whether or not to provide the surrounding vehicle OC1 with the weather condition information (step S104).

When the weather condition information is to be provided to the surrounding vehicle OC1 (step S104, Yes), the providing unit 23 provides the surrounding vehicle OC1 with the weather condition information including the weight information 31 regarding the weight of each sensor determined by the determination unit 22. (step S105). Subsequently, the providing unit 23 provides the road management server SV with weather condition information including the type of weather, the position of the vehicle C, the current time, etc. (step S106), and ends the process.

On the other hand, in step S103, the provision unit 23 does not provide the weather condition information to the outside when the weather condition determined by the determination unit 21 is not bad weather (step S103, No), that is, when the weather is fine. , terminate the process.

Moreover, in step S104, when the provision part 23 does not provide weather condition information to surrounding vehicle OC1 (step S104, No), it transfers to step S106.

As described above, the provision device 1 according to the embodiment includes the acquisition unit 20 , the determination unit 21 and the provision unit 23 . Acquisition unit 20 acquires detection results of a plurality of sensors that detect surrounding conditions. The determination unit 21 determines weather conditions based on the detection results of the plurality of sensors acquired by the acquisition unit 20 . The provision unit 23 provides the weather condition information regarding the weather conditions determined by the determination unit 21 to the outside. As a result, the weather can be detected with high accuracy and the surroundings can be alerted.

Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the invention are not limited to the specific details and representative embodiments so shown and described. Accordingly, various changes may be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and equivalents thereof.

1 Provision Device 2 Control Unit 3 Storage Unit 10 Camera 11 Radar Device 12 LiDAR
20 acquisition unit 21 determination unit 22 determination unit 23 provision unit 31 weight information 100, 101 target detection device C vehicle L lane marking OC1, OC2 surrounding vehicles

Claims (4)

A providing device mounted on a vehicle,
a sensor detection result acquisition unit that acquires detection results of a plurality of sensors that detect surrounding conditions;
a weather condition determination unit that determines weather conditions based on the detection results of the plurality of sensors acquired by the sensor detection result acquisition unit;
a providing unit that provides the weather condition information, which is information about the weather conditions determined by the weather condition determining unit, to the outside ;
The plurality of sensors are
A sensor for detecting targets existing in the surroundings, the sensor being configured to detect, as the target, a marking line that divides a road on which the vehicle travels,
The weather condition determination unit
The weather condition is determined based on the detection results of the target detected by each of the plurality of sensors, and the weather condition is determined based on the lane marking detected by each of the plurality of sensors. is configured as
Based on the result of comparing the detection results of the marking line that exists at a long distance of a predetermined distance or more from the vehicle and the marking line that exists at a short distance of less than the predetermined distance from the vehicle for each of the plurality of sensors. configured to determine the weather conditions;
delivery device. The providing unit
Configured to provide the weather condition information to an in-vehicle device mounted on a peripheral vehicle that exists in the vicinity of the vehicle, or a road management terminal that manages the road on which the vehicle travels,
2. The providing device of claim 1 . The providing unit
configured to provide the surrounding vehicle with the weather condition information including weight information regarding a weight assigned to the detection result of each of the plurality of sensors ;
3. A providing device according to claim 2 . A providing device mounted on a vehicle,
a sensor detection result acquisition unit that acquires detection results of a plurality of sensors that detect surrounding conditions;
a weather condition determination unit that determines weather conditions based on the detection results of the plurality of sensors acquired by the sensor detection result acquisition unit;
a providing unit that provides to the outside weather condition information, which is information about the weather conditions determined by the weather condition determination unit;
with
The plurality of sensors are
A sensor for detecting targets existing in the surroundings, the sensor being configured to detect, as the target, a marking line that divides a road on which the vehicle travels,
The weather condition determination unit
The weather condition is determined based on the detection results of the target detected by each of the plurality of sensors, and the weather condition is determined based on the lane marking detected by each of the plurality of sensors. configured as
The providing unit
configured to provide the weather condition information to an on-vehicle device mounted on a peripheral vehicle existing in the vicinity of the vehicle;
Furthermore, in the case where the surrounding vehicle is a vehicle equipped with a target object detection device that detects the target existing in the surroundings based on the detection results of the plurality of sensors, the on-vehicle device of the vehicle equipped with On the other hand, it is configured to provide, as the weather condition information, information including weight information relating to the weight given to the detection result of each sensor of the plurality of sensors,
When the surrounding vehicle is a non-equipped vehicle that is not equipped with the target object detection device, the type of weather and the above configured to provide information including the location of the vehicle and the current time;
delivery device.

Images