CN119164369A - A water depth detection system and electronic equipment based on polarized light - Google Patents
A water depth detection system and electronic equipment based on polarized light Download PDFInfo
- Publication number
- CN119164369A CN119164369A CN202411524944.9A CN202411524944A CN119164369A CN 119164369 A CN119164369 A CN 119164369A CN 202411524944 A CN202411524944 A CN 202411524944A CN 119164369 A CN119164369 A CN 119164369A
- Authority
- CN
- China
- Prior art keywords
- polarized light
- parallax
- eye
- image
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 123
- 238000001514 detection method Methods 0.000 title claims abstract description 29
- 230000010287 polarization Effects 0.000 claims abstract description 46
- 238000009825 accumulation Methods 0.000 claims abstract description 35
- 238000004364 calculation method Methods 0.000 claims abstract description 18
- 238000003384 imaging method Methods 0.000 claims abstract description 13
- 230000002159 abnormal effect Effects 0.000 claims abstract description 11
- 230000000007 visual effect Effects 0.000 claims abstract 3
- 238000012937 correction Methods 0.000 claims description 9
- 238000013135 deep learning Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 abstract description 5
- 238000000034 method Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 7
- 230000005856 abnormality Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000005286 illumination Methods 0.000 description 3
- 241000192710 Microcystis aeruginosa Species 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000004576 sand Substances 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000003313 weakening effect Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000002834 transmittance Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C13/00—Surveying specially adapted to open water, e.g. sea, lake, river or canal
- G01C13/008—Surveying specially adapted to open water, e.g. sea, lake, river or canal measuring depth of open water
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01F—MEASURING VOLUME, VOLUME FLOW, MASS FLOW OR LIQUID LEVEL; METERING BY VOLUME
- G01F23/00—Indicating or measuring liquid level or level of fluent solid material, e.g. indicating in terms of volume or indicating by means of an alarm
- G01F23/22—Indicating or measuring liquid level or level of fluent solid material, e.g. indicating in terms of volume or indicating by means of an alarm by measuring physical variables, other than linear dimensions, pressure or weight, dependent on the level to be measured, e.g. by difference of heat transfer of steam or water
- G01F23/28—Indicating or measuring liquid level or level of fluent solid material, e.g. indicating in terms of volume or indicating by means of an alarm by measuring physical variables, other than linear dimensions, pressure or weight, dependent on the level to be measured, e.g. by difference of heat transfer of steam or water by measuring the variations of parameters of electromagnetic or acoustic waves applied directly to the liquid or fluent solid material
- G01F23/284—Electromagnetic waves
- G01F23/292—Light, e.g. infrared or ultraviolet
Landscapes
- Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- General Physics & Mathematics (AREA)
- Thermal Sciences (AREA)
- Fluid Mechanics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Hydrology & Water Resources (AREA)
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Image Processing (AREA)
Abstract
The application discloses a polarized light-based water depth detection system and electronic equipment, wherein the system comprises a first imaging unit, a parallax calculation unit, a judgment unit, a polarization acquisition unit and a water accumulation depth determination unit, wherein the first imaging unit is used for acquiring left-eye images and right-eye images in a front view range of a vehicle, the parallax calculation unit is used for calculating a first parallax image based on the left-eye images and the right-eye images, the judgment unit is used for judging whether parallax in a front road area in the first parallax image is abnormal or not, if yes, a polarization control instruction is generated, the polarization acquisition unit is used for responding to the polarization control instruction, acquiring left-eye polarized light images and right-eye polarized light images in the front view range of the vehicle, and the water accumulation depth determination unit is used for calculating the water accumulation depth of a water accumulation area based on the left-eye polarized light images and the right-eye polarized light images. According to the technical scheme, polarized light is introduced under the condition that the visual field range and the light intensity are not affected, and road ponding depth detection is carried out based on the polarized light and an image processing mode, so that the accuracy of ponding depth detection is improved.
Description
Technical Field
The application relates to the technical field of automatic driving, in particular to a polarized light-based water depth detection system and electronic equipment.
Background
With the continuous development of autopilot technology, the applicable scenes thereof are becoming increasingly rich. In wading scenes (such as rainy days and off-road), an automatic driving system of the vehicle is required to have a water depth detection function.
In the prior art, due to the limitation of processing the water surface image (such as water surface reflection), an ultrasonic sensor can be additionally arranged on the vehicle so as to calculate the water depth according to ultrasonic echo, and on the other hand, the vehicle can be positioned by positioning signals (such as GPS positioning signals and Beidou positioning signals) and then the water depth calculation is carried out by combining road information. The first solution is undoubtedly increasing the cost of the vehicle and the ultrasound is subject to interference from environmental factors, while the second solution requires accurate map information (road surface height, gradient), whereas for some roads, in particular urban and rural roads, there is no accurate map information and the positioning signal is also subject to errors, affecting the accuracy of the road surface height. Therefore, how to detect the road ponding water depth in different fields is still a problem to be solved.
The polarized light detection technology can well solve the problem of light reflection on the water surface in an image, as shown in fig. 1 and fig. but the following problems exist when the polarized light detection technology is applied to the automatic driving field:
1) The polarized light camera can weaken the light intensity of light sources such as street lamps, traffic signal lamps and the like at night or in environments with darker light, and the difficulty of image recognition can be increased;
2) The special configuration of the polarized light camera affects the field of view, creating a field of view blind zone.
Disclosure of Invention
The application aims to improve the accuracy of the accumulated water depth detection by detecting the accumulated water depth of a road based on polarized light and image processing modes.
The technical scheme of the first aspect of the application is that the polarized light-based water depth detection system comprises a first imaging unit, a parallax calculation unit, a judgment unit, a polarization acquisition unit and a water accumulation depth determination unit, wherein the first imaging unit is used for acquiring left-eye polarized light images and right-eye polarized light images in a front view range of a vehicle, the left-eye images and the right-eye images are RGB images, the parallax calculation unit is used for calculating a first parallax map based on the left-eye images and the right-eye images, the judgment unit is used for judging whether parallax in a front road area in the first parallax map is abnormal or not, if yes, a polarization control instruction is generated, the polarization acquisition unit is used for responding to the polarization control instruction, the left-eye polarized light images and the right-eye polarized light images in the front view range of the vehicle are acquired, and the water accumulation depth determination unit is used for calculating the water accumulation depth of a water accumulation area based on the left-eye polarized light images and the right-eye polarized light images.
In any one of the above technical solutions, further, the judging unit is configured to judge that the parallax of the front road area in the first parallax map is abnormal when the parallax of the front road area in the first parallax map meets a preset judging condition, wherein the preset judging condition comprises at least one of the following conditions that the first parallax missing proportion in the front road area in the first parallax map is greater than or equal to a first preset proportion, the parallax map of the front road area in the first parallax map has a corrugated shape, and at least one underground target exists in the first parallax map.
In any one of the above technical schemes, the polarization acquisition unit further comprises a driving subunit, wherein the driving subunit is used for responding to the polarization control instruction to generate the driving instruction, and the polarization subunit at least comprises a fixing device, a polarizing plate, a moving device and a driving device, wherein the polarizing plate, the moving device and the driving device are sequentially connected and are arranged in the fixing device, the fixing device is used for installing the polarization subunit on a lens of the vehicle-mounted binocular camera, the driving device is used for responding to the driving instruction to control the moving device to make reciprocating movement so as to enable the polarizing plate to stretch out or stretch back, and when the polarizing plate stretches out, the polarizing plate is positioned in front of the lens.
In any one of the above technical schemes, the polarization acquisition unit further comprises a polarization calculation subunit, wherein the polarization calculation subunit is used for responding to the polarization control instruction and generating a left-eye polarized light image and a right-eye polarized light image based on the left-eye image, the right-eye image and Stokes parameters.
In any one of the above technical solutions, the ponding depth determining unit is further configured to calculate a second parallax map based on the left-eye polarized light image and the right-eye polarized light image, determine whether a ponding area exists in the front road area based on the second parallax map, if so, calculate a fitted road surface plane of the front road area based on the second parallax map, calculate a fitted water surface plane of the front road area based on the effective parallax in the first parallax map, and calculate the ponding depth of the ponding area based on the fitted water surface plane and the fitted road surface plane.
In any one of the above technical solutions, the ponding depth determining unit is further configured to obtain a measured pavement plane of the road area in front based on the positioning signal of the current position of the vehicle in a query manner in the map database, correct the fitted pavement plane based on the measured pavement plane, and calculate the ponding depth of the ponding area based on the corrected fitted pavement plane and the fitted water surface plane.
In any one of the above technical solutions, further, based on the second parallax map, judging whether a ponding area exists in the front road area, specifically including counting a second parallax missing proportion in the front road area in the second parallax map, judging whether the second parallax missing proportion is smaller than a second preset proportion, if yes, judging that the ponding area exists, and if not, judging that the ponding area does not exist.
In any one of the above technical solutions, further, based on the effective parallax in the first parallax map, a fitting water surface plane of the front road area is calculated, and specifically includes obtaining a left road surface boundary and a right road surface boundary in the first parallax map based on the effective parallax in the first parallax map, obtaining road point pairs in the left road surface boundary and the right road surface boundary according to preset intervals, generating road correction line segments based on the road point pairs, and calculating the fitting water surface plane of the front road area based on the road correction line segments.
In any one of the above technical solutions, the water accumulation depth determining unit is further configured to obtain ripple information of the water accumulation area based on the left eye image and the right eye image respectively, obtain turbidity information of the water accumulation area based on the left eye polarized light image and the right eye polarized light image respectively, and predict the water accumulation depth of the water accumulation area according to a preset corresponding relation based on the ripple information and the turbidity information, wherein the preset corresponding relation is determined by a deep learning network through marked sample images.
According to a second aspect of the present application, there is provided an electronic device provided with the polarized light-based water depth detection system according to any one of the first aspect.
The beneficial effects of the application are as follows:
According to the technical scheme, when the parallax in the road area in front of the first parallax map is judged to be abnormal, the polarization acquisition unit is controlled to obtain the corresponding polarized light image, the characteristic of polarized light is utilized to realize the prediction of the ponding depth of the road ponding area, the problems of weakening of light intensity and blind areas of vision caused by directly using the polarized light can be avoided, the polarized light and image processing are creatively combined, and the accuracy of ponding depth detection is improved.
The application also utilizes the characteristic of eliminating water surface reflection of polarized light energy, combines the characteristic with a parallax calculation principle, calculates the road surface hidden under the water surface by utilizing a polarized light image after judging that a water accumulation area exists, obtains the road surface height, and combines an RGB image to obtain the water surface height so as to realize the calculation of the water accumulation depth.
Drawings
The advantages of the foregoing and/or additional aspects of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a schematic representation of a prior art contrast of polarized light images;
FIG. 2 is a schematic representation of another prior art contrast of polarized light images;
FIG. 3 is a schematic block diagram of a polarized light based water depth detection system according to one embodiment of the application;
Fig. 4 is a schematic illustration of a pavement boundary according to one embodiment of the present application.
Detailed Description
In order that the above-recited objects, features and advantages of the present application will be more clearly understood, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description. It should be noted that, without conflict, embodiments of the present application and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, but the present application may be practiced in other ways than those described herein, and therefore the scope of the present application is not limited to the specific embodiments disclosed below.
Example 1:
as shown in fig. 3, the present embodiment provides a polarized light-based water depth detection system 100, which includes:
The first imaging unit 10, the first imaging unit 10 is configured to acquire a left-eye image and a right-eye image in a front view range of the vehicle, where the left-eye image and the right-eye image are RGB images;
A parallax calculation unit 20, the parallax calculation unit 20 being configured to calculate a first parallax map based on the left-eye image and the right-eye image;
The judging unit 30 is configured to judge whether the parallax in the road area in front of the first parallax map is abnormal, if yes, generate a polarization control instruction, and if not, continue to acquire the left-eye image, the right-eye image and the first parallax map by the first imaging unit 10 and the parallax calculating unit 20;
A polarization acquisition unit 40, the polarization acquisition unit 40 being configured to acquire a left-eye polarized light image and a right-eye polarized light image in a front view range of the vehicle in response to the polarization control instruction;
The ponding depth determining unit 50, the ponding depth determining unit 50 is used for calculating the ponding depth of the ponding area based on the left eye polarized light image and the right eye polarized light image.
In this embodiment, a binocular camera is installed in front of the autopilot vehicle, and the binocular camera includes a left camera and a right camera installed on a beam, and image data collected by the two cameras is processed (such as distortion correction and clipping) by a first imaging unit 10 built in the camera, so that RGB images with preset resolution can be output, and can be respectively recorded as a left-eye image and a right-eye image.
The method comprises the steps of firstly, taking a vertical point of a binocular camera optical center on the ground as an origin, and establishing a world coordinate system, wherein the horizontal right direction is an x-axis positive direction, the front of a vehicle is a z-axis positive direction, and the vertical upward direction is a y-axis positive direction. And then, by utilizing the parallax calculation principle, a parallax image (first parallax image) corresponding to the left-eye image is calculated by using the parallax calculation unit 20 as a reference, and further, the position of any target in the left-eye image under the current world coordinate system can be obtained based on the parameters of the binocular camera, wherein the left-eye image can be used for target identification. Finally, the information is in one-to-one correspondence, so that the category of the object in front of the vehicle and the distance between the object and the vehicle can be obtained.
In this embodiment, the forward road area during the running of the vehicle may be considered as a fixed area in the image, such as the middle area of the lower part in the image.
It should be noted that, in the parallax calculation process, pixel point matching is mainly performed based on pixel textures in the left-eye image and the right-eye image, but for a scene with water accumulation and reflection, a specular reflection phenomenon occurs, and in addition, light influence in different scenes needs to be considered. These factors are reflected in the disparity map and may be represented as a disparity loss, a false/underground object, a discontinuous ripple, etc., wherein the disparity of the discontinuous ripple type mainly refers to the disparity caused by the water surface fluctuation. It will be appreciated by those skilled in the art that images obtained by polarized light can eliminate the reflection of light caused by road water and enhance the texture of the image.
Therefore, it is possible to take some characteristics of the road water in the disparity map as a basic condition for judging whether or not the road water phenomenon occurs, and the judging unit 30 judges whether or not there is an abnormality in the disparity in the road area ahead in the first disparity map. If these conditions are met, that is, if it is determined that there is an abnormality in the parallax in the road area ahead in the first parallax map, the water depth is detected, and the determination unit 30 generates a polarization control instruction to control the polarization acquisition unit 40 to obtain a left-eye polarized light image and a right-eye polarized light image in the front view range of the vehicle, otherwise, normal parallax calculation and image detection are continued. In this way, the above-mentioned problems associated with the direct use of polarized light can be avoided.
The required RGB image and polarized light image can be obtained by providing a special image sensor. For example, on the basis of a conventional image sensor, a region with a certain size (such as 2 x 2) is taken as a repeating unit, polarization filters with different angles are arranged in the unit according to a certain rule at corresponding pixels, based on the polarization control instruction, when a polarized light image is not required to be acquired, only the data of the pixels which are not shielded by the polarization filters are acquired to obtain an RGB image, and when the polarized light image is required to be acquired, only the data of the pixels which are shielded by the polarization filters are acquired to obtain the polarized light image.
After the left-right polarized light image is obtained, a second parallax image corresponding to the left-right polarized light image can be obtained by the ponding depth determining unit 50 through a parallax calculation mode, and then a pavement plane hidden under the water surface under the current world coordinate system can be obtained based on a pavement fitting mode of the parallax image, so that the pavement height is obtained. By the method, the method of inquiring the road surface information in the map through the vehicle positioning signals to obtain the road surface height can be avoided, and the problems of inaccurate positioning signals and missing map are solved.
When the water surface plane of the accumulated water is obtained, the water surface plane under the current world coordinate system can be obtained by fitting based on parallax data except the water accumulation area/parallax abnormal area in the first parallax map, so that the water surface height is obtained. So far, the difference value of the water surface height and the road surface height is the accumulated water depth.
Since the water surface is a plane, the plane may be obtained by obtaining the edge of the water surface in the image and then performing plane fitting. And for the pavement plane, the lowest point of the pavement in the second parallax map can be selected as a correction parameter of the pavement plane so as to obtain the potential maximum water accumulation depth and avoid danger in the vehicle wading process.
In the embodiment, the polarized light technology is combined with the binocular vision technology, and the polarized light is only introduced when the road water accumulation phenomenon is judged, so that the problems of 'blind area of view and light weakening' caused by using the polarized light in a normal running environment are avoided.
In any of the foregoing embodiments, further, in order to avoid frequent activation of the polarization acquisition unit 40, the parallax abnormality detection in the front road area in the first parallax map is purposefully optimized, and the determination unit 30 is configured to:
When the parallax of the front road area in the first parallax map meets a preset judging condition, judging that the parallax of the front road area in the first parallax map is abnormal, wherein the preset judging condition comprises at least one of the following conditions:
Considering that when the integral reflection exists in the front road area, pixel textures in the area are deleted in the left eye image and the right eye image, so that parallax does not exist in the part of the parallax map, therefore, the number of parallax points with the parallax value of 0 in the front road area in the first parallax map can be counted to obtain a first parallax deletion proportion/area, and the first parallax deletion proportion in the front road area in the first parallax map is larger than or equal to a first preset proportion as the preset judging condition,
Considering that there are a plurality of running vehicles in the road, the front vehicles can cause the water surface to fluctuate after running through the water accumulation area, and the raindrops in the rain process can also fluctuate on the water surface, under the influence of certain illumination, the 'ripple' can be used as the texture of the image, and can be reflected in the parallax map, the parallax map corresponding to the situation can also be used as the above-mentioned preset judging condition, namely that the parallax map of the front road area in the first parallax map has ripple/scale distribution,
Considering that under certain illumination conditions, reflection exists in a ponding area, such as a tail lamp of a front vehicle, trees on a roadside, a lamp post and the like, after the corresponding parallax map is subjected to V-map projection, objects in the V-parallax map and fitting ground can be obtained in a clustering and fitting mode, and an object formed by the reflection is positioned below the fitting ground and is determined to be a false target, so that the target can be used as the preset determination condition, namely, at least one target positioned underground exists in the first parallax map.
In any of the above embodiments, further, considering implementation cost and difficulty of technology, the present embodiment also refers to a shutter structure of a rolling-shutter camera, and proposes an implementation manner of an external polarizing plate/polarizer, where each lens of a binocular camera is blocked by controlling movement of the polarizing plate, so as to obtain a polarized light image. Accordingly, the corresponding polarization acquisition unit 40 includes:
The driving subunit is used for responding to the polarization control instruction and generating a driving instruction;
a polarization subunit, at least comprising a fixing device, a polarization plate, a motion device and a driving device, wherein,
The polarizing plate, the moving device and the driving device are sequentially connected and are arranged in the fixing device, the fixing device is used for arranging the polarizing subunit on the lens of the vehicle-mounted binocular camera, and the fixing device can be arranged into a cover shape, so that the mounting is convenient;
the driving device is used for responding to the driving instruction and controlling the moving device to move back and forth so as to enable the polarizing plate to extend or retract, and when the polarizing plate extends, the polarizing plate is positioned in front of the lens.
Specifically, it can be set that when the driving device receives the driving instruction, a first driving signal is generated, the moving device is driven to move, the moving device is horizontally moved to extend out of the polarizing plate to enable the polarizing plate to be blocked in front of the lens, external light enters the lens of the vehicle-mounted binocular camera through the polarizing plate, photoelectric conversion is carried out by a conventional image sensor, polarized light images can be obtained after corresponding processing, delay time can be set to be 1 second, after the vehicle-mounted binocular camera collects a plurality of images, a second driving signal is generated by the driving device, the moving device is enabled to reversely move, the polarizing plate is retracted, and external light directly enters the lens of the vehicle-mounted binocular camera.
The parameters such as transmittance, extinction ratio, acceptance angle, etc. of the polarizing plate may be set according to actual requirements.
In any of the above embodiments, further, a deep learning network may be further utilized to generate a corresponding polarized light image based on an optical algorithm through the RGB image and stokes parameters. Thus, the polarization acquisition unit 40 further includes:
And the polarization calculating subunit is used for responding to the polarization control instruction and generating a left-eye polarized light image and a right-eye polarized light image based on the left-eye image, the right-eye image and the Stokes parameter.
In any of the above embodiments, further, considering that the factor that causes the first parallax map to satisfy the preset determination condition is not necessarily accumulated water on the road surface, but may be other situations, such as a large area of missing of the parallax on the road surface caused by "broken roads" in the rural roads, strong illumination effect in the urban roads, and a large-scale billboard projection lamp at night. Therefore, it is also necessary to make a secondary determination as to whether or not there is a water accumulation area in the road area ahead, the water accumulation depth determination unit 50 being configured to:
calculating a second parallax map based on the left-eye polarized light image and the right-eye polarized light image;
Judging whether a ponding area exists in the front road area or not based on the second parallax map, namely whether all/part of parallax in the area with abnormal parallax of the first parallax map in the second parallax map is normal (the parallax value is not 0), if so, calculating a fitting road surface plane of the front road area by adopting a plane fitting mode based on the second parallax map, otherwise, indicating that the abnormality in the first parallax map is not caused by road ponding, and at the moment, no subsequent ponding depth detection is needed;
And calculating a fitting water surface plane of the front road area based on the effective parallax in the first parallax map, wherein the effective parallax is the parallax except the parallax abnormal area in the front road area in the first parallax map, and can be positioned in the front road area or positioned outside the front road area.
It will be appreciated by those skilled in the art that objects on both sides of the road, such as curbs/shoulders, rails, plants, etc., contained in the left eye image may be identified by means of image recognition. The objects correspond to a target frame in the left-eye image, the target frame contains position information (pixel coordinates of the target frame) in the image, and the left-eye image corresponds to the parallax map, so that the image position of the objects in the first parallax map can be obtained, and the corresponding parallax value of the objects can be obtained.
For example, when fitting the water surface plane, the parallax values of the bottoms of the road edges at the two sides of the road and the bottom of the railing can be selected as the data of the water surface plane. In the fitting process, the parallax value of the target can be converted into 3D point cloud data under the current world coordinate system based on parameters (internal reference and external reference) of the vehicle-mounted binocular camera, and a plane fitting mode is combined to obtain a fitting water surface plane.
Based on the fitting water surface plane and the fitting road surface plane, the water accumulation depth of the water accumulation area can be calculated, namely the difference value of the y-axis heights of the two planes under the current coordinate system.
In any of the foregoing embodiments, further, based on the second disparity map, determining whether a ponding area exists in the front road area specifically includes:
considering the complexity of the road, and particularly for non-dense disparity maps, there is a disparity cavitation phenomenon, so after obtaining a second disparity map calculated based on the polarized light image, a second disparity missing proportion in a road area in front of the second disparity map, that is, a proportion of points with a disparity value of 0, is counted;
judging whether a second parallax missing proportion in a road area in front of the second parallax map is smaller than a second preset proportion, if so, judging that a ponding area exists, and if not, judging that the ponding area does not exist.
In any of the above embodiments, further, in order to improve accuracy of fitting the road surface plane, for a vehicle traveling in a city, the fitted road surface plane may be corrected by using road surface information in a map obtained by a positioning signal, and the accumulated water depth determining unit 50 may be configured to:
Acquiring a measured pavement plane of a front road area by inquiring in a map database based on a positioning signal of the current position of the vehicle; if no positioning signal is acquired or no measured road surface is acquired, the road surface correction process is skipped.
And correcting the fitted pavement plane based on the measured pavement plane, and calculating the ponding depth of the ponding region based on the corrected fitted pavement plane and the corrected fitted water surface plane. When the difference is smaller than a preset threshold value, the fitted road surface can be corrected according to a set weight coefficient, for example, the weight of the measured road surface plane is set to be 20%, and the weight of the original fitted road surface plane is set to be 80%.
In any of the above embodiments, further, as shown in fig. 4, considering that the water surface plane is not affected by the terrain, the fitted water surface plane may be corrected based on the boundary of the water surface, and the line between two points on the boundary should be located on the plane of the water surface. Accordingly, the effective parallax is set as the parallax of the first parallax map excluding the parallax abnormality region in the front road region, and the fitting water surface plane of the front road region is calculated based on the effective parallax in the first parallax map, specifically including:
Based on the effective parallax in the first parallax map, the left boundary and the right boundary of the road surface in the first parallax map are obtained, namely, the curve part in fig. 4, it should be noted that the boundary can also be set as a part of curve in a certain range, for example, 20% of the whole width of the outermost side is taken as the obtained road surface boundary;
According to a preset interval, if the maximum step length is set to be 5 pixels/parallax points, a random mode is adopted to respectively obtain a road surface point pair (A, B) in a left boundary and a right boundary of a road surface, and a road surface correction line segment is generated based on the road surface point pair, wherein the road surface point pair (A, B) can be connected with a straight line which is positioned on a fitting water surface plane;
Specifically, the point a may be randomly selected on the left boundary, then a step length is randomly selected within a preset interval, for example, 3, and then the row coordinate of the point a is moved up/down by 3 pixels, and the corresponding point B on the right boundary is the selected point, and forms a road point pair (a, B) with the point a.
And calculating a fitting water surface plane of the front road area based on the road surface correction line segment.
Example 2:
Considering that the polarized light technology can not only eliminate water surface reflection, but also detect the concentration of liquid/gas by a polarized light rotation method, a polarized light imaging method and the like, therefore, the concentration/turbidity of a road ponding area can be predicted based on the polarized light image combined with a deep learning network.
Based on the embodiment, in the embodiment, by introducing a deep learning network, network training is performed to obtain the corresponding relation between the ponding depth, the ponding turbidity and the water surface ripple, so as to predict the road ponding depth based on the acquired RGB image and polarized light image. Accordingly, the water accumulation depth determination unit 50 is configured to:
the ripple information of the ponding area is obtained based on the left eye image and the right eye image respectively;
acquiring turbidity information of a water accumulation area based on the left-eye polarized light image and the right-eye polarized light image respectively;
Based on the ripple information and the turbidity information, predicting the water accumulation depth of the water accumulation area according to a preset corresponding relation, wherein the preset corresponding relation is determined by a deep learning network through a marked sample image.
Specifically, for ponding in urban roads, the following conditions may be set:
1. Under the condition of the same rainfall, if the accumulated water is deeper, the ripple generated by the raindrops falling on the water surface is larger and the water bloom is smaller, otherwise, the ripple is smaller and the water bloom is larger;
2. under the condition of considering the flow of the road ponding, the road ponding is deeper, the sand content in the water is larger, the ponding is more turbid, otherwise, the sand content is smaller, and the ponding is more clear.
Based on the set conditions, marking the acquired data, marking the water depth corresponding to the acquired data as known data, constructing a database of a deep learning network so as to train the database, and determining the (preset) corresponding relation with the water depth based on the size of water surface waves and the turbidity of the water accumulation in the image so as to predict the water accumulation depth based on the acquired image.
Considering that the vehicle-mounted binocular camera can acquire left and right eye images (RGB) and left and right eye polarized light images, the images are respectively input into the network in order to improve the accuracy of the deep learning network on the prediction of the accumulated water depth. The method comprises the steps of obtaining the size of water surface waves from left and right eye images to serve as wave information, obtaining the accumulated water turbidity from left and right eye polarized light images to serve as turbidity information, and predicting the accumulated water depth of an accumulated water area according to the wave information and the turbidity information and the preset corresponding relation obtained after deep learning network training.
Example 3:
The embodiment also provides an electronic device, which is provided with the polarized light-based water depth detection system according to any one of the embodiments, and the electronic device may include a processor, a memory, an input/output interface, and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the data acquisition unit and the input device are connected to the system bus through the input/output interface.
It should be noted that the composition structure of the electronic device may be adjusted according to actual needs.
Thus, various embodiments of the present application have been described in detail. In order to avoid obscuring the concepts of the application, some details known in the art have not been described. How to implement the solutions disclosed herein will be fully apparent to those skilled in the art from the above description.
While certain specific embodiments of the application have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the application.
The steps in the application can be sequentially adjusted, combined and deleted according to actual requirements.
Although the application has been disclosed in detail with reference to the accompanying drawings, it is to be understood that such description is merely illustrative and is not intended to limit the application of the application. The scope of the application is defined by the appended claims and may include various modifications, alterations and equivalents of the application without departing from the scope and spirit of the application.
Claims (10)
1. A polarized light-based water depth detection system, the system comprising:
the system comprises a first imaging unit, a second imaging unit and a third imaging unit, wherein the first imaging unit is used for acquiring left-eye images and right-eye images in a front visual field range of a vehicle, and the left-eye images and the right-eye images are RGB images;
A parallax calculation unit for calculating a first parallax map based on the left-eye image and the right-eye image;
the judging unit is used for judging whether the parallax in the front road area in the first parallax map is abnormal or not, and if yes, a polarization control instruction is generated;
The polarization acquisition unit is used for responding to the polarization control instruction and acquiring a left-eye polarized light image and a right-eye polarized light image in the front visual field range of the vehicle;
the ponding depth determining unit is used for calculating the ponding depth of the ponding area based on the left-eye polarized light image and the right-eye polarized light image.
2. The polarized light-based water depth detection system according to claim 1, wherein the determination unit is configured to:
when the parallax of the front road area in the first parallax map meets a preset judging condition, judging that the parallax of the front road area in the first parallax map is abnormal, wherein the preset judging condition comprises at least one of the following conditions:
The first parallax absence proportion in the front road region in the first parallax map is greater than or equal to a first preset proportion,
The disparity map of the forward road region in the first disparity map has a moire-like shape,
At least one object located underground is present in the first disparity map.
3. The polarized light-based water depth detection system according to claim 1, wherein the polarization acquisition unit comprises:
The driving subunit is used for responding to the polarization control instruction and generating a driving instruction;
a polarization subunit, wherein the polarization subunit at least comprises a fixing device, a polarization plate, a motion device and a driving device,
The polarizing plate, the motion device and the driving device are connected in sequence and are arranged in the fixing device, the fixing device is used for arranging the polarizing subunit on the lens of the vehicle-mounted binocular camera,
The driving device is used for responding to the driving instruction and controlling the moving device to move back and forth so as to extend or retract the polarizing plate,
When the polarizing plate extends out, the polarizing plate is positioned in front of the lens.
4. The polarized light-based water depth detection system according to claim 1, wherein the polarization acquisition unit comprises:
And the polarization calculation subunit is used for responding to the polarization control instruction and generating the left-eye polarized light image and the right-eye polarized light image based on the left-eye image, the right-eye image and Stokes parameters.
5. The polarized light-based water depth detection system of claim 1, wherein the water depth determination unit is configured to:
Calculating a second parallax map based on the left-eye polarized light image and the right-eye polarized light image;
Judging whether a ponding area exists in the front road area or not based on the second parallax map, if so, calculating a fitted road surface plane of the front road area based on the second parallax map;
Calculating a fitting water surface plane of the front road area based on the effective parallax in the first parallax map;
And calculating the ponding depth of the ponding area based on the fitting water surface plane and the fitting road surface plane.
6. The polarized light-based water depth detection system of claim 5, wherein the water depth determination unit is configured to:
acquiring a measured pavement plane of the front road area by inquiring in a map database based on a positioning signal of the current position of the vehicle;
And correcting the fitted pavement plane based on the measured pavement plane, and calculating the ponding depth of the ponding region based on the corrected fitted pavement plane and the fitted water surface plane.
7. The polarized light-based water depth detection system according to claim 5 or 6, wherein the determining whether there is a water accumulation area in the front road area based on the second parallax map, specifically comprises:
counting a second parallax missing proportion in a road area in front of the second parallax map;
Judging whether the second parallax error proportion is smaller than a second preset proportion, if so, judging that the water accumulation area exists, and if not, judging that the water accumulation area does not exist.
8. The polarized light-based water depth detection system of claim 5, wherein the calculating the fitted water surface plane of the forward road region based on the effective parallax in the first parallax map comprises:
acquiring a road surface left boundary and a road surface right boundary in the first disparity map based on the effective disparity in the first disparity map;
Respectively acquiring a road surface point pair from the left boundary and the right boundary of the road surface according to a preset interval, and generating a road surface correction line segment based on the road surface point pair;
and calculating a fitting water surface plane of the front road area based on the road surface correction line segment.
9. The polarized light-based water depth detection system according to any one of claims 1 to 4, wherein the water depth determination unit is configured to:
based on the left eye image and the right eye image respectively, ripple information of the ponding area is obtained;
Acquiring turbidity information of the ponding region based on the left-eye polarized light image and the right-eye polarized light image respectively;
And predicting the water accumulation depth of the water accumulation area according to a preset corresponding relation based on the ripple information and the turbidity information, wherein the preset corresponding relation is determined by a deep learning network through a marked sample image.
10. An electronic device, characterized in that the electronic device is provided with a polarized light based water depth detection system according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411524944.9A CN119164369A (en) | 2024-10-30 | 2024-10-30 | A water depth detection system and electronic equipment based on polarized light |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411524944.9A CN119164369A (en) | 2024-10-30 | 2024-10-30 | A water depth detection system and electronic equipment based on polarized light |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN119164369A true CN119164369A (en) | 2024-12-20 |
Family
ID=93882079
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411524944.9A Pending CN119164369A (en) | 2024-10-30 | 2024-10-30 | A water depth detection system and electronic equipment based on polarized light |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119164369A (en) |
-
2024
- 2024-10-30 CN CN202411524944.9A patent/CN119164369A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109752701B (en) | Road edge detection method based on laser point cloud | |
| RU2636120C2 (en) | Three-dimensional object detecting device | |
| US20210104058A1 (en) | Estimating distance to an object using a sequence of images recorded by a monocular camera | |
| US9558546B2 (en) | Three-dimensional object detection device, and three-dimensional object detection method | |
| CN104011737B (en) | Method for detecting mist | |
| Hautière et al. | Real-time disparity contrast combination for onboard estimation of the visibility distance | |
| US6813370B1 (en) | Lane marker recognizing apparatus | |
| US9569675B2 (en) | Three-dimensional object detection device, and three-dimensional object detection method | |
| JP5664152B2 (en) | Imaging device, in-vehicle imaging system, and object identification device | |
| CN103080978B (en) | Object recognition equipment | |
| RU2563534C1 (en) | Detector of solid objects and detection of solid objects | |
| US9591274B2 (en) | Three-dimensional object detection device, and three-dimensional object detection method | |
| RU2571368C1 (en) | Device for detecting three-dimensional objects, method of detecting three-dimensional objects | |
| CN1223964C (en) | Apparatus and method for measuring vehicle queue length | |
| EP2463806A1 (en) | Vehicle detection device and vehicle detection method | |
| CN101750049A (en) | Monocular vision vehicle distance measuring method based on road and vehicle information | |
| JP3727400B2 (en) | Crossing detection device | |
| CN106978774A (en) | A kind of road surface pit automatic testing method | |
| CN112749584B (en) | Vehicle positioning method based on image detection and vehicle-mounted terminal | |
| CN103679121A (en) | Method and system for detecting roadside using visual difference image | |
| CN109241855B (en) | A detection method of intelligent vehicle driving area based on stereo vision | |
| JP3747863B2 (en) | In-vehicle distance measuring device | |
| CN112101316B (en) | Target detection method and system | |
| JP3807583B2 (en) | Road area determination device | |
| CN103453890A (en) | Nighttime distance measuring method based on taillight detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |