CN114022860A - Target detection method, device and electronic device - Google Patents

Abstract

Embodiments of the present invention provide a target detection method, device, and electronic device, wherein the target detection method includes: acquiring an original image collected by a vehicle sensor and road boundary information from a high-precision map; The road boundary information is generated, and a lane image is generated; N ROI images are extracted from the lane image, and N is a positive integer; The target detector is obtained by training an initial target detection network constructed by sample target pairs. The embodiments of the present invention can effectively improve the extraction effect of the region of interest image, thereby improving the accuracy of target detection for the region of interest image.

Description

Target detection method, device and electronic device

technical field

The present invention relates to the technical field of image processing, and in particular, to a target detection method, device and electronic device.

Background technique

As we all know, autonomous driving technology has gradually entered people's lives; due to the complexity of road traffic, the detection of different obstacles is the key to the application of autonomous driving technology; currently, based on deep learning models and images obtained by vehicle sensors, obstacles can be detected. object detection. In the prior art, a road area identified from an image acquired by a vehicle sensor is generally input into a deep learning model as a region of interest (ROI) image to detect objects such as obstacles. However, the prior art usually has the problem of poor ROI image extraction effect when performing target detection.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a target detection method, device, and electronic device, so as to solve the problem that the ROI extraction effect is generally poor when performing target detection in the prior art.

In order to solve the above-mentioned technical problems, the present invention is achieved in this way:

In a first aspect, an embodiment of the present invention provides a target detection method, including:

acquiring a lane image, the lane image including a road area;

Extracting N ROI images in the lane image along a first direction, wherein the first direction matches the length extension direction of the road area, and the aspect ratios of the N ROI images are all equal to A preset ratio, each of the region of interest images includes all the areas where the road region is located between the boundaries of the region of interest images along the first direction, and N is a positive integer;

Each of the region-of-interest images is respectively input into a target detector to obtain a target detection result, and the target detector is obtained by training an initial target detection network constructed through sample target pairs.

In a second aspect, an embodiment of the present invention further provides a target detection device, including:

The first acquisition module is used to acquire the original image collected by the vehicle sensor and the road boundary information from the high-precision map;

a generating module, configured to generate a lane image according to the original image and the road boundary information;

an extraction module, used for extracting N ROI images from the lane image, where N is a positive integer;

The second acquisition module is configured to input each of the region of interest images into a target detector respectively to obtain a target detection result, and the target detector is obtained by training an initial target detection network constructed by sample target pairs.

In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program implement the above method.

In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program implements the foregoing method when executed by a processor.

The target detection method provided by the embodiment of the present invention generates a lane image according to the original image collected by the vehicle sensor and the road boundary information from the high-precision map, extracts the ROI image from the lane image, and further inputs the ROI image as input. It is not necessary to directly identify the road boundary and road area from the original image collected by the vehicle sensor. On the one hand, it can effectively increase the recognition efficiency of the road boundary, on the other hand, it can also effectively eliminate the The obtained road boundary is more accurate due to the interference of other information such as vehicles and road equipment; it can be seen that the embodiment of the present invention can effectively improve the extraction effect of the region of interest image.

Description of drawings

1 is a flowchart of a target detection method provided by an embodiment of the present invention;

FIG. 2 is a flowchart of image extraction of a region of interest in an embodiment of the present invention;

3 is a schematic diagram of extracting a region of interest image from a lane image in an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of a target detection apparatus provided by an embodiment of the present invention.

Detailed ways

In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, the following will be described in detail with reference to the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided merely to assist in a comprehensive understanding of embodiments of the present invention. Accordingly, it should be apparent to those skilled in the art that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

Unless otherwise defined, technical or scientific terms used in the present invention should have the ordinary meaning as understood by those of ordinary skill in the art to which the present invention belongs. The terms "first," "second," and similar terms used herein do not denote any order, quantity, or importance, but are merely used to distinguish different components. Likewise, "a" or "an" and the like do not denote a quantitative limitation, but rather denote the presence of at least one.

As shown in FIG. 1, the target detection method provided by the embodiment of the present invention includes:

Step 101, obtaining the original image collected by the vehicle sensor and the road boundary information from the high-precision map;

Step 102, generating a lane image according to the original image and the road boundary information;

Step 103, extracting N ROI images from the lane image, where N is a positive integer;

Step 104: Input each of the region-of-interest images into a target detector to obtain a target detection result, and the target detector is obtained by training an initial target detection network constructed by sample target pairs.

In this embodiment, the original image collected by the vehicle sensor may be a real-time original image of the road, and the coordinate information of each pixel in the image may be represented by an image coordinate system. The original image usually contains a lot of information, including roads, vehicles, road facilities, or objects other than roads, such as buildings and trees.

For the high-precision map, there are usually road elements such as road boundaries, lane lengths, etc., and this embodiment can use the road boundary information from the high-precision map; The road boundary information of the road where the vehicle is currently located is obtained from the precise map. The road boundary information from the high-precision map defined here may refer to the coordinate information of the left and right road boundaries in the world coordinate system.

According to the original image and the road boundary information, a lane image is generated, which can be based on a high-precision map, and the coordinate information of the road boundary is projected from the world coordinate system to the image coordinate system of the original image, so as to extract the road including the road in the original image. The lane image of the area; the road area here, can refer to the area between the two sides of the road boundary. It is easy to understand that the above-mentioned road boundary may refer to the boundary corresponding to the lane line of the current lane of the vehicle, or the boundary corresponding to the outermost lane line of the road where the vehicle is currently located, or the boundary of the entire drivable area of the road. , or the overall boundary of the road such as the road shoulder, which is not specifically limited here, and can be set as needed; and for the detection of targets in the lane image, it can be considered that the detection is mainly for targets in the road area.

Because the road boundary information of the high-precision map is relatively stable, the interference of other information such as vehicles and road equipment can be effectively eliminated. Compared with the method of real-time road boundary recognition directly from the original image collected by the vehicle sensor, the obtained road boundary is more accurate. , and the real-time performance is higher, which is more conducive to practical applications.

On the basis of obtaining the lane image, the region of interest (ROI) image can be extracted from the lane image. For the obtained ROI image, it can be input into the target detector to obtain the target detection result. The target detection result here can refer to whether there are objects such as obstacles in the road area; of course, if there are obstacles in the road area When waiting for the target, the above target detection result can also be used to identify the specific position and size of the obstacle. The specific implementation method will be introduced below, and will not be repeated here.

It is easy to understand that for the object detector, it can be a trained neural network model. Specifically, an initial target detection network can be built first. The specific network type is not limited here. It can be, for example, Convolutional Neural Networks (CNN), The Single Shot Detector (SSD), YOLO Then, the training samples can be used to train the initial target detection network, and the training samples here can be sample targets, that is, the training samples of targets such as vehicles and roadblocks; after the initial target detection network is trained, A target detector can be obtained. At this time, the ROI image is input to the target detector, and the target detector can automatically detect the target and output the target detection result.

The target detection method provided by the embodiment of the present invention generates a lane image according to the original image collected by the vehicle sensor and the road boundary information from the high-precision map, extracts the ROI image from the lane image, and further inputs the ROI image as input. It is not necessary to directly identify the road boundary and road area from the original image collected by the vehicle sensor. On the one hand, it can effectively increase the recognition efficiency of the road boundary, on the other hand, it can also effectively eliminate the The obtained road boundary is more accurate due to the interference of other information such as vehicles and road equipment. It can be seen that the embodiments of the present invention can effectively improve the extraction effect of the ROI image, and further improve the target detection accuracy for the ROI image.

Optionally, in step 103, N ROI images are extracted from the lane image, including:

extracting N region-of-interest images in the lane image along a first direction, wherein the lane image includes a road region, the first direction matches a lengthwise extension of the road region, and the N region-of-interest images The aspect ratios of the area images are all equal to the preset ratio, and each area of interest image includes the entire area where the road area is located between the two straight lines where the two first boundaries of the area of interest image are located. The first borders are two borders of the region of interest image that are oppositely arranged in the first direction.

It is easy to understand that the road area in the lane image usually has a length extension direction. In the length extension direction, the road area has a gradient process of width, which is reflected in the actual scene. Visually, the near road is wider, and the far road is narrower.

Referring to Figure 3, for the road area, it can be reflected in the form of road boundaries; for the original image that normally includes lanes, the left road boundary and the right road boundary can be identified for it, and the road area can correspond to the left road boundary and The area between the road boundaries on the right. When the vehicle travels in a straight line, or the vehicle travels approximately in a straight line, or the road is approximately straight, the road boundaries on both sides of the lane image are approximately symmetrical in the extension direction of the pixel column of the lane image. At this time, the length extension direction of the road area can be Considered to be along the extending direction of the pixel row, the first direction can be regarded as the extending direction of the pixel row correspondingly.

Referring also to FIG. 3 , in the case where the first direction is determined as the extension direction of the pixel column, the N ROI images may refer to the images corresponding to each ROI continuously arranged in the extension direction of the pixel column. There may also be a certain degree of stacking between the images corresponding to the ROI. The aspect ratios of the above N ROI images are all equal to the preset ratio, and the specific length value or width value may not be specifically limited, but it is necessary to make each ROI image include two road areas located in the ROI image. The entire area between the two straight lines where the first boundary is located; with reference to Figure 3, for any ROI image, the two first boundaries can be considered as the upper and lower boundaries of the ROI image, and the road area is located in these two first boundaries. All areas between two straight lines where a boundary is located all fall into the ROI image, which helps to ensure a more comprehensive detection of obstacles and other types of targets in the road area.

Usually, the target detector needs to scale the input image to an image of a specific size to further detect the target; the image of the specific size has a fixed aspect ratio, and in this embodiment, the length and width of each ROI image are The ratio is unified as a preset ratio. When input to the target detector, the deformation of each ROI image is roughly the same, which helps to detect the target more accurately. It is worth emphasizing that the deformation here can correspond to the degree of stretching and extrusion, or it can also mean that each ROI is not deformed. When the ratio is the same, it can be considered that each ROI is not deformed after being input into the object detector.

In this embodiment, a lane image including a road area is acquired, and N ROI images are extracted in the lane image along the first direction, that is, in a direction matching the length extension direction of the road area, and the aspect ratio of each ROI image is equal to The preset ratio includes all areas of the road area between the boundaries of each ROI image along the first direction, and each ROI image is sent to the trained target detector to obtain target detection results. In this embodiment, the aspect ratio of each ROI image input to the target detector can be guaranteed to be a fixed value. After input to the target detector, the deformation of each ROI image is approximately the same, which helps to solve the problem of ROI The image has different degrees of stretching or squeezing deformation, which leads to the problem of poor target detection effect, which effectively improves the accuracy of target detection results. At the same time, by extracting the ROI image from the lane image along the first direction, the target can be detected separately in the road area at different distances, thereby helping to achieve accurate detection of small targets at long distances.

In some application scenarios, for example, when the vehicle has a large turning angle or the lane is relatively curved, the road boundaries on both sides may extend in the same inclined direction in the lane image, or the road boundaries on both sides have more curvature; , based on conventional definitions, the above-mentioned first direction may not be the extension direction of the pixel row, but a direction with a certain inclination angle relative to the extension direction of the pixel row, or the extension direction of the spline curve.

For example, if the first direction is a direction with a certain inclination angle relative to the extension direction of the pixel row, the extension directions of the left and right borders of each ROI image are consistent with the first direction, and there will also be a certain angle relative to the extension direction of the pixel row. Inclination angle; for another example, if the first direction is the extension direction of the spline curve, the extension directions of the left and right boundaries of each ROI image may be consistent with the tangent direction of the spline curve at the corresponding position of the ROI image.

Of course, in practical applications, regardless of the vehicle operating state or lane shape, in the corresponding lane image, the road area can be considered to have an extension in the extension direction of the pixel column. Therefore, the first direction can also be fixed as the pixel column. The extension direction of ROI is convenient for ROI division and ROI image extraction.

For ease of description, unless additional emphasis is placed on the following, the target detection method provided by the embodiment of the present invention will be introduced mainly by taking an example of a lane image corresponding to a vehicle traveling in a straight line on a straight lane.

In an example, each ROI is a rectangular frame. Referring to FIG. 3 , the two endpoints of the lower straight side of each rectangular frame are located on the road boundaries on both sides of the road area, respectively. In this way, the aspect ratio of the ROI image is In the case of being fixed, each ROI can frame the entire area of the road area between the upper and lower borders of the ROI through the minimum area, which helps to reduce the detection workload of the target detector and improve the detection efficiency.

Similarly, for the vehicle image obtained when the vehicle has a large turning angle or the lane is relatively curved, on the premise of ensuring the aspect ratio of the ROI image, the minimum area ROI can be used to frame the road area along the ROI. The entire area between the two borders in one direction; the process of determining the specific size of the minimum area ROI here can be adjusted according to actual needs, which will not be repeated here.

Of course, in practical applications, for the lane image shown in FIG. 3, it is not limited that the two end points of the lower straight side of each rectangular frame are located on the road boundaries on both sides of the road area, for example, the ROI image The length may be greater than the maximum width of the road area in the ROI image, and the two endpoints of the lower straight side of the corresponding rectangular frame of the ROI image are located outside the road boundaries on both sides of the road area, respectively.

The following describes a specific implementation of the above-mentioned acquisition of a lane image. In this specific implementation, the above-mentioned vehicle-mounted sensor may be a camera specifically:

Step 1: Obtain the world coordinates of the point set of the left and right road boundaries in the high-precision map, as well as the internal and external parameters of the camera;

Among them, the internal parameters usually include 1/dx, 1/dy, u ₀ , v ₀ , f and other parameters, and the external parameters usually include rotation parameters and translation parameters, which are usually represented by rotation matrices and translation matrices respectively. The specific meaning of the algebraic formula of the parameters will be explained one by one in conjunction with the specific calculation process below;

Step 2: Convert the point set coordinates of the road boundary from the world coordinate system to the camera coordinate system, as shown in the following formula:

Where: (X _c , Y _c , Z _c ) are the coordinates in the camera coordinate system, (X _w , Y _w , Z _w ) are the coordinates in the world coordinate system, R is a 3×3 rotation matrix, and t is 1 ×3 translation matrix.

Step 3: Convert the point set coordinates of the road boundary from the camera coordinate system to the image plane physical coordinate system (image plane real physical size coordinate system), as shown in the following formula:

where f is the focal length of the camera, (X _c , Y _c , Z _c ) are the coordinates in the camera coordinate system, and (x, y) are the coordinates in the image plane physical coordinate system;

Step 4: Convert the point set coordinates of the road boundary from the image plane physical coordinate system to the image pixel coordinate system, as shown in the following formula:

Among them: dx, dy respectively represent the physical size of a single pixel on the photosensitive chip on the x-axis and y-axis (unit is usually mm/pixel), the two are usually used to connect the image pixel coordinate system and the image plane physical coordinate system, (u , v) are the coordinates in the image pixel coordinate system, and (u ₀ , v ₀ ) are the coordinates of the origin of the physical coordinate system of the image plane in the image pixel coordinate system.

Optionally, in the step 102, N ROI images are extracted from the lane image along the first direction, including:

determining a first target road width and a second target road width, wherein the first target road width is greater than the second target road width;

Obtain the road width at each position of the road area in the first direction, take the position where the road width in the road area matches the first target road width as the starting position, and move along the first direction Extract ROI images sequentially until the maximum value of the road width in the n+1 th ROI image of the road area is smaller than the second target road width for the first time. The region of interest images are used as the N region of interest images, where n is a positive integer.

In this embodiment, the road area to be extracted from the ROI image and the extraction sequence of the N ROI images are limited.

Below in conjunction with Fig. 2 and Fig. 3, a feasible implementation manner of this embodiment is introduced:

This implementation is based on the premise that the road boundary has been converted to the physical coordinate system of the image plane, and specifically includes:

Step 201, obtaining the pixel spacing W _start of the starting scan line, the pixel spacing W _end of the ending scan line, and the ROI aspect ratio K;

Here, W _start and W _end respectively correspond to the above-mentioned first target road width and second target road width, and can be represented by the number of pixels between the boundaries of the two roads.

The image directly used for target detection in the target detector, that is, the length, width and aspect ratio of the image after the target detector scales the input image is usually fixed; The above-mentioned W _start , W _end and K can be determined based on these fixed parameters; for example, the length of the above-mentioned image directly used for target detection is W, and the aspect ratio is K ₀ ; then the coefficient can be set f ₁ and f ₂ , so that W _start =W/f ₁ , W _end =W/f ₂ , and K=K ₀ ; the values of f ₁ and f ₂ can be set as required, for example, set f ₁ = 2, f ₂ =8; in practical applications, f ₁ and f ₂ can be adjusted according to the ability of the target detector to identify targets. ₂ can be set to be larger.

Step 202, matching the left and right coordinates (u ^r _start , v _start ) and (u ^l _start , v _start ) of the road area according to the starting distance W _start and the point set of the road boundary;

The lane image is searched from bottom to top scan line by scan line, and the number of pixels between the projection points of the left and right road boundaries on the scan line is calculated. For the convenience of description, the number of pixels here is defined as the first number of pixels; When the number of a pixel matches W _start , for example, in the process of scanning line movement, when the first pixel number is equal to W _start , or when the first pixel number is smaller than W _start for the first time, the corresponding scan line is determined as the starting point. The initial scan line Scanline _start , the coordinates of the intersection of the initial scan line Scanline _start and the left and right road boundaries are the above (u ^r _start , v _start ), (u ^l _start , v _start ), where the superscript r can be understood as The abbreviation of the right (right), the superscript l can be understood as the abbreviation of the left (left).

Figure 3 also shows the vanishing point of the end scan line Scanline _end and the lane, which can only provide a reference for the position of the ROI; the end scan line Scanline _end corresponds to the start scan line Scanline _start , which can correspond to the first number of pixels Equal to W _start , or the scan line when it is smaller than W _start for the first time; and the vanishing point of the lane corresponds to the far end intersection of the left and right road boundaries in the lane image, which may or may not exist in the actual lane image.

Step 203, calculate the height d _i =K×W _i of the i-th ROI;

It is easy to understand that the ROI image corresponds to the image of the vehicle image in the area where the ROI is located. For the convenience of explanation, the following description is mainly based on the ROI; in addition, the height of the ROI may refer to the number of pixels on the v-axis.

Since the aspect ratio of all ROIs is a fixed value K, the height d _i of the i-th ROI can be calculated according to the determined length Wi of the _i -th ROI (which can be considered as the number of pixels on the u-axis);

The length of the first ROI is W _start above, and the method for determining the length of the subsequent ROI will be further described below.

Step 204, store the size of the _i - _th ROI, coordinate information Wi, di, vi ^, _{uri , uli} ^;

Referring to FIG. 3, each ROI can be a rectangular frame, and for the i-th ROI (referred to as ROI _i in the figure), the two endpoints of the edge line on the lower side can fall on the left and right road boundaries respectively, and the two endpoints The coordinates of , can be respectively recorded as ( _u ^ri ,vi ) and ( _{u li} ,vi ₎ ; in the case that _{Wi and d i} ^have been obtained in the previous step _, the above-mentioned size and coordinate information can be stored.

Step 205, calculate the i+1th ROI ordinate v _{i +1} =vi -d _i +ε;

Here, ε can be a preset value greater than or equal to 0. It is easy to understand that when ε=0, the u-axis coordinate of the upper boundary of the i-th ROI is the lower boundary of the i+1-th ROI. The u-axis coordinates of , so that the continuity of the finally obtained N ROIs in the u-axis direction can be preliminarily guaranteed.

When ε>0, there may be an overlapping area between two adjacent ROIs, so compared with the case of ε=0, the target detection effect of the connection area between the two adjacent ROIs can be effectively improved.

Step 206, according to the ordinate v _i+1 of the i+1 ROI and the point set of the road boundary, determine the left and right abscissas u ^r _i+1 and u ^l _i+1 of the i+1 ROI;

When the ordinate v _i+1 of the i+1th ROI is determined, the coordinates of the points of the left and right road boundaries at the ordinate v _i+1 can be obtained respectively, that is, u ^r _i+1 and u ^l _{i+ 1} .

Step 207: Calculate the length of the i+1 ROI W _i+1 =ur _i+1 -u ^{l i} ₊₁ ;

Step 208, judge whether W _i+1 is less than W _end ;

If W _i+1 <W _end , it means that the image of the road area located between the boundary line corresponding to W _start and the boundary line corresponding to W _end has been comprehensively extracted, so the images corresponding to the previous i ROIs can be used as An image that needs to be input to the object detector for object detection.

If W _i+1 ≧W _end , return to step 203 , that is, continue to extract the ROI.

Combining the above-mentioned feasible implementations, it can be seen that the target detection method provided by the embodiment of the present invention can make the extracted ROI image include all areas where the road area is within a specific ordinate range, so as to ensure the comprehensiveness of the target detection result; The specific extraction method of the image is set, and the length of the ROI can be adjusted according to the width of the road area. In this way, on the one hand, when the ROI is subsequently input to the target detector for scaling, compared with the short-range large target, the long-distance small target can be detected. Perform relative amplification to improve the detection effect of long-distance small targets; on the other hand, it can effectively narrow the detection range, improve the target detection speed, and meet the real-time detection requirements.

Of course, as mentioned above, in the case that the road boundaries on both sides of the road area extend along the same inclined direction, or there are many bends, steps similar to the above implementation methods can also be used to extract the ROI, just to ensure that the The comprehensive extraction of the road area between the boundary line corresponding to W _start and the boundary line corresponding to W _end may require adjustment of the length of the ROI. For example, the road area can be located between the two straight lines corresponding to the two long sides of each ROI. The part between the two lines, the vertical projection on these two lines, is located between the two end points of the long side of the ROI; at the same time, the aspect ratio of the ROI is still guaranteed to be fixed.

In general, the target detection method provided by the embodiment of the present invention can effectively reduce the detection effect of large and small targets while ensuring the detection effect of large and small targets, compared with the method of using 4k or even 8k and above high-resolution images for target detection in the prior art. Computational amount of object detection.

In a preferred embodiment, when the N is greater than 1, any two adjacent ROI images in the N ROI images have a preset overlapping distance in the first direction.

As mentioned above, when the value of ε is greater than 0, in the process of determining the ROI, any two ROIs have a certain overlapping area; on this basis, if the number of ROI images finally extracted is greater than 1, it can make There is a preset overlapping distance between any two ROI images, which helps to accurately detect the target in the connection area of the two ROI images.

Of course, in some feasible implementation manners, the preset overlap distance here may not be a fixed value, but may also be determined according to the length or width of each ROI image, for example, multiplying the length of the i-th ROI image by A certain preset value is used as the overlap distance between the i-th ROI image and the i+1-th ROI image.

Optionally, the target detector includes a scaling module and a target detection module;

In the step 104, each image of the region of interest is input into the target detector respectively to obtain a target detection result, including:

The first region of interest image is input into the zoom module to obtain a zoomed image, the aspect ratio of the zoomed image is equal to the aspect ratio of the first region of interest image, and the length of the zoomed image is equal to that of the first region of interest image. The widths are all preset fixed values, and the first ROI image is any ROI image in the N ROI images;

The zoomed image is input to the target detection module to obtain the target detection result.

In this implementation, the target detector includes a scaling module and a target detection module. The main function of the scaling module is to scale the input image, such as the above-mentioned ROI image, into an image of a specific size, so that the target detection module can detect the target.

Further, by limiting the aspect ratio of the ROI image, the aspect ratio between the ROI image and the zoomed image is made equal, so that the ROI image can be effectively prevented from being stretched or squeezed during zooming, and the accuracy of the target detection result can be improved. accuracy.

Optionally, in the case that the target detection result includes M target regions, after the target detection result is obtained by inputting each of the region-of-interest images into the target detector, the method further includes:

Each of the M target regions is projected onto the lane image respectively.

As mentioned above, the target detector can identify objects such as obstacles from the input ROI image. When there is a target in the ROI image, the area where the target is usually can be marked, for example, a detection frame is used to frame the area where the target is located. ; Therefore, the M target regions here can be considered as some detection frames.

The projection of the target area into the lane image can be considered as projecting the detection frame into the lane image, and reflected in the actual application, it can be considered that the specific location of the object such as the obstacle is framed in the lane image. It can be seen that, in this embodiment, when the target detection results include target areas, projecting these target areas into the lane image can reflect the specific position of the target, thereby helping to provide a basis for the vehicle to avoid the target in the future. .

For example, in a feasible implementation manner, the coordinates of the center of the ROI image in the above-mentioned image pixel coordinate system are (u _origin , v _origin ), which can be acquired during the process of determining the ROI, and the ROI image is input to the target detector. The scaling factor scale after the middle is W/W _i , W is the length of the zoomed image _, Wi is the length of the ROI image, the coordinates of the center of the detection frame in the ROI image are (u _center , v _center ), and the The length and width (or the height along the v-axis) are w and h respectively, then the detection frame can be represented by the following parameter set (u _center , v _center , w, h); similarly, the detection frame in the lane image The projection can be represented by a parameter set (u', v', w', h'), and each parameter in the parameter set corresponds to each parameter in the parameter set of the detection frame, where (u', v') represents the coordinates of the center of the projection of the detection frame in the physical coordinate system of the image plane of the lane image, and w' and h' are the length and width of the projection of the detection frame, respectively.

Then the projection formula for projecting a detection frame in a certain ROI image to the lane image can be:

u'=u _center /scale+u _origin /2

v'=v _center /scale+v _origin /2

w'=w/scale

h'=h/scale

Optionally, when the M is greater than 1, after projecting each of the M target areas into the lane image, the method further includes:

Obtain the intersection area between the projection of the first target area in the lane image and the projection of the second target area in the lane image, wherein the first target area and the second target area are the any two target areas in the M target areas;

In the case where the intersection area is greater than an area threshold, the projection of the first target area in the lane image is merged with the projection of the second target area in the lane image.

For convenience of description, the projection of the first target area in the lane image is referred to as the first projection, and the projection of the second target area in the lane image is referred to as the second projection. Generally speaking, for the first projection and the second projection whose intersection area is greater than the area threshold, it can be considered that they correspond to the same target. Combining the first projection and the second projection can effectively avoid the repeated marking of the same target and improve the speed of the lane. Object recognition effects in images.

In this embodiment, the area threshold may be a fixed value or a relative value. For example, the area threshold may be 1/3 of the area of the projection with the smaller area in the first projection and the second projection. Of course, this The specific proportion of the area can be set according to actual needs; further, the way of determining the area threshold can also be set according to actual needs.

As shown in FIG. 4 , an embodiment of the present invention further provides a target detection device, including:

The first acquisition module 401 is used to acquire the original image collected by the vehicle sensor and the road boundary information from the high-precision map;

a generating module 402, configured to generate a lane image according to the original image and the road boundary information;

Extraction module 403, configured to extract N ROI images from the lane image, where N is a positive integer;

The second acquisition module 404 is configured to input each image of the region of interest into a target detector respectively to obtain a target detection result, and the target detector is obtained by training an initial target detection network constructed by sample target pairs.

Optionally, the extraction module 403 is specifically used for:

extracting N region-of-interest images in the lane image along a first direction, wherein the lane image includes a road region, the first direction matches a lengthwise extension of the road region, and the N region-of-interest images The aspect ratios of the area images are all equal to the preset ratio, and each area of interest image includes the entire area where the road area is located between the two straight lines where the two first boundaries of the area of interest image are located. The first borders are two borders of the region of interest image that are oppositely arranged in the first direction.

Optionally, the extraction module 403 includes:

a determining unit, configured to determine a first target road width and a second target road width, wherein the first target road width is greater than the second target road width;

an acquisition and extraction unit, configured to acquire the road width at each position of the road area in the first direction, taking the position where the road width in the road area matches the first target road width as a starting position, The region of interest images are sequentially extracted along the first direction, until the maximum value of the road width in the n+1 th region of interest image of the road region is smaller than the second target road width for the first time, and The first n images of the region of interest are used as the N images of the region of interest, where n is a positive integer.

Optionally, in the case where the N is greater than 1, any two adjacent ROI images in the N ROI images have a preset overlapping distance in the first direction.

Optionally, the target detector includes a scaling module and a target detection module;

The second obtaining module 404 includes:

a first acquiring unit, configured to input a first region of interest image into the zoom module to obtain a zoomed image, the aspect ratio of the zoomed image is equal to the aspect ratio of the first region of interest image, and The length and width of the zoomed image are both preset fixed values, and the first ROI image is any ROI image in the N ROI images;

The second acquiring unit is configured to input the zoomed image to the target detection module to obtain the target detection result.

Optionally, when the target detection result includes M target regions, the target detection device further includes:

The projection module is used for projecting each target area in the M target areas to the lane image respectively.

Optionally, when the M is greater than 1, the target detection device includes:

A third acquiring module, configured to acquire the intersection area between the projection of the first target area in the lane image and the projection of the second target area in the lane image, wherein the first target area and the The second target area is any two target areas in the M target areas;

A merging module, configured to merge the projection of the first target area in the lane image with the projection of the second target area in the lane image when the intersection area is greater than an area threshold.

It should be noted that the target detection apparatus is an electronic device corresponding to the above target detection method, and all implementations in the above method embodiments are applicable to the embodiments of the apparatus, and the same technical effects can also be achieved.

Optionally, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program When the above-mentioned target detection method is implemented.

Optionally, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the foregoing target detection method is implemented.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above-mentioned embodiments, those of ordinary skill in the art should understand that: it can still be used for the above-mentioned implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the application, and should be included in the within the scope of protection of this application.

Claims (10)

1. A method of object detection, comprising:

acquiring an original image acquired by a vehicle sensor and road boundary information from a high-precision map;

generating a lane image according to the original image and the road boundary information;

extracting N interested area images from the lane image, wherein N is a positive integer;

and respectively inputting each region-of-interest image into a target detector to obtain a target detection result, wherein the target detector is obtained by training the constructed initial target detection network through a sample target.

2. The method of claim 1, wherein extracting N region-of-interest images in the lane image comprises:

extracting N interested area images in the lane image along a first direction, wherein the lane image comprises a road area, the first direction is matched with the length extending direction of the road area, the length-width ratios of the N interested area images are all equal to a preset ratio, each interested area image comprises all areas of the road area between two straight lines where two first boundaries of the interested area image are located, and the two first boundaries are two boundaries of the interested area image which are oppositely arranged in the first direction.

3. The method of claim 2, wherein said extracting N region-of-interest images in the lane image along the first direction comprises:

determining a first target road width and a second target road width, wherein the first target road width is larger than the second target road width;

the method comprises the steps of obtaining the road width of a road area at each position in the first direction, taking the position, matched with a first target road width, of the road width in the road area as a starting position, sequentially extracting interested area images along the first direction until the maximum value of the road width of the road area in the (N + 1) th interested area image is smaller than the second target road width for the first time, and taking the first N interested area images as the N interested area images, wherein N is a positive integer.

4. The method according to claim 3, wherein in case that N is larger than 1, there is a preset overlap distance between any two adjacent region-of-interest images in the N region-of-interest images in the first direction.

5. The method of claim 1, wherein the object detector comprises a scaling module and an object detection module;

the step of inputting each region of interest image into a target detector to obtain a target detection result includes:

inputting a first region-of-interest image into the scaling module to obtain a scaled image, wherein the aspect ratio of the scaled image is equal to that of the first region-of-interest image, the length and the width of the scaled image are both preset fixed values, and the first region-of-interest image is any one of the N region-of-interest images;

and inputting the zoomed image to the target detection module to obtain the target detection result.

6. The method according to claim 1, wherein in a case that the target detection result includes M target regions, the method further includes, after inputting each of the region-of-interest images into a target detector respectively and obtaining a target detection result:

and respectively projecting each target area in the M target areas into the lane image.

7. The method of claim 6, wherein after said projecting each of said M target regions into said lane image if M is greater than 1, said method further comprises:

acquiring an intersection area between a projection of a first target area in the lane image and a projection of a second target area in the lane image, wherein the first target area and the second target area are any two target areas in the M target areas;

merging a projection of the first target region in the lane image with a projection of the second target region in the lane image if the intersection area is greater than an area threshold.

8. An object detection device, comprising:

the first acquisition module is used for acquiring an original image acquired by a vehicle sensor and road boundary information from a high-precision map;

the generating module is used for generating a lane image according to the original image and the road boundary information;

the extraction module is used for extracting N interested area images from the lane image, wherein N is a positive integer;

and the second acquisition module is used for respectively inputting each region-of-interest image into a target detector to obtain a target detection result, wherein the target detector is obtained by training the established initial target detection network through a sample target.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.

Images