CN112488118B - A target detection method and related device - Google Patents

Abstract

The application provides a target detection method and a related device, and relates to the technical field of image recognition, wherein the target detection method comprises the following steps: extracting feature information of the original image to obtain feature information of the original image; based on the original image characteristic information, generating a corresponding density map and a segmentation map; determining a target area in the original image based on the segmentation map, and generating a target area density map based on the target area and the density map; based on the target area density map, cutting out a block image related to the target area density map from the original image; and respectively carrying out target detection on each segmented image so as to output a target detection result corresponding to each segmented image. Based on the technical scheme of the application, the accuracy of target detection can be effectively improved.

Description

A target detection method and related device

technical field

The present application relates to the technical field of image recognition, in particular to a target detection method and a related device.

Background technique

With the development of the times, people have higher and higher requirements for the accuracy of target detection. How to improve the accuracy of target detection has become the focus of research in this field.

In the prior art, there is a large amount of noise in the original image, and the target detection directly performed on the original image is easily disturbed by the noise, thereby ignoring useful information and reducing the accuracy of target detection.

Contents of the invention

The present application provides an object detection method and a related device, which are beneficial to reducing waste of resources in the object detection process.

In order to achieve the above technical effects, the first aspect of the present application provides a target detection method, including:

Extract feature information from the original image to obtain feature information of the original image;

Generate the corresponding density map and segmentation map based on the above original image feature information;

Determine the target area in the original image based on the segmentation map, and generate a target area density map based on the target area and the density map, wherein the target area is a point with a brightness value greater than a first preset value in the segmentation map In the formed area, the density map of the above-mentioned target area is a density map that only includes the density information of the above-mentioned target area;

Based on the above-mentioned target area density map, cutting out block images related to the target area density map from the above-mentioned original image;

Target detection is performed on each of the above-mentioned block images, so as to output a target detection result corresponding to each of the above-mentioned block images.

Based on the first aspect of the present application, in a first possible implementation manner, based on the above-mentioned target area density map, cutting out block images related to the target area density map from the above-mentioned original image includes:

Based on the above-mentioned target area density map, generate a minimum bounding box for enclosing the density-connected area in the above-mentioned target area density map, wherein the above-mentioned density-connected area is an area composed of connected points whose density value is greater than a second preset value;

Based on each of the above-mentioned minimum bounding boxes, the above-mentioned target area density map, and the above-mentioned original image, the predicted target area ratio corresponding to each of the above-mentioned minimum bounding boxes is obtained. In the mapped area above, the average value of the ratio of the area of each predicted target to the total area of the above mapped area;

Based on the size relationship between each of the above-mentioned predicted target area ratios and the preset standard ratio, the scale of each of the above-mentioned minimum bounding boxes is adjusted, so that the predicted target area ratio of each adjusted minimum bounding box is smaller than that before adjustment. Close to the above standard ratio;

Based on the position of each of the above-mentioned adjusted minimum bounding boxes mapped on the above-mentioned original image, the original image is cropped to obtain one or more block images corresponding to each of the above-mentioned adjusted minimum bounding boxes.

Based on the first possible implementation of the first aspect of the present application, in the second possible implementation, based on the size relationship between each of the above-mentioned predicted target area ratios and the preset standard ratio, each of the above-mentioned minimum enclosing The scale of the frame is adjusted so that the proportion of the predicted target area of each adjusted minimum bounding box is closer to the above-mentioned standard proportion than before adjustment, including:

Calculate the ratio of the predicted target area ratio of each of the above minimum bounding boxes to the above standard ratio;

If there is a first-type bounding box, increase the scale of the above-mentioned first-type bounding box, so that the predicted target area ratio of each adjusted first-type bounding box is closer to the above-mentioned standard ratio than before adjustment, where , the above-mentioned first type of bounding box is the smallest bounding box with the above-mentioned ratio greater than the third preset value among the above-mentioned minimum bounding boxes;

If there is a second-type bounding box, reduce the size of the second-type bounding box, and/or, divide the second-type bounding box into two or more minimum bounding boxes, so that each adjusted second-type bounding box The proportion of the predicted target area of the frame is closer to the above-mentioned standard proportion than before adjustment, wherein the above-mentioned second type of bounding box is the smallest bounding box with the above-mentioned ratio smaller than the fourth preset value in each of the above-mentioned smallest bounding boxes;

Wherein, the above-mentioned third preset value is not less than the above-mentioned fourth preset value.

Based on the first or second possible implementation of the first aspect of the present application, in the third possible implementation, based on the above-mentioned target area density map, generate the density used to enclose the above-mentioned target area density map After the minimum bounding box of the connected area, the above target detection method also includes:

Merging two minimum bounding boxes whose distance is less than a preset distance among the above minimum bounding boxes to form a minimum bounding box for enclosing the density-connected area of the two minimum bounding boxes whose distance is less than a preset distance.

Based on the first aspect of the present application or the first or second possible implementation of the first aspect of the present application, in the fourth possible implementation, after performing target detection on each of the above-mentioned block images, the above-mentioned Object detection methods also include:

Based on the non-maximum value suppression algorithm, the above-mentioned target detection results are combined to obtain the target detection result of the above-mentioned original image.

Based on the first aspect of the present application or the first or second possible implementation of the first aspect of the present application, in the fifth possible implementation, the feature information of the original image is extracted to obtain the feature information of the original image, Based on the above original image feature information, generating the corresponding density map and segmentation map includes:

Based on the first preset model, feature information is extracted from the original image, and a corresponding density map and segmentation map are generated.

Based on the first or second possible implementation of the first aspect of the present application, in the sixth possible implementation, each of the above-mentioned The proportion of predicted target area corresponding to the smallest bounding box includes:

Based on the above-mentioned target area density map and each of the above-mentioned minimum enclosing boxes, determine the area of each of the above-mentioned minimum enclosing boxes, the sum of the density values in each of the above-mentioned minimum enclosing boxes, and the area of the density connected region in each of the above-mentioned minimum enclosing boxes;

Based on the second preset model, the area of each of the aforementioned smallest bounding boxes, the sum of the density values in each of the aforementioned smallest bounding boxes, the area of the density-connected regions in each of the aforementioned smallest bounding boxes, and the total area of the aforementioned original image, each of the aforementioned The proportion of the predicted target area corresponding to the smallest bounding box.

Based on the first aspect of the present application or the first or second possible implementation of the first aspect of the present application, in a seventh possible implementation, the above-mentioned target detection for each of the above-mentioned block images includes:

Based on the third preset model, target detection is performed on each of the aforementioned block images.

The second aspect of the present application provides a target detection device, including:

The extraction unit is used to extract feature information from the original image to obtain feature information of the original image;

The first generation unit is used to generate a corresponding density map and segmentation map based on the above-mentioned original image feature information;

The second generation unit is configured to determine the target area in the original image based on the segmentation map, and generate a density map of the target area based on the target area and the density map, wherein the target area has a brightness value greater than The area formed by the points of the first preset value, the above-mentioned target area density map is a density map that only includes the density information of the above-mentioned target area;

A cropping unit, configured to cut out block images related to the target area density map from the above-mentioned original image based on the above-mentioned target area density map;

The target detection unit is configured to perform target detection on each of the above-mentioned block images, so as to output a target detection result corresponding to each of the above-mentioned block images.

The third aspect of the present application provides a target detection device, including a memory and a processor, the memory stores a computer program, and the processor implements the first aspect or any possible implementation of the first aspect when executing the computer program The steps of the target detection method mentioned in the way.

It can be seen from the above that the technical solution of the present application includes extracting the feature information of the original image to obtain the feature information of the original image; generating the corresponding density map and segmentation map based on the feature information of the original image; determining the target area in the original image based on the segmentation map , and based on the target area and the density map, the target area density map is generated; based on the target area density map, the block images related to the target area density map are cut out from the original image; the target detection is performed on each block image to output The target detection results corresponding to each block image. Based on the technical solution of this application, the segmentation map and density map of the original image can be extracted first, and the density map can be further deleted based on the segmentation map to improve the pertinence of the target area density map to the target in the original image, and then based on the target The area density map cuts the original image, and finally performs target detection on each cropped block image to obtain each target detection result. Since the target detection is only performed on block images related to the target area density map, it can effectively exclude the original image. Noise interference, improve the accuracy of target detection.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the accompanying drawings that need to be used in the descriptions of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are only for the present application For some embodiments, those skilled in the art can also obtain other drawings according to these drawings without paying creative efforts.

FIG. 1 is a schematic flow chart of an embodiment of a target detection method provided by the present application;

FIG. 2 is a schematic structural diagram of an embodiment of a target detection device provided by the present application;

Fig. 3 is a schematic structural diagram of another embodiment of the target detection device provided by the present application.

Detailed ways

In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and/or components, but does not exclude one or more other features. , whole, step, operation, element, component and/or the presence or addition of a collection thereof.

It should also be understood that the terminology used in the specification of the present application is for the purpose of describing specific embodiments only and is not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural referents unless the context clearly dictates otherwise.

The technical solutions in the embodiments of the present application are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

In the following description, a lot of specific details are set forth in order to fully understand the application, but the application can also be implemented in other ways different from those described here, and those skilled in the art can do it without violating the content of the application. By analogy, the present application is therefore not limited by the specific embodiments disclosed below.

Embodiment one

This application provides a target detection, as shown in Figure 1, including:

Step 101, extracting feature information from the original image to obtain feature information of the original image;

In the embodiment of the present application, the original image that needs to be detected can be obtained first, and then the feature information of the original image is extracted to obtain the feature information of the original image, that is, the feature information of the original image, for subsequent use.

Step 102, generating a corresponding density map and segmentation map based on the above-mentioned original image feature information;

In the embodiment of the present application, the density map is that the density value of each point in the density map is used to represent the target density value of the corresponding area in the original image; the segmentation map is obtained based on the preset image segmentation method and original image feature information The image segmentation method can be any one of target and background binary classification method, threshold segmentation method, maximum inter-class variance method and other image segmentation methods.

Specifically, the target and background binary classification method is used to segment the original image to obtain the above segmentation map. The specific process is as follows:

Based on the trained neural network and the original image, the corresponding target probability map and the corresponding background probability map are generated, wherein the above target probability map and the above background probability map have the same resolution, and the above target probability map and the above background probability map have the same The value of each point represents the probability value of the point being the target or the background;

Compare the size of the points mapped to each other on the above target probability map and the above background probability map one by one, the point whose probability of the target is greater than the probability of the background is determined as the target point, and the probability of the target is not greater than that of the background The point of probability is determined as the background point;

Based on the above-mentioned target probability map and the above-mentioned background probability map, the above-mentioned segmentation map is obtained by assigning the background point to 0 and the target point to a preset value.

Optionally, the feature information extraction of the original image is carried out above to obtain the feature information of the original image, and based on the feature information of the original image, generating a corresponding density map and a segmentation map includes:

Based on the first preset model, feature information is extracted from the original image, and a corresponding density map and segmentation map are generated.

Specifically, the above-mentioned first preset model may be a first preset neural network, and when training the above-mentioned first preset neural network, the training samples are more than one diversified sample graph in the field, corresponding to each of the above-mentioned sample graphs. The corresponding training labels include: coarse-grained density map, segmentation map of the target obtained based on the target and background binary classification method;

Wherein, the value of each point of the above-mentioned coarse-grained density map can be used to represent the target density value of a region in the above-mentioned original image;

The above-mentioned generation method of the coarse-grained density map used as the training label can be as follows:

In the first step, the sample image is uniformly divided into blocks, and a corresponding density frame image is generated, and each point of the above density frame image corresponds to the density value of each block in the above sample image;

The second step is to determine the minimum bounding box of each target in the sample map, and assign the points on the above-mentioned density frame diagram corresponding to each minimum bounding box to be 1, and assign the points corresponding to each minimum bounding box in the above-mentioned density frame map The points other than the points on the above-mentioned density frame map of are assigned a value of 0 to obtain a density transition map;

The third step is to normalize the total values of the points corresponding to the targets in the above-mentioned density transition map, so as to finally obtain the above-mentioned coarse-grained density map.

It should be noted that, using the above-mentioned coarse-grained density map to train the above-mentioned first preset neural network can make the density map obtained by the above-mentioned first preset neural network have good clustering characteristics, so as to highlight the density information of each target. , In addition, the normalization process can also make the targets of different sizes have the same total density value, preventing the loss of small targets.

Step 103: Determine the target area in the original image based on the segmentation map, and generate a target area density map based on the target area and the density map;

Wherein, the above-mentioned target area is an area formed by points whose luminance values are greater than a first preset value in the above-mentioned segmentation map, and the above-mentioned target area density map is a density map that only includes density information of the above-mentioned target area;

In the embodiment of the present application, the above-mentioned target area can be determined based on the above-mentioned segmentation map, that is, the area where the target exists in the original image, and then the density map is processed to clear the density values of the areas other than the corresponding target area in the density map. Zero, resulting in the above target region density values.

Optionally, determining the target area in the original image based on the above-mentioned segmentation map, and generating the target area density map based on the above-mentioned target area and the above-mentioned density map include:

updating the luminance values of points whose luminance values are greater than the first preset value in the above-mentioned segmented map to 1, and updating the luminance values of points whose luminance values are not greater than the first preset value in the above-mentioned segmented map to 0;

The brightness value of each point in the updated segmentation map is multiplied by the density value of each point in the above density map in a one-to-one correspondence to obtain the density map of the target area.

Specifically, the above-mentioned first preset value may be any number among 0, 1 and other positive integers, which is not limited here.

Step 104, based on the above-mentioned target area density map, cut out block images related to the target area density map from the above-mentioned original image;

In the embodiment of the present application, the position of the target to be detected in the original image can be roughly determined based on the density value of each point in the target area density map, and then the original image can be tailored based on the position of the target to be detected , to obtain a block image related to the density map of the target region.

Optionally, based on the above-mentioned target area density map, cutting out block images related to the target area density map from the above-mentioned original image includes:

Based on the above-mentioned target area density map, generate a minimum bounding box for enclosing the density-connected area in the above-mentioned target area density map, wherein the above-mentioned density-connected area is an area composed of connected points whose density value is greater than a second preset value;

Based on each of the above-mentioned minimum bounding boxes, the above-mentioned target area density map, and the above-mentioned original image, the predicted target area ratio corresponding to each of the above-mentioned minimum bounding boxes is obtained. In the mapped area above, the average value of the ratio of the area of each predicted target to the total area of the above mapped area;

Based on the size relationship between each of the above-mentioned predicted target area ratios and the preset standard ratio, the scale of each of the above-mentioned minimum bounding boxes is adjusted, so that the predicted target area ratio of each adjusted minimum bounding box is smaller than that before adjustment. Close to the above standard ratio;

Based on the position of each of the above-mentioned adjusted minimum bounding boxes mapped on the above-mentioned original image, the original image is cropped to obtain one or more block images corresponding to each of the above-mentioned adjusted minimum bounding boxes.

Specifically, the above-mentioned second preset value may be any number among 0, 1 and other positive integers, which is not limited here.

It should be noted that if the predicted target area ratios of each block image are relatively close, the effect of subsequent target detection on each block image will be better, which can improve the detection of targets with smaller areas in the original image. Accuracy, where the standard ratio can be determined according to actual needs.

Further, based on the size relationship between each of the above-mentioned predicted target area ratios and the preset standard ratio, adjust the scale of each of the above-mentioned minimum bounding boxes, so that the predicted target area ratios of the adjusted minimum bounding boxes are compared with each other. Proportions closer to the above standard before adjustment include:

Calculate the ratio of the predicted target area ratio of each of the above minimum bounding boxes to the above standard ratio;

If there is a first-type bounding box, increase the scale of the above-mentioned first-type bounding box, so that the predicted target area ratio of each adjusted first-type bounding box is closer to the above-mentioned standard ratio than before adjustment, where , the above-mentioned first type of bounding box is the smallest bounding box with the above-mentioned ratio greater than the third preset value among the above-mentioned minimum bounding boxes;

If there is a second-type bounding box, reduce the size of the second-type bounding box, and/or, divide the second-type bounding box into two or more minimum bounding boxes, so that each adjusted second-type bounding box The proportion of the predicted target area of the frame is closer to the above-mentioned standard proportion than before adjustment, wherein the above-mentioned second type of bounding box is the smallest bounding box with the above-mentioned ratio smaller than the fourth preset value in each of the above-mentioned smallest bounding boxes;

Wherein, the above-mentioned third preset value is not less than the above-mentioned fourth preset value.

Specifically, the above-mentioned third preset value may be equal to the above-mentioned fourth preset value, or may be greater than the above-mentioned fourth preset value.

Furthermore, increasing the scale of the first type of bounding box includes: increasing the length and/or width of the first type of bounding box, wherein the first type of bounding box is a rectangular box;

The reducing the size of the second-type bounding frame includes: reducing the length and/or width of the second-type bounding frame, wherein the second-type bounding frame is a rectangular frame.

The above-mentioned division of the second type of bounding box into more than two minimum bounding boxes includes:

Perform a binary operation on the length of the above-mentioned second type of bounding box to generate two new minimum bounding boxes;

After the two new minimum bounding boxes are generated, the lengths of the sides of the two new minimum bounding boxes subjected to the bisection operation are adjusted so that the two new minimum bounding boxes partially overlap.

Further, after generating the minimum bounding box for enclosing the density-connected regions in the above-mentioned target area density map based on the above-mentioned target area density map, the above-mentioned target detection method further includes:

Merging the two minimum bounding boxes whose distance is less than a preset distance among each of the above minimum bounding boxes, so as to merge into a minimum bounding box for enclosing the density-connected area of the two minimum bounding boxes whose distance is less than a preset distance, In order to prevent too many minimum bounding boxes with a small area, the efficiency of target detection is improved.

Further, based on the above-mentioned minimum bounding boxes, the above-mentioned target area density map and the above-mentioned original image, the predicted target area ratio corresponding to each of the above-mentioned minimum bounding boxes includes:

Based on the above-mentioned target area density map and each of the above-mentioned minimum enclosing boxes, determine the area of each of the above-mentioned minimum enclosing boxes, the sum of the density values in each of the above-mentioned minimum enclosing boxes, and the area of the density connected region in each of the above-mentioned minimum enclosing boxes;

Based on the second preset model, the area of each of the aforementioned smallest bounding boxes, the sum of the density values in each of the aforementioned smallest bounding boxes, the area of the density-connected regions in each of the aforementioned smallest bounding boxes, and the total area of the aforementioned original image, each of the aforementioned The proportion of the predicted target area corresponding to the smallest bounding box.

Specifically, the above-mentioned second preset model may be a second preset neural network. When training the above-mentioned second preset neural network, the training samples are more than one sample density map and sample original image containing several minimum bounding boxes (the The information contained in the density map includes at least: the sum of the density values of each minimum bounding box, the area of the density connected region of each minimum bounding box, the area of each minimum bounding box and the total area of the sample original image), corresponding to each of the above sample density maps The training label of is the average value of the ratio of the area of each actual target to the total area of the above-mentioned mapping area in the mapping area of each minimum bounding box on the above-mentioned original image.

In step 105, target detection is performed on each of the aforementioned block images, so as to output target detection results corresponding to each of the aforementioned block images.

In the embodiment of the present application, target detection can be performed on the above-mentioned block images respectively, so as to output the target detection results corresponding to each of the above-mentioned block images, and the target detection results can include: Bounding boxes and corresponding category information.

Optionally, after performing target detection on each of the above-mentioned block images, the above-mentioned target detection method further includes:

Based on the non-maximum value suppression algorithm, the above-mentioned target detection results are combined to obtain the target detection result of the above-mentioned original image.

Specifically, based on the non-maximum value suppression algorithm, the target detection results of each block image are integrated on the original image, so as to output the target detection result of the original image.

Optionally, the above-mentioned target detection for each of the above-mentioned block images includes:

Based on the third preset model, target detection is performed on each of the aforementioned block images.

Specifically, the above-mentioned third preset model may be a third preset neural network. When training the above-mentioned third preset neural network, the training sample is more than one sample block image cut out, and each of the above sample block images The corresponding training labels include: object bounding boxes used to enclose each object in the sample block image and corresponding category information.

Optionally, each of the minimum bounding boxes above may be a minimum vertical bounding box.

It can be seen from the above that the technical solution of the present application includes extracting the feature information of the original image to obtain the feature information of the original image; generating the corresponding density map and segmentation map based on the feature information of the original image; determining the target area in the original image based on the segmentation map , and based on the target area and the density map, the target area density map is generated; based on the target area density map, the block images related to the target area density map are cut out from the original image; the target detection is performed on each block image to output The target detection results corresponding to each block image. Based on the technical solution of this application, the segmentation map and density map of the original image can be extracted first, and the density map can be further deleted based on the segmentation map to improve the pertinence of the target area density map to the target in the original image, and then based on the target The area density map cuts the original image, and finally performs target detection on each cropped block image to obtain each target detection result. Since the target detection is only performed on block images related to the target area density map, it can effectively exclude the original image. Noise interference, improve the accuracy of target detection.

Embodiment two

The present application provides a target detection device. As shown in FIG. 2, the target detection device 20 includes:

An extraction unit 201, configured to extract feature information from the original image to obtain feature information of the original image;

The first generation unit 202 is configured to generate a corresponding density map and segmentation map based on the above-mentioned original image feature information;

The second generation unit 203 is configured to determine the target area in the original image based on the segmentation map, and generate a density map of the target area based on the target area and the density map, wherein the target area is the brightness value in the segmentation map For an area formed by points larger than the first preset value, the above-mentioned target area density map is a density map that only includes density information of the above-mentioned target area;

A cropping unit 204, configured to crop a block image related to the target area density map from the above-mentioned original image based on the above-mentioned target area density map;

The target detection unit 205 is configured to perform target detection on each of the above-mentioned block images, so as to output a target detection result corresponding to each of the above-mentioned block images.

Optionally, the cropping unit 204 is specifically used for:

Based on the above-mentioned target area density map, generate a minimum bounding box for enclosing the density-connected area in the above-mentioned target area density map, wherein the above-mentioned density-connected area is an area composed of connected points whose density value is greater than a second preset value;

Based on each of the above-mentioned minimum bounding boxes, the above-mentioned target area density map, and the above-mentioned original image, the predicted target area proportion corresponding to each of the above-mentioned minimum bounding boxes is obtained. In the mapped area above, the average value of the ratio of the area of each predicted target to the total area of the above mapped area;

Based on the size relationship between each of the above-mentioned predicted target area ratios and the preset standard ratio, the scale of each of the above-mentioned minimum bounding boxes is adjusted, so that the predicted target area ratio of each adjusted minimum bounding box is smaller than that before adjustment. Close to the above standard ratio;

Based on the position of each of the above-mentioned adjusted minimum bounding boxes mapped on the above-mentioned original image, the original image is cropped to obtain one or more block images corresponding to each of the above-mentioned adjusted minimum bounding boxes.

Further, the cropping unit 204 is also specifically used for:

Calculate the ratio of the predicted target area ratio of each of the above minimum bounding boxes to the above standard ratio;

If there is a first-type bounding box, increase the scale of the above-mentioned first-type bounding box, so that the predicted target area ratio of each adjusted first-type bounding box is closer to the above-mentioned standard ratio than before adjustment, where , the above-mentioned first type of bounding box is the smallest bounding box with the above-mentioned ratio greater than the third preset value among the above-mentioned minimum bounding boxes;

If there is a second-type bounding box, reduce the size of the second-type bounding box, and/or, divide the second-type bounding box into two or more minimum bounding boxes, so that each adjusted second-type bounding box The proportion of the predicted target area of the frame is closer to the above-mentioned standard proportion than before adjustment, wherein the above-mentioned second type of bounding box is the smallest bounding box with the above-mentioned ratio smaller than the fourth preset value in each of the above-mentioned smallest bounding boxes;

Wherein, the above-mentioned third preset value is not less than the above-mentioned fourth preset value.

Further, the cropping unit 204 is also specifically used for:

Merging two minimum bounding boxes whose distance is less than a preset distance among the above minimum bounding boxes to form a minimum bounding box for enclosing the density-connected area of the two minimum bounding boxes whose distance is less than a preset distance.

Further, the cropping unit 204 is also specifically used for:

Based on the above-mentioned target area density map and each of the above-mentioned minimum enclosing boxes, determine the area of each of the above-mentioned minimum enclosing boxes, the sum of the density values in each of the above-mentioned minimum enclosing boxes, and the area of the density connected region in each of the above-mentioned minimum enclosing boxes;

Based on the second preset model, the area of each of the aforementioned smallest bounding boxes, the sum of the density values in each of the aforementioned smallest bounding boxes, the area of the density-connected regions in each of the aforementioned smallest bounding boxes, and the total area of the aforementioned original image, each of the aforementioned The proportion of the predicted target area corresponding to the smallest bounding box.

Optionally, the target detection unit 205 is also used for:

Based on the non-maximum value suppression algorithm, the above-mentioned target detection results are combined to obtain the target detection result of the above-mentioned original image.

Optionally, the first generating unit 202 is specifically configured to:

Based on the first preset model, feature information is extracted from the original image, and a corresponding density map and segmentation map are generated.

Optionally, the target detection unit 205 is specifically used for:

Based on the third preset model, target detection is performed on each of the aforementioned block images.

It can be seen from the above that the technical solution of the present application includes extracting the feature information of the original image to obtain the feature information of the original image; generating the corresponding density map and segmentation map based on the feature information of the original image; determining the target area in the original image based on the segmentation map , and based on the target area and the density map, the target area density map is generated; based on the target area density map, the block images related to the target area density map are cut out from the original image; the target detection is performed on each block image to output The target detection results corresponding to each block image. Based on the technical solution of this application, the segmentation map and density map of the original image can be extracted first, and the density map can be further deleted based on the segmentation map to improve the pertinence of the target area density map to the target in the original image, and then based on the target The area density map cuts the original image, and finally performs target detection on each cropped block image to obtain each target detection result. Since the target detection is only performed on block images related to the target area density map, it can effectively exclude the original image. Noise interference, improve the accuracy of target detection.

Embodiment three

The present application also provides another target detection device. As shown in FIG. 3 , the target detection device in the embodiment of the present application includes: a memory 301, a processor 302, and a computer stored in the memory 301 and operable on the processor 302 program, wherein: the memory 301 is used to store software programs and modules, and the processor 302 executes various functional applications and data processing by running the software programs and modules stored in the memory 301 , and the memory 301 and processor 302 are connected through a bus 303 .

Specifically, the processor 302 implements the following steps by running the above computer program stored in the memory 301:

Extract feature information from the original image to obtain feature information of the original image;

Generate the corresponding density map and segmentation map based on the above original image feature information;

Determine the target area in the original image based on the segmentation map, and generate a target area density map based on the target area and the density map, wherein the target area is a point with a brightness value greater than a first preset value in the segmentation map In the formed area, the density map of the above-mentioned target area is a density map that only includes the density information of the above-mentioned target area;

Based on the above-mentioned target area density map, cutting out block images related to the target area density map from the above-mentioned original image;

Target detection is performed on each of the above-mentioned block images, so as to output a target detection result corresponding to each of the above-mentioned block images.

Assuming that the above is the first possible implementation manner, then in the second possible implementation manner based on the above first possible implementation manner, the target area is cut out from the above original image based on the above target area density map Density map related tiled images include:

Based on the above-mentioned target area density map, generate a minimum bounding box for enclosing the density-connected area in the above-mentioned target area density map, wherein the above-mentioned density-connected area is an area composed of connected points whose density value is greater than a second preset value;

Based on each of the above-mentioned minimum bounding boxes, the above-mentioned target area density map, and the above-mentioned original image, the predicted target area ratio corresponding to each of the above-mentioned minimum bounding boxes is obtained. In the mapped area above, the average value of the ratio of the area of each predicted target to the total area of the above mapped area;

Based on the size relationship between each of the above-mentioned predicted target area ratios and the preset standard ratio, the scale of each of the above-mentioned minimum bounding boxes is adjusted, so that the predicted target area ratio of each adjusted minimum bounding box is smaller than that before adjustment. Close to the above standard ratio;

Based on the position of each of the above-mentioned adjusted minimum bounding boxes mapped on the above-mentioned original image, the original image is cropped to obtain one or more block images corresponding to each of the above-mentioned adjusted minimum bounding boxes.

In a third possible implementation manner based on the above second possible implementation manner, based on the size relationship between each of the above-mentioned predicted target area ratios and the preset standard ratio, the scale of each of the above-mentioned minimum bounding boxes is adjusted, To make the proportion of the predicted target area of each adjusted minimum bounding box closer to the above-mentioned standard proportion than before the adjustment includes:

Calculate the ratio of the predicted target area ratio of each of the above minimum bounding boxes to the above standard ratio;

If there is a first-type bounding box, increase the scale of the above-mentioned first-type bounding box, so that the predicted target area ratio of each adjusted first-type bounding box is closer to the above-mentioned standard ratio than before adjustment, where , the above-mentioned first type of bounding box is the smallest bounding box with the above-mentioned ratio greater than the third preset value among the above-mentioned minimum bounding boxes;

If there is a second-type bounding box, reduce the size of the second-type bounding box, and/or, divide the second-type bounding box into two or more minimum bounding boxes, so that each adjusted second-type bounding box The proportion of the predicted target area of the frame is closer to the above-mentioned standard proportion than before adjustment, wherein the above-mentioned second type of bounding box is the smallest bounding box with the above-mentioned ratio smaller than the fourth preset value in each of the above-mentioned smallest bounding boxes;

Wherein, the above-mentioned third preset value is not less than the above-mentioned fourth preset value.

In a fourth possible implementation based on the above-mentioned second or third possible implementation, based on the above-mentioned target area density map, generate the minimum enclosing After the box, the above target detection method also includes:

Merging two minimum bounding boxes whose distance is less than a preset distance among the above minimum bounding boxes to form a minimum bounding box for enclosing the density-connected area of the two minimum bounding boxes whose distance is less than a preset distance.

In the fifth possible implementation manner based on the first, second, or third possible implementation manners above, after performing target detection on each of the above-mentioned block images, the above-mentioned target detection method further includes:

Based on the non-maximum value suppression algorithm, the above-mentioned target detection results are combined to obtain the target detection result of the above-mentioned original image.

In the sixth possible implementation manner based on the above-mentioned first, second, or third possible implementation manners, the feature information of the original image is extracted to obtain the feature information of the original image, and based on the above-mentioned feature information of the original image, Generate the corresponding density map and segmentation map including:

Based on the first preset model, feature information is extracted from the original image, and a corresponding density map and segmentation map are generated.

In the seventh possible implementation manner based on the above-mentioned second or third possible implementation manner, based on each of the above-mentioned minimum bounding boxes, the above-mentioned target area density map and the above-mentioned original image, the corresponding The proportion of predicted target area includes:

Based on the above-mentioned target area density map and each of the above-mentioned minimum enclosing boxes, determine the area of each of the above-mentioned minimum enclosing boxes, the sum of the density values in each of the above-mentioned minimum enclosing boxes, and the area of the density connected region in each of the above-mentioned minimum enclosing boxes;

Based on the second preset model, the area of each of the aforementioned smallest bounding boxes, the sum of the density values in each of the aforementioned smallest bounding boxes, the area of the density-connected regions in each of the aforementioned smallest bounding boxes, and the total area of the aforementioned original image, each of the aforementioned The proportion of the predicted target area corresponding to the smallest bounding box.

In an eighth possible implementation manner based on the above-mentioned first or second or third possible implementation manner, the above-mentioned performing target detection on each of the above-mentioned block images respectively includes:

Based on the third preset model, target detection is performed on each of the aforementioned block images.

It can be seen from the above that the technical solution of the present application includes extracting the feature information of the original image to obtain the feature information of the original image; generating the corresponding density map and segmentation map based on the feature information of the original image; determining the target area in the original image based on the segmentation map , and based on the target area and the density map, the target area density map is generated; based on the target area density map, the block images related to the target area density map are cut out from the original image; the target detection is performed on each block image to output The target detection results corresponding to each block image. Based on the technical solution of this application, the segmentation map and density map of the original image can be extracted first, and the density map can be further deleted based on the segmentation map to improve the pertinence of the target area density map to the target in the original image, and then based on the target The area density map cuts the original image, and finally performs target detection on each cropped block image to obtain each target detection result. Since the target detection is only performed on block images related to the target area density map, it can effectively exclude the original image. Noise interference, improve the accuracy of target detection.

Embodiment four

The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed, the steps provided in the above-mentioned embodiments can be realized. Specifically, the computer program includes computer program code, which may be in the form of source code, object code, executable file or some intermediate form, which is not limited here; the computer-readable storage medium may be Any entity or device capable of carrying the above computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-OnlyMemory), random access memory (RAM, RandomAccessMemory) , an electrical carrier signal, a telecommunication signal, and a software distribution medium, which are not limited herein. It should be noted that the content contained in the above computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

It can be seen from the above that the technical solution of the present application includes extracting the feature information of the original image to obtain the feature information of the original image; generating the corresponding density map and segmentation map based on the feature information of the original image; determining the target area in the original image based on the segmentation map , and based on the target area and the density map, the target area density map is generated; based on the target area density map, the block images related to the target area density map are cut out from the original image; the target detection is performed on each block image to output The target detection results corresponding to each block image. Based on the technical solution of this application, the segmentation map and density map of the original image can be extracted first, and the density map can be further deleted based on the segmentation map to improve the pertinence of the target area density map to the target in the original image, and then based on the target The area density map cuts the original image, and finally performs target detection on each cropped block image to obtain each target detection result. Since the target detection is only performed on block images related to the target area density map, it can effectively exclude the original image. Noise interference, improve the accuracy of target detection.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Those skilled in the art can clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used for illustration. In practical applications, the above-mentioned functions can be assigned to different functional units, Module completion means that the internal structure of the above-mentioned device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated into one processing unit, or each unit may exist separately physically, or two or more units may be integrated into one unit, and the above-mentioned integrated units may adopt hardware It can also be implemented in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present application. For the specific working processes of the units and modules in the above system, reference may be made to the corresponding processes in the aforementioned method embodiments, and details will not be repeated here.

It should be noted that the methods provided in the above embodiments and their detailed examples can be combined into the devices and equipment provided in the embodiments, refer to each other, and will not be repeated here.

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

In the embodiments provided in this application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the device/device embodiments described above are only illustrative. For example, the division of the above modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components May be combined or may be integrated into another system, or some features may be omitted, or not implemented.

The foregoing embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be described in the foregoing embodiments Modifications to the technical solutions recorded, or equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the application, and should be included in this application. within the scope of protection.

Claims (8)

1. A method of detecting an object, comprising:

extracting feature information of the original image to obtain feature information of the original image;

generating a corresponding density map and a corresponding segmentation map based on the original image characteristic information;

determining a target area in the original image based on the segmentation map, and generating a target area density map based on the target area and the density map, wherein the target area is an area formed by points with brightness values larger than a first preset value in the segmentation map, and the target area density map is a density map only containing density information of the target area;

Based on the target area density map, cutting out a block image related to the target area density map from the original image;

performing target detection on each segmented image respectively to output a target detection result corresponding to each segmented image;

the clipping the segmented image related to the target region density map from the original image based on the target region density map comprises:

generating a minimum bounding box for bounding a density connected region in the target region density map based on the target region density map, wherein the density connected region is a region formed by connected points with a density value larger than a second preset value;

obtaining a predicted target area occupation ratio corresponding to each minimum bounding box based on each minimum bounding box, the target area density map and the original image, wherein the predicted target area occupation ratio is an average value of the proportion of the area of each predicted target to the total area of a mapping area of the corresponding minimum bounding box on the original image;

based on the size relation between the predicted target area ratio and the preset standard ratio, carrying out scale adjustment on each minimum bounding box so that the difference between the predicted target area ratio of each adjusted minimum bounding box and the standard ratio is smaller than the difference between the predicted target area ratio of the minimum bounding box before adjustment and the standard ratio;

Clipping the original image based on the position of each adjusted minimum bounding box mapped on the original image to obtain more than one block image corresponding to each adjusted minimum bounding box one by one;

and scaling each minimum bounding box based on the size relation between each predicted target area duty ratio and a preset standard duty ratio, so that the difference between the predicted target area duty ratio of each adjusted minimum bounding box and the standard duty ratio is smaller than the difference between the predicted target area duty ratio of the minimum bounding box and the standard duty ratio before adjustment, wherein the step of scaling each minimum bounding box comprises the following steps:

calculating the ratio of the predicted target area occupation ratio of each minimum bounding box to the standard occupation ratio respectively;

if a first type bounding box exists, increasing the scale of the first type bounding box so that the difference between the predicted target area occupation ratio of each adjusted first type bounding box and the standard occupation ratio is smaller than the difference between the predicted target area occupation ratio of the minimum bounding box before adjustment and the standard occupation ratio, wherein the first type bounding box is the minimum bounding box in each minimum bounding box, and the ratio of the minimum bounding box is larger than a third preset value;

if a second type bounding box exists, reducing the scale of the second type bounding box, and/or dividing the second type bounding box into more than two minimum bounding boxes, so that the difference between the predicted target area occupation ratio of each adjusted second type bounding box and the standard occupation ratio is smaller than the difference between the predicted target area occupation ratio of the minimum bounding box before adjustment and the standard occupation ratio, wherein the second type bounding box is the minimum bounding box with the ratio smaller than a fourth preset value in each minimum bounding box;

Wherein the third preset value is not less than the fourth preset value.

2. The target detection method according to claim 1, wherein after the generating of the minimum bounding box for bounding a density connected region in the target region density map based on the target region density map, the target detection method further comprises:

and merging the two minimum bounding boxes with the distance smaller than the preset distance in the minimum bounding boxes to form a minimum bounding box for surrounding the density communication area of the two minimum bounding boxes with the distance smaller than the preset distance.

3. The object detection method according to claim 1, wherein after said object detection is performed on each of the block images, the object detection method further comprises:

and merging the target detection results based on a non-maximum suppression algorithm to obtain the target detection result of the original image.

4. The method of claim 1, wherein extracting feature information of the original image to obtain feature information of the original image, and generating the corresponding density map and segmentation map based on the feature information of the original image comprises:

And extracting characteristic information of the original image based on the first preset model, and generating a corresponding density map and a corresponding segmentation map.

5. The method of claim 1, wherein obtaining a predicted target area occupation ratio corresponding to each minimum bounding box based on each minimum bounding box, the target area density map, and the original image comprises:

determining the area of each minimum bounding box, the sum of density values in each minimum bounding box and the area of a density communication area in each minimum bounding box based on the target area density map and each minimum bounding box;

and obtaining a predicted target area occupation ratio corresponding to each minimum bounding box based on a second preset model, the area of each minimum bounding box, the sum of density values in each minimum bounding box, the area of a density communication area in each minimum bounding box and the total area of the original image.

6. The method according to claim 1, wherein the performing object detection on each of the block images respectively includes:

and respectively carrying out target detection on each segmented image based on a third preset model.

7. An object detection apparatus, comprising:

the extraction unit is used for extracting the characteristic information of the original image to obtain the characteristic information of the original image;

the first generation unit is used for generating a corresponding density map and a corresponding segmentation map based on the original image characteristic information;

a second generating unit, configured to determine a target area in the original image based on the segmentation map, and generate a target area density map based on the target area and the density map, where the target area is an area formed by points in the segmentation map where a luminance value is greater than a first preset value, and the target area density map is a density map that only includes density information of the target area;

a clipping unit, configured to clip a segmented image related to the target area density map from the original image based on the target area density map;

the target detection unit is used for respectively carrying out target detection on each segmented image so as to output a target detection result corresponding to each segmented image;

the clipping unit is specifically configured to generate, based on the target area density map, a minimum bounding box for bounding a density connected area in the target area density map, where the density connected area is an area formed by connected points with a density value greater than a second preset value;

Obtaining a predicted target area occupation ratio corresponding to each minimum bounding box based on each minimum bounding box, the target area density map and the original image, wherein the predicted target area occupation ratio is an average value of the proportion of the area of each predicted target to the total area of a mapping area of the corresponding minimum bounding box on the original image;

based on the size relation between the predicted target area ratio and the preset standard ratio, carrying out scale adjustment on each minimum bounding box so that the difference between the predicted target area ratio of each adjusted minimum bounding box and the standard ratio is smaller than the difference between the predicted target area ratio of the minimum bounding box before adjustment and the standard ratio;

clipping the original image based on the position of each adjusted minimum bounding box mapped on the original image to obtain more than one block image corresponding to each adjusted minimum bounding box one by one;

the clipping unit is specifically further configured to calculate a ratio of a predicted target area occupation ratio of each minimum bounding box to the standard occupation ratio;

if a first type bounding box exists, increasing the scale of the first type bounding box so that the difference between the predicted target area occupation ratio of each adjusted first type bounding box and the standard occupation ratio is smaller than the difference between the predicted target area occupation ratio of the minimum bounding box before adjustment and the standard occupation ratio, wherein the first type bounding box is the minimum bounding box in each minimum bounding box, and the ratio of the minimum bounding box is larger than a third preset value;

If a second type bounding box exists, reducing the scale of the second type bounding box, and/or dividing the second type bounding box into more than two minimum bounding boxes, so that the difference between the predicted target area occupation ratio of each adjusted second type bounding box and the standard occupation ratio is smaller than the difference between the predicted target area occupation ratio of the minimum bounding box before adjustment and the standard occupation ratio, wherein the second type bounding box is the minimum bounding box with the ratio smaller than a fourth preset value in each minimum bounding box;

wherein the third preset value is not less than the fourth preset value.

8. An object detection device comprising a memory and a processor, the memory storing a computer program, the processor implementing the steps of the method according to any one of claims 1 to 6 when the computer program is executed.