CN108140291A - Mist detecting device, method and image processing equipment - Google Patents

Abstract

A smoke detection device, method and image processing equipment. The smoke detection method includes: performing background image modeling on the current image to obtain a foreground image and a background image of the current image (101); obtaining one or more candidate regions for detecting moving objects in the current image based on the foreground image (102) ; calculating attribute information (103) of a candidate area corresponding to the current image and/or background image; and determining whether there is smoke in the candidate area according to the attribute information (104). Therefore, not only can smoke be detected quickly and accurately through video images, but also the detection accuracy of smoke detection based on video under illumination changes and complex environments can be improved.

Description

Smoke detection device, method and image processing device technical field

The invention relates to the technical field of graphics and images, in particular to a smoke detection device, method and image processing equipment.

Background technique

Currently, smoke detection is required in video surveillance. For example, when a fire breaks out in a certain part of the building, if the smoke in this area can be automatically detected through the video image, the fire alarm can be issued as soon as possible to reduce the loss caused by the fire.

However, due to the diffuse nature of smoke motion, it is difficult to accurately detect smoke based on video images. In the prior art, the technical solutions for detecting the presence of smoke by detecting video images all have the problems of low detection accuracy and incapable of fast and accurate detection.

It should be noted that the above introduction of the technical background is only for the convenience of a clear and complete description of the technical solution of the present invention, and for the convenience of understanding by those skilled in the art. It cannot be considered that the above technical solutions are known to those skilled in the art just because these solutions are described in the background of the present invention.

Contents of the invention

Embodiments of the present invention provide a smoke detection device, method, and image processing equipment, which can quickly and accurately detect smoke through video images, and improve the detection accuracy of video-based smoke detection under illumination changes and complex environments.

According to a first aspect of an embodiment of the present invention, a smoke detection device is provided, wherein the smoke detection device includes:

a background image modeling unit, performing background image modeling on the current image to obtain a foreground image and a background image of the current image;

A candidate area acquiring unit, based on the foreground image, acquires one or more candidate areas for detecting moving objects in the current image;

an attribute information calculation unit, which calculates the attribute information of a candidate area corresponding to the current image and/or the background image; and

The smoke determination unit is configured to determine whether there is smoke in the certain candidate area according to the attribute information.

According to a second aspect of an embodiment of the present invention, a smoke detection method is provided, wherein the smoke detection method includes:

performing background image modeling on the current image to obtain a foreground image and a background image of the current image;

Acquiring one or more candidate regions for detecting moving objects in the current image based on the foreground image;

calculating attribute information of a candidate area corresponding to the current image and/or the background image; and

Determine whether there is smoke in the certain candidate area according to the attribute information.

According to a third aspect of the embodiments of the present invention, there is provided an image processing device, wherein the image processing device includes the above-mentioned smoke detection device.

According to yet another aspect of an embodiment of the present invention, there is provided a computer-readable program, wherein when the program is executed in an image processing device, the program causes a computer to perform smoke detection as described above in the image processing device method.

According to yet another aspect of the embodiments of the present invention, there is provided a storage medium storing a computer-readable program, wherein the computer-readable program enables a computer to execute the above smoke detection method in an image processing device.

The beneficial effect of the embodiment of the present invention is that one or more candidate areas are acquired based on the foreground image, the attribute information of a certain candidate area corresponding to the current image and/or the background image is calculated, and it is determined according to the attribute information whether there is smoke. Therefore, not only can the smoke be detected quickly and accurately through video images, but also the detection accuracy of video-based smoke detection under illumination changes and complex environments can be improved.

With reference to the following description and accompanying drawings, there are disclosed in detail specific embodiments of the invention, indicating the manner in which the principles of the invention may be employed. It should be understood that embodiments of the invention are not limited thereby in scope. Embodiments of the invention encompass many changes, modifications and equivalents within the spirit and scope of the appended claims.

Features described and/or illustrated with respect to one embodiment can be used in the same or similar manner in one or more other embodiments, in combination with, or instead of features in other embodiments .

It should be emphasized that the term "comprising/comprising" when used herein refers to the presence of a feature, integer, step or component, but does not exclude the presence or addition of one or more other features, integers, steps or components.

Description of drawings

Many aspects of the invention can be better understood with reference to the following figures. The components in the figures are not drawn to scale, merely illustrating the principles of the invention. In order to facilitate illustration and description of some parts of the present invention, corresponding parts in the drawings may be exaggerated or reduced.

Elements and features described in one drawing or one embodiment of the present invention may be combined with elements and features shown in one or more other drawings or embodiments. Furthermore, in the drawings, like numerals indicate corresponding parts in the several figures and may be used to indicate corresponding parts used in more than one embodiment.

Fig. 1 is a schematic diagram of the smoke detection method of embodiment 1 of the present invention;

Fig. 2 is a schematic diagram of the extracted connected domain in Embodiment 1 of the present invention;

Fig. 3 is another schematic diagram of the smoke detection method of Embodiment 1 of the present invention;

Fig. 4 is a schematic diagram of acquiring a continuous motion area in Embodiment 1 of the present invention;

Fig. 5 is a schematic diagram of smoke detection for a certain candidate area according to Embodiment 1 of the present invention;

Fig. 6 is a schematic diagram of the direction of Embodiment 1 of the present invention;

FIG. 7 is another schematic diagram of smoke detection for a certain candidate area according to Embodiment 1 of the present invention;

FIG. 8 is another schematic diagram of smoke detection for a certain candidate area according to Embodiment 1 of the present invention;

FIG. 9 is another schematic diagram of smoke detection for a certain candidate area according to Embodiment 1 of the present invention;

Fig. 10 is another schematic diagram of smoke detection for a certain candidate area according to Embodiment 1 of the present invention;

Fig. 11 is a schematic diagram of obtaining the remaining motion area according to Embodiment 1 of the present invention;

Fig. 12 is another schematic diagram of smoke detection for a certain candidate area according to Embodiment 1 of the present invention;

Fig. 13 is a schematic diagram of a smoke detection device according to Embodiment 2 of the present invention;

Fig. 14 is a schematic diagram of the candidate area acquisition unit in Embodiment 2 of the present invention;

Fig. 15 is another schematic diagram of the smoke detection device according to Embodiment 2 of the present invention;

FIG. 16 is a schematic diagram of an attribute information calculation unit according to Embodiment 2 of the present invention;

Fig. 17 is another schematic diagram of the attribute information calculation unit according to Embodiment 2 of the present invention;

Fig. 18 is another schematic diagram of the attribute information calculation unit according to Embodiment 2 of the present invention;

Fig. 19 is another schematic diagram of the attribute information calculation unit according to Embodiment 2 of the present invention;

Fig. 20 is another schematic diagram of the attribute information calculation unit according to Embodiment 2 of the present invention;

Fig. 21 is another schematic diagram of the attribute information calculation unit according to Embodiment 2 of the present invention;

Fig. 22 is a schematic diagram of an image processing apparatus according to Embodiment 3 of the present invention.

Detailed ways

The foregoing and other features of the invention will become apparent from the following description, taken with reference to the accompanying drawings. In the specification and drawings, specific embodiments of the invention are disclosed, which illustrate some embodiments in which the principles of the invention may be employed. It is to be understood that the invention is not limited to the described embodiments, but rather, the invention The invention includes all modifications, variations and equivalents that come within the scope of the appended claims.

Example 1

An embodiment of the present invention provides a smoke detection method. Fig. 1 is a schematic diagram of a smoke detection method according to an embodiment of the present invention. As shown in Fig. 1, the smoke detection method includes:

Step 101, performing background image modeling on the current image to obtain a foreground image and a background image of the current image;

Step 102, acquiring one or more candidate regions for detecting moving objects in the current image based on the foreground image;

Step 103, calculating attribute information of a candidate area corresponding to the current image and/or background image; and

Step 104, determine whether there is smoke in the candidate area according to the attribute information.

In this embodiment, a device such as a camera may be used to obtain a video containing multiple frames. The background image modeling method based on Gaussian Mixture Model (GMM, Gaussian Mixture Model) can be used to obtain the foreground image and the background image after performing background modeling on the colored current image (or called the current frame) of the input video. But the present invention is not limited thereto, and any method of background image modeling can be used.

In this embodiment, one or more candidate regions may be acquired based on the foreground image. Specifically, the foreground image may be represented as a binary image to obtain a binary image of the foreground image; for example, the pixel value of the foreground part of the pixels is "1", and the pixel value of the background part of the pixels is "0".

In this embodiment, median filtering may be performed on the binarized image to remove small noise points. Then, multiple pixels in the binarized image with the same pixel value (for example, "1") and connected to each other are used as a connected domain, so as to obtain one or more connected domains representing moving objects in the foreground image. For example, several connected domains of different sizes can be extracted from a binarized image.

FIG. 2 is a schematic diagram of an extracted connected domain according to an embodiment of the present invention. As shown in FIG. 2 , a plurality of connected pixels with pixel values such as “1” may form a connected domain. A total of 5 connected domains can be extracted from the binarized image, which are denoted as connected domain 201 , connected domain 202 , ... connected domain 205 .

In this embodiment, one or more connected domains may be selected to obtain one or more candidate regions. For example, connected domains whose area is less than or equal to a preset threshold (first threshold) may be removed, and/or connected domains whose average color depth is outside a preset range may be removed. The specific value of the first threshold may be preset based on empirical values, for example, and the present invention is not limited to the first threshold.

In the example shown in FIG. 2 , for example, the areas of the connected domains 203 and 204 are all smaller than the first threshold, and the areas of the connected domains 201, 202 and 205 are all greater than the first threshold, then the connected domains 201, 202 and 205 can be as a candidate area.

In this embodiment, for each candidate area, it may be determined whether there is smoke in the candidate area. For example, for a certain candidate area, attribute information corresponding to the current image and/or background image of the candidate area may be calculated; and whether smoke exists in the candidate area is determined according to the attribute information.

Wherein, the attribute information may include one or more of the following: saturation information, grayscale variance information, gradient direction information, grayscale average information, and motion direction information. However, the present invention is not limited thereto, and other attribute information corresponding to the current image and/or background image may also be used, and the present invention only uses the above attribute information as an example for illustration.

Therefore, not only can the smoke be detected quickly and accurately through video images, but also the detection accuracy of video-based smoke detection under illumination changes and complex environments can be improved.

FIG. 3 is another schematic diagram of the method for detecting smoke according to an embodiment of the present invention, further using continuous motion regions based on candidate regions to detect smoke. As shown in Figure 3, the smoke detection method includes:

Step 301, perform background image modeling on the current image to obtain a foreground image and a background image of the current image.

Step 302, based on the foreground image, one or more candidate regions for detecting moving objects in the current image are acquired.

Step 303, selecting a certain candidate area.

Step 304, acquiring the continuous motion region corresponding to the candidate region according to the positions of the candidate region in the plurality of image frames respectively;

In this embodiment, a plurality of consecutive (for example N) image frames before the current frame may be acquired, and then the candidate regions corresponding to the N+1 frames are combined to construct a continuous motion region corresponding to the candidate region, That is, the continuous motion area is the "motion trajectory" of the candidate area in the N+1 image frames.

Fig. 4 is a schematic diagram of obtaining a continuous motion region according to an embodiment of the present invention. As shown in Fig. 4, the current frame is marked as the Nth frame, and there are candidate regions in a total of N+1 image frames from the Nth frame to the 0th frame 401 , the positions and shapes of the candidate regions 401 from frame 0 to frame N may be different, and a continuous motion region 402 can be obtained by merging these candidate regions 401 .

Step 305, calculating attribute information of the candidate area corresponding to the current image and/or the background image based on the continuous motion area.

Step 306, determine whether there is smoke in the candidate area according to the attribute information.

Step 307, judging whether there are other candidate areas; if so, execute step 303 to select another candidate area, and continue to judge the other candidate area.

The flow of the smoke detection method in the embodiment of the present invention has been schematically described above, and the following will take the moving direction information, saturation average value, grayscale variance value, gradient direction information average value, and grayscale average value as examples to describe this The smoke detection of a certain candidate area in the invention is further described. For how to obtain the candidate region and how to obtain the continuous motion region, you can refer to the above content.

In one embodiment (implementation 1), it may be determined whether there is smoke in a candidate region according to whether the main motion direction of the candidate region is downward in multiple image frames.

Fig. 5 is a schematic diagram of smoke detection for a certain candidate area according to an embodiment of the present invention. As shown in Fig. 5, the method includes:

Step 501 , based on the position of the center of mass and the position of the center of gravity of the candidate area, calculate the motion direction of the candidate area relative to the current image in multiple image frames.

For example, the centroid Mc(Xc, Yc) of the "candidate region" in the current frame can be calculated by the following formula (1):

Among them, N is the number of pixels contained in the "candidate region", p∈component means that the "candidate region" contains pixel p, p.x refers to the x coordinate of pixel p, and p.y refers to the y coordinate of pixel p.

Suppose the centroid of the current frame "candidate area" is Mc(Xc, Yc), and the center of gravity of the corresponding "candidate area" in several frames before the current frame (for example, the previous 5th or 10th frame) is Mp(Xp, Yp ), then calculate the following value:

ΔX= _Xc - _Xp ;

ΔY=Y _c -Y _p

FIG. 6 shows a schematic diagram of directions in an embodiment of the present invention. As shown in FIG. 6 , eight directions can be defined. However, the present invention is not limited thereto, for example, more or fewer directions may be defined, which may be specifically defined according to actual conditions.

If ΔX>0 and ΔY=0, the direction of motion is 1;

If ΔX>0 and ΔY<0, the direction of motion is 2;

If ΔX=0 and ΔY<0, the direction of motion is 3;

If ΔX<0 and ΔY<0, the direction of motion is 4;

If ΔX<0 and ΔY=0, the direction of motion is 5;

If ΔX<0 and ΔY>0, the direction of motion is 6;

If ΔX=0 and ΔY>0, the direction of movement is 7;

If ΔX>0 and ΔY>0, the direction of motion is 8.

Thus, the motion direction of the candidate region in each image frame relative to the current frame can be obtained.

Step 502, count the frequency of each motion direction appearing in multiple image frames.

Step 503, taking the motion direction with the highest frequency as the main motion direction of the candidate area.

In this embodiment, the motion direction of the "candidate area" in several consecutive frames can be recorded, and the frequency of occurrence of each motion direction can be recorded, and then the motion direction with the highest frequency is regarded as the main motion of the "candidate area" direction.

Step 504, judging whether the main movement direction is downward; if the main movement direction is downward, execute step 505;

Step 505, determining that there is no smoke in the candidate area.

In this embodiment, for example, if the main motion direction of the "candidate area" is downward (such as 6, 7, 8 as shown in Figure 6), the "candidate area" of the current frame can be moved from the "candidate area" to list; that is, it is determined that there is no smoke in the candidate area. In addition, if the main motion direction is not downward, it may be determined that there is smoke in the candidate area, or in order to make the detection result more accurate, the candidate area may continue to be detected for other items.

In another implementation (implementation 2), it may be determined whether there is smoke in a candidate area according to whether the saturation information of the candidate area in the continuous motion area is less than a preset threshold.

Fig. 7 is another schematic diagram of smoke detection for a certain candidate area according to an embodiment of the present invention. As shown in Fig. 7, the method includes:

Step 701, performing color space transformation on the current image, and calculating saturation color components according to the color components to obtain a saturation map of the current image;

For example, the calculation formula of saturation is shown in the following formula (2):

The above formula is only an example to illustrate how to calculate the saturation of a certain pixel. As for how to calculate the saturation, any existing method can be used, which will not be repeated here.

Step 702, based on the saturation map of the current image, calculate the current average saturation of the candidate area in the continuous motion area.

For example, the calculation formula of the current saturation average value may be shown in the following formula (3):

Among them, S _avg is the average value of the current saturation, Ω is the continuous motion area, N is the number of pixels in the continuous motion area, i is a certain pixel in the continuous motion area, S _i is the saturation value of the pixel i .

The above formula only exemplifies how to calculate the current average value of saturation, but the present invention is not limited thereto, and may also be properly adjusted or modified according to actual conditions.

Step 703, judging whether the current average value of saturation is greater than or equal to a preset threshold (second threshold); if the current average value of saturation is greater than or equal to the preset threshold, perform step 704;

In this embodiment, the specific value of the second threshold may be preset based on empirical values, for example, and the present invention is not limited to the second threshold.

Step 704, determining that there is no smoke in the candidate area.

In this embodiment, for example, if the current average saturation value is greater than or equal to the second threshold, it indicates that the saturation of the moving object is high, while the saturation of general smoke is low, so it can be determined that there is no smoke in the candidate area , the "candidate area" of the current frame can be removed from the "candidate area" list. In addition, when the current average saturation value is less than the second threshold, it may be determined that there is smoke in the candidate area, or in order to make the detection result more accurate, the candidate area may continue to be detected for other items.

In another embodiment (implementation 3), it may be determined whether there is smoke in a candidate region according to a comparison result of the current saturation information of a candidate region in the continuous motion region and the background saturation information.

Fig. 8 is another schematic diagram of smoke detection for a certain candidate area according to an embodiment of the present invention. As shown in Fig. 8, the method includes:

Step 801, perform color space transformation on the current image, and calculate the saturation color component according to the color component to get the saturation map of the current image;

Step 802, based on the saturation map of the current image, calculate the current average saturation of the candidate area in the continuous motion area.

Step 803, performing color space transformation on the background image, and calculating saturation color components according to the color components to obtain a saturation map of the background image;

Step 804, based on the saturation map of the background image, calculate the average background saturation of the candidate area in the continuous motion area.

In this embodiment, the formula (3) may be used as the calculation formula of the background saturation average value, for example.

Step 805, judging whether the current average saturation value is greater than or equal to the background saturation average value; if the current saturation average value is greater than or equal to the background saturation average value, perform step 806;

Step 806, determine that there is no smoke in the candidate area.

In this embodiment, for example, if the current average saturation value is greater than or equal to the background saturation average value, it indicates that the overall saturation of the candidate area is relatively high, and generally the overall saturation of an area with smoke will be low, so it can be It is determined that there is no smoke in the candidate area, and the "candidate area" of the current frame may be removed from the "candidate area" list. In addition, in the case that the current average saturation value is smaller than the average background saturation value, it can be determined that there is smoke in the candidate area, or in order to make the detection result more accurate, the detection of other items can continue to be performed on the candidate area.

In another embodiment (embodiment 4), it may be determined whether there is smoke in a candidate region according to the gray variance information of the candidate region in the continuous motion region.

Fig. 9 is another schematic diagram of smoke detection for a certain candidate area according to an embodiment of the present invention. As shown in Fig. 9, the method includes:

Step 901, based on the grayscale image of the current image, calculate the grayscale variance value of the candidate area in the continuous motion area.

For example, the calculation formula of the gray variance value can be shown in the following formula (4):

Among them, Ω is the continuous motion area, N is the number of pixels in the continuous motion area, i is a certain pixel in the continuous motion area, Y _i is the gray value of the pixel i, Y _avg is the gray value of the continuous motion area The average value of the gray level; Var is the variance value of the gray level.

The above formula only exemplifies how to calculate the gray variance value, but the present invention is not limited thereto, and can also be properly adjusted or modified according to actual conditions. In addition, as for how to calculate the grayscale image or the grayscale value, any existing method can be used, which will not be repeated here.

Step 902, judging whether the grayscale variance value is greater than or equal to a preset threshold (the third threshold); if the grayscale variance value is greater than or equal to the preset threshold value, perform step 903;

In this embodiment, the specific value of the third threshold may be preset based on empirical values, for example, and the present invention is not limited to the third threshold.

Step 903, determining that there is no smoke in the candidate area.

In this embodiment, for example, if the grayscale variance value is greater than or equal to the third threshold, it means that the texture of the object is relatively high, while the texture of general smoke is low, so it can be determined that there is no smoke in the candidate area, and the current This "candidate region" of the frame is removed from the list of "candidate regions". In addition, when the gray variance value is less than the third threshold, it can be determined that there is smoke in the candidate area, or in order to make the detection result more accurate, the candidate area can continue to be detected for other items.

In another embodiment (implementation 5), it may be determined whether there is smoke in a candidate region according to the gray average information of the candidate region in the continuous motion region.

Fig. 10 is another schematic diagram of smoke detection for a certain candidate area according to an embodiment of the present invention. As shown in Fig. 10, the method includes:

Step 1001, remove the candidate area from the continuous motion area to obtain the remaining motion area.

FIG. 11 is a schematic diagram of obtaining a remaining motion area according to an embodiment of the present invention, schematically showing the remaining motion area obtained on the basis of FIG. 4 . As shown in FIG. 11 , the candidate area 401 of the current image (the Nth frame) can be removed from the continuous motion area 402 shown in FIG. 4 , so as to obtain the remaining motion area 1101 .

Step 1002, based on the grayscale image of the current image, calculate the current average grayscale of the remaining candidate area;

For example, the calculation formula of the current gray-scale average value can be shown in the following formula (5):

Among them, Ω is the remaining motion area, N is the number of pixels in the remaining motion area, i is a certain pixel in the remaining motion area, Y _i is the gray value of the pixel i in the current image, F _avg is the remaining The current grayscale average of the motion area.

Step 1003, based on the grayscale image of the background image, calculate the background grayscale average value of the remaining candidate area.

For example, the calculation formula of the average value of the background gray level can be shown in the following formula (6):

Among them, Ω is the remaining motion area, N is the number of pixels in the remaining motion area, j is a certain pixel in the remaining motion area, Y _j is the gray value of the pixel j in the background image, B _avg is the remaining The background gray level average of the motion area.

Step 1004, calculating the difference between the current grayscale average and the background grayscale average;

Step 1005, judging whether the difference is less than or equal to a preset threshold (the fourth threshold); if the difference is less than or equal to the preset threshold, perform step 1006;

In this embodiment, the specific value of the fourth threshold may be preset based on empirical values, for example, and the present invention is not limited to the fourth threshold.

Step 1006, determine that there is no smoke in the candidate area.

In this embodiment, for example, if the difference between the current grayscale average value and the background grayscale average value is less than or equal to the fourth threshold, it indicates that the moving object in the candidate area is a rigid object, and generally smoke has a diffuse Therefore, it can be determined that there is no smoke in the candidate area, and this "candidate area" of the current frame can be removed from the "candidate area" list. In addition, if the difference is greater than the fourth threshold, it may be determined that there is smoke in the candidate area, or in order to make the detection result more accurate, the candidate area may continue to be detected for other items.

In another embodiment (implementation 6), it may be determined whether there is smoke in a candidate region according to the gradient direction information of the candidate region.

Fig. 12 is another schematic diagram of smoke detection for a certain candidate area according to an embodiment of the present invention. As shown in Fig. 12, the method includes:

Step 1201, for a certain pixel in the candidate area, calculate the horizontal gradient and vertical gradient of the pixel based on the grayscale image of the current image to obtain the current image gradient direction of the pixel;

Step 1202, calculate the horizontal gradient and vertical gradient of the pixel based on the grayscale image of the background image to obtain the The direction of the background image gradient in pixels.

In this embodiment, for example, the horizontal gradient of a certain pixel can be calculated as described in the following formula (7):

Gx=(-1)*f(x-1,y-1)+0*f(x,y-1)+1*f(x+1,y-1)

+(-2)*f(x-1,y)+0*f(x,y)+2*f(x+1,y)

+(-1)*f(x-1,y+1)+0*f(x,y+1)+1*f(x+1,y+1)

＝[f(x+1,y-1)+2*f(x+1,y)+f(x+1,y+1)]-[f(x-1,y-1)+2* f(x-1,y)+f(x-1,y+1)] (7)

The vertical gradient of this pixel can be calculated as described in equation (8):

Gy＝1*f(x-1,y-1)+2*f(x,y-1)+1*f(x+1,y-1)

+0*f(x-1,y)0*f(x,y)+0*f(x+1,y)

+(-1)*f(x-1,y+1)+(-2)*f(x,y+1)+(-1)*f(x+1,y+1)

＝[f(x-1,y-1)+2f(x,y-1)+f(x+1,y-1)]-[f(x-1,y+1)+2*f( x,y+1)+f(x+1,y+1)] (8)

Wherein, f is the pixel, x refers to the x coordinate of the pixel f, and y refers to the y coordinate of the pixel f.

Step 1203, calculating the angle correlation value between the gradient direction of the current image of the pixel and the gradient direction of the background image;

In this embodiment, the included angle can be obtained according to the gradient direction of the current image and the gradient direction of the background image, and then the correlation value (such as a cosine value) of the included angle can be calculated, but this embodiment is not limited thereto, for example, other correlation value (such as cotangent value, etc.), the following only takes the cosine value of the included angle as an example for illustration.

Step 1204: Statistically and average the angle correlation values of multiple pixels (for example, all pixels) in the candidate area, and use the average angle correlation value of the candidate area as the average value of the gradient direction information.

Step 1205, judging whether the average value of the gradient direction information is greater than or equal to a preset threshold (the fifth threshold); if the average value of the gradient direction information is greater than or equal to the preset threshold, perform step 1206;

In this embodiment, the specific value of the fifth threshold may be preset based on empirical values, for example, and the present invention is not limited to the fifth threshold.

Step 1206, determine that there is no smoke in the candidate area.

In this embodiment, for example, if the average value of the gradient direction information is greater than or equal to the fifth threshold, it indicates that the candidate area is not a foreground generated by a real moving object, but a false foreground caused by a change in illumination, so the This "candidate region" of the current frame is removed from the "candidate region" list. In addition, when the average value of the gradient direction information is less than the fifth threshold, it may be determined that there is smoke in the candidate area, or in order to make the detection result more accurate, the candidate area may continue to be detected for other items.

The above schematically illustrates how to judge the existence of smoke in a certain candidate area, but the present invention is not limited to Here, for example, other attribute information may also be used for determination. And the above formulas (1) to (8) only schematically illustrate the present invention, but the present invention is not limited thereto, and the above formulas (1) to (8) can be appropriately modified according to the actual situation.

In addition, one or more of the above-mentioned implementation modes 1 to 6 may be used, for example, only one of the above-mentioned implementation modes may be used, or all of the above-mentioned six implementation modes may be used. Moreover, there is no restriction on the order of execution among the above-mentioned embodiments; for example, Embodiments 1 to 6 can be respectively executed according to the smoothness, or Embodiment 2 can be executed after Embodiment 4 is executed, and so on. In actual application, the specific detection scheme can be determined according to the actual situation.

It can be seen from the above embodiments that one or more candidate regions are obtained based on the foreground image, attribute information of a candidate region corresponding to the current image and/or background image is calculated, and whether there is smoke in the candidate region is determined according to the attribute information. Therefore, not only can the smoke be detected quickly and accurately through video images, but also the detection accuracy of video-based smoke detection under illumination changes and complex environments can be improved.

Example 2

An embodiment of the present invention provides a smoke detection device, which corresponds to the smoke detection method described in Embodiment 1, and the same content will not be repeated here.

FIG. 13 is a schematic diagram of a smoke detection device according to an embodiment of the present invention. As shown in FIG. 13 , the smoke detection device 1300 includes:

A background image modeling unit 1301, performing background image modeling on the current image to obtain a foreground image and a background image of the current image;

A candidate area acquiring unit 1302, based on the foreground image, acquires one or more candidate areas for detecting moving objects in the current image;

An attribute information calculation unit 1303, which calculates the attribute information of a candidate area corresponding to the current image and/or the background image; and

The smoke determination unit 1304 determines whether there is smoke in the candidate area according to the attribute information of the candidate area corresponding to the current image and/or the background image.

FIG. 14 is a schematic diagram of a candidate area acquisition unit according to an embodiment of the present invention. As shown in FIG. 14 , the candidate area acquisition unit 1302 may include:

A binarized image acquisition unit 1401, which acquires a binarized image of the foreground image;

Connected domain acquisition unit 1402, using a plurality of pixels with the same pixel value and connected with each other in the binarized image as a connected domain, so as to obtain one or more connected domains representing moving objects in the foreground image;

The connected domain selection unit 1403 selects the connected domains to obtain one or more candidate regions.

Wherein, the connected domain selection unit 1403 may be configured to: remove connected domains whose area is smaller than or equal to a preset threshold, and/or remove connected domains whose average color depth is outside a preset range. But the present invention is not limited thereto, and the connected domains can also be screened according to other rules.

Fig. 15 is another schematic diagram of a smoke detection device according to an embodiment of the present invention. As shown in Fig. 15 , the smoke detection device 1500 includes: a background image modeling unit 1301, a candidate area acquisition unit 1302, an attribute information calculation unit 1303 and a smoke determination unit 1304, as above.

As shown in Figure 15, the smoke detection device 1500 may also include:

The motion area obtaining unit 1501, according to the position of the candidate area in multiple image frames including the current image, obtains the continuous motion area corresponding to the candidate area;

The attribute information calculating unit 1303 may also be configured to: calculate attribute information of the candidate area corresponding to the current image and/or the background image based on the continuous motion area.

In one embodiment, it may be determined whether there is smoke in a candidate area according to whether the main motion direction of the candidate area is downward in multiple image frames.

In this embodiment, the attribute information calculation unit 1303 can also be used to: obtain the main motion direction of the candidate area in multiple image frames; the smoke determination unit 1304 can also be used to: the main motion direction of the candidate area is In the case of the following, it is determined that there is no smoke in the candidate area.

FIG. 16 is a schematic diagram of an attribute information calculation unit according to an embodiment of the present invention. As shown in FIG. 16 , the attribute information calculation unit 1303 may include:

The motion direction calculation unit 1601 calculates the motion direction of the candidate area relative to the current image in multiple image frames based on the position of the center of mass and the position of the center of gravity of the candidate area;

A motion direction statistics unit 1602, which counts the frequency of each motion direction appearing in multiple image frames; and

The main motion direction determining unit 1603 takes the motion direction with the highest occurrence frequency as the main motion direction of the candidate area.

In another embodiment, it may be determined whether there is smoke in a candidate area according to whether the saturation information of the candidate area in the continuous motion area is less than a preset threshold.

Fig. 17 is another schematic diagram of the attribute information calculation unit according to the embodiment of the present invention. As shown in Fig. 17, the attribute information The information calculation unit 133 may include:

The current saturation map acquisition unit 1701 performs color space transformation on the current image, and calculates saturation color components according to the color components to obtain a saturation map of the current image;

The current saturation calculation unit 1702 calculates the current average saturation of the candidate area in the continuous motion area based on the saturation map of the current image.

In this embodiment, the smoke determining unit 1304 may also be configured to: determine that there is no smoke in the candidate area when the current saturation average value is greater than or equal to a preset threshold.

In another embodiment, it may be determined whether there is smoke in a candidate area according to a comparison result of the current saturation information of the candidate area in the continuous motion area and the background saturation information.

Fig. 18 is another schematic diagram of an attribute information calculation unit according to an embodiment of the present invention. As shown in Fig. 18 , the attribute information calculation unit 1303 may include: a current saturation map acquisition unit 1701 and a current saturation calculation unit 1702, as described above.

As shown in Figure 18, the attribute information calculation unit 1303 may also include:

The background saturation map acquisition unit 1801 performs color space transformation on the background image, and calculates saturation color components according to the color components to obtain a saturation map of the background image;

The background saturation calculation unit 1802 calculates the average background saturation of the candidate area in the continuous motion area based on the saturation map of the background image.

In this embodiment, the smoke determination unit 1304 may also be configured to: determine that there is no smoke in the candidate area when the current average saturation value is greater than or equal to the average background saturation value.

In another embodiment, it may be determined whether there is smoke in a candidate area according to the gray variance information of the candidate area in the continuous motion area.

FIG. 19 is another schematic diagram of an attribute information calculation unit according to an embodiment of the present invention. As shown in FIG. 19 , the attribute information calculation unit 1303 may include:

The variance value calculation unit 1901 calculates the gray scale variance value of the candidate area in the continuous motion area based on the gray scale image of the current image.

In this embodiment, the smoke determining unit 1304 may also be configured to: determine that there is no smoke in the candidate area when the gray variance value is greater than or equal to a preset threshold.

In another embodiment, it may be determined whether there is smoke in a candidate area according to gray average information of the candidate area in the continuous motion area.

FIG. 20 is another schematic diagram of an attribute information calculation unit according to an embodiment of the present invention. As shown in FIG. 20 , the attribute information calculation unit 1303 may include:

The motion area adjustment unit 2001 removes the candidate area from the continuous motion area to obtain the remaining motion area;

The current average value calculation unit 2002, based on the grayscale image of the current image, calculates the current grayscale average value of the remaining candidate regions;

The background average calculation unit 2003, based on the grayscale image of the background image, calculates the background grayscale average value of the remaining candidate area; and

The difference calculation unit 2004 calculates the difference between the current grayscale average value and the background grayscale average value.

In this embodiment, the smoke determining unit 1304 may also be configured to: determine that there is no smoke in the candidate area when the difference between the current average gray level and the background gray level is less than or equal to a preset threshold.

In another embodiment, it may be determined whether there is smoke in a candidate area according to the gradient direction information of the candidate area.

In this embodiment, the attribute information calculation unit 1303 can also be used to: calculate the average value of the gradient direction information of the candidate area; the smoke determination unit 1304 can also be used to: when the average value of the gradient direction information is greater than or equal to a preset threshold Next, it is determined that there is no smoke in the candidate area.

FIG. 21 is a schematic diagram of an attribute information calculation unit according to an embodiment of the present invention. As shown in FIG. 21 , the attribute information calculation unit 1303 may include:

The current gradient calculation unit 2101, for a certain pixel in the candidate area, calculate the horizontal gradient and vertical gradient of the pixel based on the grayscale image of the current image to obtain the current image gradient direction of the pixel;

The background gradient calculation unit 2102 calculates the horizontal gradient and vertical gradient of the pixel based on the grayscale image of the background image to obtain the background image gradient direction of the pixel;

The angle correlation value calculation unit 2103, which calculates the angle correlation value between the gradient direction of the current image of the pixel and the gradient direction of the background image; and

The average value gradient acquisition unit 2104 is used to calculate and average the angle correlation values of multiple pixels in the candidate area, and use the average angle correlation value of the candidate area as the average value of the gradient direction information.

In this embodiment, the attribute information may include one or more of the following: saturation information, grayscale variance information, gradient direction information, grayscale average information, and motion direction information. But the present invention is not limited thereto, for example, other attribute information can also be used for judgment. In addition, one or more of the above-mentioned embodiments can be adopted, and can be based on Determine the specific testing plan according to the actual situation.

It can be seen from the above embodiments that one or more candidate regions are obtained based on the foreground image, attribute information of a candidate region corresponding to the current image and/or background image is calculated, and whether there is smoke in the candidate region is determined according to the attribute information. Therefore, not only can the smoke be detected quickly and accurately through video images, but also the detection accuracy of video-based smoke detection under illumination changes and complex environments can be improved.

Example 3

An embodiment of the present invention provides an image processing device, and the image processing device includes the smoke detection device as described in Embodiment 2.

Fig. 22 is a schematic diagram of the image processing apparatus of the embodiment of the present invention. As shown in FIG. 22 , an image processing device 2200 may include: a central processing unit (CPU) 100 and a memory 110 ; the memory 110 is coupled to the central processing unit 100 . Among them, the memory 110 can store various data; in addition, it also stores information processing programs, and executes the programs under the control of the central processing unit 100 .

In one embodiment, the function of the smoke detection device can be integrated into the central processing unit 100 . Wherein, the central processing unit 100 may be configured to control the smoke detection method described in Embodiment 1.

In another embodiment, the smoke detection device can be configured separately from the central processing unit 100, for example, the smoke detection device can be configured as a chip connected to the central processing unit 100, and the function of the smoke detection device can be realized through the control of the central processing unit 100 .

In this embodiment, the central processing unit 100 may be configured to perform the following control:

Perform background image modeling on the current image to obtain the foreground image and background image of the current image; obtain one or more candidate regions in the current image for detecting moving objects based on the foreground image; calculate a candidate region corresponding to the current image and/or or the attribute information of the background image; and determine whether there is smoke in the candidate area according to the attribute information.

Further, the central processing unit 100 may also be configured to perform the following control: acquire the continuous motion area corresponding to the candidate area according to the position of the candidate area in the plurality of image frames respectively; and calculate the candidate area based on the continuous motion area Attribute information corresponding to the current image and/or background image.

In addition, as shown in FIG. 22 , the image processing device 2200 may further include: an input/output (I/O) device 120 and a display 130 ; wherein, the functions of the above components are similar to those of the prior art, and will not be repeated here. It should be noted that the image processing device 2200 does not necessarily include all components shown in FIG. 22; in addition, the image The processing device 2200 may also include components not shown in FIG. 22 , and reference may be made to the prior art.

An embodiment of the present invention provides a computer-readable program, wherein when the program is executed in the image processing device, the program causes the computer to execute the smoke detection method as described in Embodiment 1 in the image processing device.

An embodiment of the present invention provides a storage medium storing a computer-readable program, wherein the computer-readable program enables a computer to execute the smoke detection method as described in Embodiment 1 in an image processing device.

The above devices and methods of the present invention can be implemented by hardware, or by combining hardware and software. The present invention relates to such a computer-readable program that, when the program is executed by a logic component, enables the logic component to realize the above-mentioned device or constituent component, or enables the logic component to realize the above-mentioned various methods or steps. The present invention also relates to a storage medium for storing the above program, such as hard disk, magnetic disk, optical disk, DVD, flash memory and the like.

One or more of the functional blocks described in the accompanying drawings and/or one or more combinations of the functional blocks can be implemented as a general-purpose processor, a digital signal processor (DSP) for performing the functions described in this application ), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any suitable combination thereof. One or more of the functional blocks described in the drawings and/or one or more combinations of the functional blocks can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors processor, one or more microprocessors in communication with a DSP, or any other such configuration.

The present invention has been described above in conjunction with specific embodiments, but those skilled in the art should be clear that these descriptions are all exemplary and not limiting the protection scope of the present invention. Those skilled in the art can make various variations and modifications to the present invention according to the principles of the present invention, and these variations and modifications are also within the scope of the present invention.

Claims (20)

A smoke detection device, wherein the smoke detection device includes: a background image modeling unit, which performs background image modeling on the current image to obtain a foreground image and a background image of the current image; a candidate region acquiring unit, which acquires one or more candidate regions for detecting moving objects in the current image based on the foreground image; an attribute information calculation unit, which calculates attribute information of a candidate area corresponding to the current image and/or the background image; and A smoke determining unit, which determines whether there is smoke in the certain candidate area according to the attribute information. The smoke detection device according to claim 1, wherein the smoke detection device further comprises: A motion area acquisition unit, which acquires the continuous motion area corresponding to the certain candidate area according to the positions of the certain candidate area in the plurality of image frames respectively; The attribute information calculation unit is further configured to: calculate the attribute information of the certain candidate area corresponding to the current image and/or the background image based on the continuous motion area. The smoke detection device according to claim 2, wherein the attribute information calculation unit comprises: A current saturation map acquisition unit, which performs color space transformation on the current image, and calculates saturation color components according to the color components to obtain a saturation map of the current image; A current saturation calculation unit, which calculates the current saturation average value of the certain candidate area in the continuous motion area based on the saturation map of the current image. The smoke detection device according to claim 3, wherein the smoke determination unit is further configured to: determine that there is no smoke. The smoke detection device according to claim 3, wherein the attribute information calculation unit further comprises: A background saturation map acquisition unit, which performs color space transformation on the background image, and calculates saturation color components according to the color components to obtain a saturation map of the background image; A background saturation calculation unit, which calculates the average background saturation of the certain candidate area in the continuous motion area based on the saturation map of the background image. The smoke detection method according to claim 5, wherein the smoke determination unit is further used for: In a case where the current average saturation value is greater than or equal to the background saturation average value, it is determined that there is no smoke in the certain candidate area. The smoke detection device according to claim 2, wherein the attribute information calculation unit comprises: A variance value calculation unit, which calculates the gray scale variance value of the certain candidate area in the continuous motion area based on the gray scale image of the current image. The smoke detection device according to claim 7, wherein the smoke determination unit is further configured to: determine that there is no smoke. The smoke detection device according to claim 2, wherein the attribute information calculation unit comprises: a motion area adjustment unit, which removes the certain candidate area from the continuous motion area to obtain the remaining motion area; a current average calculation unit, which calculates the current gray average of the remaining candidate regions based on the grayscale image of the current image; a background average calculation unit, which calculates the background gray average of the remaining candidate regions based on the gray image of the background image; and A difference calculation unit, which calculates the difference between the current average gray level and the average background gray level. The smoke detection device according to claim 9, wherein the smoke determination unit is further configured to: determine that there is no smoke in the certain candidate area when the difference is less than or equal to a preset threshold. The smoke detection device according to claim 1, wherein the attribute information calculation unit comprises: The current gradient calculation unit, for a certain pixel in the certain candidate area, calculates the horizontal gradient and vertical gradient of the certain pixel based on the grayscale image of the current image to obtain the current image of the certain pixel gradient direction; A background gradient calculation unit, which calculates the horizontal gradient and vertical gradient of the certain pixel based on the grayscale image of the background image to obtain the background image gradient direction of the certain pixel; An included angle correlation value calculation unit, which calculates an included angle correlation value between the gradient direction of the current image and the gradient direction of the background image of the certain pixel; and A gradient mean value acquisition unit, which counts and averages the angle correlation values of multiple pixels in the certain candidate area, and takes the average angle correlation value of the certain candidate area as the average value of the gradient direction information. The smoke detection device according to claim 11, wherein the smoke determination unit is further used for: In a case where the average value of the gradient direction information is greater than or equal to a preset threshold, it is determined that there is no smoke in the certain candidate area. The smoke detection device according to claim 1, wherein the candidate area acquisition unit comprises: a binarized image acquisition unit, which acquires a binarized image of the foreground image; A connected domain acquisition unit, which uses a plurality of pixels in the binarized image that have the same pixel value and are connected to each other as a connected domain, so as to obtain one or more connected domains representing moving objects in the foreground image; A connected domain selection unit, which selects the connected domains to obtain the one or more candidate regions. The smoke detection device according to claim 13, wherein the connected domain selection unit is configured to: remove the connected domains whose area is less than or equal to a preset threshold, and/or remove the average color depth outside the preset range The connected domain of . The smoke detection device according to claim 1, wherein the attribute information calculation unit comprises: A motion direction calculation unit, which calculates the motion direction of the certain candidate area relative to the current image in the plurality of image frames based on the position of the centroid and the position of the center of gravity of the certain candidate area; a motion direction statistics unit, which counts the frequency of occurrence of each motion direction in the plurality of image frames; and A main motion direction determining unit, which determines the motion direction with the highest frequency of occurrence as the main motion direction of the certain candidate area. The smoke detection device according to claim 15, wherein the smoke determination unit is further configured to: determine that there is no smoke. A smoke detection method, wherein the smoke detection method comprises: performing background image modeling on the current image to obtain a foreground image and a background image of the current image; Acquiring one or more candidate regions for detecting moving objects in the current image based on the foreground image; calculating attribute information of a candidate area corresponding to the current image and/or the background image; and Determine whether there is smoke in the certain candidate area according to the attribute information. The smoke detection method according to claim 17, wherein the method further comprises: Acquiring a continuous motion region corresponding to the certain candidate region according to the respective positions of the certain candidate region in multiple image frames; And, based on the continuous motion area, calculating that the certain candidate area corresponds to the current image and/or Attribute information of the background image. The smoke detection method according to claim 18, wherein the attribute information includes one or more of the following: saturation information, grayscale variance information, gradient direction information, grayscale average information, and motion direction information. An image processing device, wherein the image processing device comprises the smoke detection device according to claim 1.