CN110648490A - Multi-factor flame identification method suitable for embedded platform - Google Patents

Abstract

The present application discloses a multi-factor flame identification method suitable for an embedded platform, which includes establishing a fire sample library, and the fire sample library comes from network fire pictures and combustion experiment pictures; respectively acquiring multiple live video frames; One or more quasi-fire areas are obtained from the moving objects in the video frame; fire confirmation is performed on one or more quasi-fire areas respectively according to the fire sample library, and it is judged whether there is fire information in one or more quasi-fire areas; The information is divided into fire grades, and corresponding alarm signals are generated according to the divided fire grades. In this application, by separately processing the acquired live video frames, it analyzes in real time whether there is fire information that may develop into a fire in the on-site monitoring environment, and confirms the fire again to further determine whether there is real fire information. By analyzing and generating alarms, fire information can be identified more accurately.

Description

A multi-factor flame identification method suitable for embedded platforms

technical field

The present application relates to the technical field of electronic intelligent fire protection, and in particular, to a multi-factor flame identification method suitable for embedded platforms.

Background technique

At present, there are three mainstream methods of fire identification. The first one uses traditional fire detection sensors to detect fire information, which generally has the disadvantage of long detection time and low accuracy. The second is to use image recognition to detect fire information, that is, to use traditional digital image processing methods to manually set the feature dimensions of fire for fire identification, that is, to manually design multiple representative features to represent fire information. However, due to the limited characteristics of artificially representing fires, fire information in different scenes or backgrounds cannot be expressed normally, and there are generally disadvantages of high misjudgment rate and low robustness. The third method is to use deep learning to automatically learn the fire features in the sample pictures. By automatically learning the fire features in the sample pictures instead of manually designing the fire feature dimension, the robustness and accuracy are improved. As shown in Figure 1, it is a diagram of a traditional fire identification system. When the general deep learning method is used for fire identification, the live videos collected by multiple video acquisition terminals are uniformly transmitted to the background server through the switch, and the background server performs centralized calculation. , resulting in a huge amount of computation.

SUMMARY OF THE INVENTION

In view of the deficiencies of the prior art, the present application provides a multi-factor flame identification method suitable for embedded platforms.

A multi-factor flame identification method applicable to an embedded platform disclosed in this application includes:

Establish a fire sample library, which comes from network fire pictures and combustion experiment pictures;

Obtain multiple live video frames respectively;

Extract the moving objects in each live video frame separately to obtain one or more quasi-fire areas;

Carry out fire confirmation on one or more quasi-fire areas according to the fire sample database, and determine whether there is fire information in one or more quasi-fire areas;

If so, the fire information is divided into fire grades, and a corresponding alarm signal is generated according to the divided fire grades.

According to an embodiment of the present application, performing fire confirmation on one or more quasi-fire areas according to the fire sample library, and determining whether there is fire information in the one or more quasi-fire areas includes:

The BP neural network algorithm, SSD algorithm and Yolo algorithm are respectively used to detect one or more quasi-fire areas. When the BP neural network algorithm is used to detect the fire area, the network fire pictures and combustion experiment pictures in the fire sample library are extracted. Constitute the BP neural network training set;

Output the fire confidence according to the detection;

Determine whether there is fire information according to the fire confidence.

According to an embodiment of the present application, when the BP neural network algorithm is used to detect one or more quasi-fire areas, at least half of the network fire pictures and at least half of the combustion experiment pictures contained in the fire sample library are extracted. BP neural network training set.

According to an embodiment of the present application, judging whether there is fire information according to the fire confidence level includes: when the BP neural network algorithm is used for detection, outputting the BP network fire confidence level P, P∈[0,1], and judging whether or not according to the fire confidence level P Fire information exists.

According to an embodiment of the present application, the SSD algorithm and the Yolo algorithm are respectively used to detect the fire information, and the SSD fire confidence level P_A and the Yolo fire confidence level are respectively output as P_B. If P>0.8 and P_A>0.8, the fire information is Big fire information; if P>0.6 and P_B>0.7, the fire information is small fire information.

According to an embodiment of the present application, classifying the fire information into a fire level and generating a corresponding alarm signal according to the classified fire level includes: if the fire information is determined to be a large fire information, acquiring a live video of more than 30 consecutive frames but less than 60 frames If there is a large fire information in the live video frames of more than 30 consecutive frames but less than 60 frames, the fire information is classified as a first-level fire warning, and a first-level alarm signal is generated.

According to an embodiment of the present application, classifying the fire information according to the fire level and generating the corresponding alarm signal according to the classified fire level includes: classifying the fire information according to the fire level, and generating the corresponding alarm signal according to the classified fire level includes: If it is judged that the fire information is the big fire information, obtain the live video frames with more than 60 frames and less than 90 frames in a row. When there is large fire information in the live video frames with more than 60 frames and less than 90 frames in a row, the fire information is classified as secondary fire. Early warning, generating a secondary alarm signal.

According to an embodiment of the present application, classifying the fire information according to the fire level and generating the corresponding alarm signal according to the classified fire level includes: classifying the fire information according to the fire level, and generating the corresponding alarm signal according to the classified fire level includes: If it is judged that the fire information is large fire information, then obtain more than 90 consecutive live video frames. When large fire information appears in the continuous live video frames of more than 90 consecutive frames, the fire information is a three-level fire warning, and a three-level alarm signal is generated.

According to an embodiment of the present application, classifying the fire information by fire level, and generating a corresponding alarm signal according to the divided fire level includes: if the fire information is determined to be small fire information, acquiring a live video of more than 15 frames and less than 30 frames in a row If there is small fire information in the live video frames of more than 15 consecutive frames but less than 30 frames, the fire information is a level 0 fire warning, and a level 0 alarm signal is generated.

According to an embodiment of the present application, extracting a moving object in a video frame to obtain a quasi-fire area includes:

The background modeling of the acquired live video frames is carried out by using a mixture of Gaussian model modeling method;

Update the parameters in the mixed Gaussian model to obtain the background image;

The live video frame is subtracted from the obtained background image, and the moving objects in the live video frame are extracted to obtain the quasi-fire area.

In the multi-factor flame identification method applicable to the embedded platform of the present application, after acquiring multiple live video frames, each live video frame is processed separately, so as to extract the moving objects in each live video frame respectively. , and then carry out the fire confirmation separately, that is, adopt the distributed processing method to avoid the huge amount of calculation caused by the centralized processing. At the same time, by processing the acquired live video frames, we can analyze in real time whether there is fire information that may develop into a fire in the monitoring environment of the site, obtain the quasi-fire area, and conduct fire confirmation at the fire area, and further determine whether there is a fire in the quasi-fire area. If it is determined that there is fire information, a fire level analysis will be performed on the fire information and an alarm will be generated concurrently. Moreover, when the method is applied to the existing fire intelligent alarm system, it can solve the problem of fire information detection in a fire scene with a large space in combination with the fire intelligent alarm system, increase the scope of fire scene detection, and compare with traditional sensor type fires. Detector, the method has shorter detection time and higher accuracy, and at the same time, the acquired live video frames can also be stored, which is convenient for subsequent investigation and evidence collection of the fire scene.

Description of drawings

The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The schematic embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the attached image:

Figure 1 is a diagram of an existing fire identification system;

2 is a diagram of a multi-factor flame identification system for an embedded platform in an embodiment;

Fig. 3 is the fire information identification flow chart in the embodiment;

Fig. 4 is the image of the standard normal distribution in the embodiment;

5 is a schematic diagram of a process of extracting a moving target from a current live video frame using a Gaussian mixture model in an embodiment;

FIG. 6 is a flow chart of fire classification in the embodiment.

Detailed ways

Various embodiments of the present application will be disclosed in the drawings below, and for the sake of clarity, many practical details will be described together in the following description. It should be understood, however, that these practical details should not be used to limit this application. That is, in some embodiments of the present application, these practical details are unnecessary. In addition, for the purpose of simplifying the drawings, some well-known structures and components will be shown in a simple schematic manner in the drawings.

In addition, the descriptions such as "first", "second", etc. in this application are only for the purpose of description, and do not specifically refer to the order or order, nor are they used to limit the application, but are only for the purpose of distinguishing The components or operations are described by the same technical terms, and should not be construed as indicating or implying their relative importance or implying the quantity of the indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In addition, the technical solutions between the various embodiments can be combined with each other, but must be based on the realization by those of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be realized, it should be considered that the combination of such technical solutions does not exist. , is not within the scope of protection claimed in this application.

The present application provides a multi-factor flame identification method suitable for embedded platforms, including four stages, the first stage is the construction stage of the fire sample library, the second stage is the moving target extraction stage, and the third stage is the fire area Confirmation stage, the fourth stage is the fire grade classification and early warning stage. At the same time, this application also specifically provides how to transplant the multi-factor flame identification method to the embedded platform, so that the multi-factor flame identification method can be run on a chip similar to the Hi 3519A series of HiSilicon. The multi-factor flame identification method applicable to the embedded platform of the present application will be described in detail below.

As shown in Figure 2, it is a diagram of a multi-factor flame recognition system suitable for an embedded platform in the present application. The multi-factor flame recognition system suitable for an embedded platform in the present application includes a plurality of video collection terminals, a plurality of algorithm boxes , switch, server, display terminal and alarm terminal, each video acquisition terminal is connected to an algorithm box, multiple algorithm boxes are connected to the switch, and the switch, display terminal and alarm terminal are respectively connected to the server. Among them, multiple video acquisition terminals are respectively installed in multiple monitored areas to collect live video frames of each monitored area, and multiple video acquisition terminals respectively transmit the collected live video frames to each corresponding algorithm box. A corresponding algorithm box completes the extraction of the moving target area in the live video frame according to the received live video frame, and obtains the quasi-fire area. Each algorithm box transmits the quasi-fire area to the server. A fire sample library is established in the server. The established fire sample library trains and tests the pictures in the fire sample library to obtain an algorithm model, so that according to the algorithm model obtained by training and testing, the fire area can be checked for fire, and whether there is fire information is judged. The information is divided into fire grades, and the alarm terminal is controlled to generate corresponding alarm signals according to the divided fire grades. The multi-factor flame identification method applicable to the embedded platform of the present application is described in detail below.

In the multi-factor flame identification method applicable to the embedded platform of the present application, before fire identification, a fire sample library is established, so that the pictures in the fire sample library can be trained and tested in the BP neural network algorithm, and the BP neural network can be obtained. Algorithmic model. In the construction of the fire sample database, in order to make the confirmation of fire information more accurate, in this example, a combination of network virtual pictures and real life fire pictures is used to build a fire sample database, so that the established fire sample database has both sources from the network. The network fire pictures of , and also the burning experiment pictures obtained through burning experiments in real life. For network fire pictures and burning experiment pictures, they are collected through the network and artificial simulation respectively, and the total number of pictures in the fire sample library is at least 100,000. Among them, the network fire pictures are crawled by crawlers on two search engines, Baidu and Google, and downloaded from international open source databases and screened 10,000 pieces. The acquisition of network fire pictures is as rich as possible, including fire pictures obtained in different scenarios. A total of 50,000 network pictures are formed. When obtaining the pictures of the combustion experiment, in order to make the pictures in the fire sample library more comprehensive, rich and real, we combine different scenes such as indoor and outdoor, combustion of different materials such as beech wood, plastic, waste paper, fabric and natural gas, and different disturbances such as sunlight, incandescent Lamps, mosquito coils, cigarettes and yellow/red objects are fully combined with common scenes in real life when considering the scene, and through a large number of burning experiments, 50,000 sample pictures of real scenes have been collected to form burning experiment pictures, so that the obtained Realistic pictures of combustion experiments. Network fire pictures and combustion experiment pictures together form a fire sample library of no less than 100,000 pieces. As shown in Table 1 below, it is the composition of the pictures in the fire sample library.

Table 1 Source of pictures from the fire sample library

经过运动目标的提取，可以筛选出准火灾区域，便于对准火灾区域进行二次确认，将运动状态中的部分背景区域剔除以获得真实的火灾区域。2、从现场视频帧中提取准火灾区域后，将该准火灾区域作为研究对象，减少了现场视频帧中对应的图像的像素点，从而减少了需要运算的区域，对提升算法的性能有很大的作用，可以减少运算量。After completing the construction of the fire sample database and the testing and training of the pictures in the fire sample database, after obtaining the BP neural network algorithm model, the fire information identification begins. Please refer to FIG. 3 , which is a flowchart of fire information identification. When multiple video capture terminals collect live video frames and transmit them to each algorithm box that is connected to the corresponding communication, each algorithm box obtains the corresponding live video frames, and extracts the moving objects in the live video frames to obtain quasi-fire area. To identify the fire information in the live video frame, it is necessary to extract the fire area, that is, the fire area. After the fire occurs, due to the development of the fire and the effect of the ambient air flow, the fire area and the background image will continue to move. Therefore, to identify the fire information, after acquiring the live video frame, it is first necessary to convert the moving target in the live video frame. Extracted, the obtained moving targets are composed of quasi-fire areas, and because there are a large number of non-fire moving targets in the quasi-fire area besides the fire area, to accurately confirm the fire information, it is necessary to If the fire moving target is eliminated, the fire confirmation can be carried out, and then it can be judged whether there is fire information in the quasi-fire area. In this example, the reason why the moving objects in the live video frames are extracted first, and then the fire confirmation is performed after obtaining the quasi-fire area is mainly based on the following two considerations: 1. In the process of fire development, the fire area and some background areas All must be in a state of motion. Assuming that the quasi-fire area composed of moving objects extracted from the live video frame is M, the quasi-fire area must include the real fire area and some moving background areas, then M contains N, that is,

After the extraction of the moving target, the quasi-fire area can be screened out, which is convenient for the secondary confirmation of the fire area, and some background areas in the moving state are eliminated to obtain the real fire area. 2. After the quasi-fire area is extracted from the live video frame, the quasi-fire area is taken as the research object, and the corresponding image pixels in the live video frame are reduced, thereby reducing the area that needs to be calculated, which greatly improves the performance of the algorithm. It has a large effect and can reduce the amount of calculation.

How to extract moving objects from live video frames, there are two commonly used specific methods, the first type is the difference method, specifically the background difference method and the inter-frame difference method, these two methods are for different two frames of live video. The frame is subtracted, and the differenced live video frame is used as the moving target. The difference is that the inter-frame difference method is to differentiate the adjacent live video frames, and the background difference method is to differentiate the current live video frame and the background image. The establishment of this visible background image directly affects the extraction of moving objects. The establishment of the background image is generally divided into two categories. The first category is to fix the background image and obtain the moving target by taking the difference between the current live video frame and the background image. frame as a background image. However, in practice, the background image usually changes. For example, the moving target is the object in the original background image. If the background image remains unchanged, the moving target will be treated as the background, resulting in the extracted moving target being unsatisfactory. For example, in real life, affected by natural factors (such as light brightness, natural wind, etc.), the background will change slowly, and the background image will naturally follow the changes. If the background image remains unchanged, the error with the actual background image will be It gradually becomes larger, which will also cause a great error to the extracted moving target.

The second method of establishing the background is that the background image can change slowly with the change of the environment, so that it can keep a small error with the actual background environment. In order to obtain a background image with adaptive capability, a background modeling algorithm is usually used. The commonly used modeling algorithms can be roughly divided into two types. One is to store the live video frames before the current moment, and then to store these live video The newly appeared data in the frame is used as samples, and then these samples are added to the background image according to certain rules, such as the median background modeling method and the average background modeling method. The pixel value of the corresponding position in the video frame is the median value, and this median value is used as the pixel value of the corresponding position of the background image at this time, and the average background modeling method is to average the pixel values of the corresponding positions in each live video frame. And use this average value as the background of the current live video frame, this method is ideal, but because the live video frame stored for a period of time is used as a sample, it increases the burden on the server memory and increases the amount of data calculation. Hardware requirements are relatively high. The average background modeling method overcomes these shortcomings. It does not need to store the live video frame as a sample, but changes the original background image according to the current live video frame through regression, such as the Kaleman filter model. Gaussian and mixture Gaussian models. After repeated experiments and comparisons, this example uses a Gaussian mixture model. The following describes in detail how to use the Gaussian mixture model to extract moving target areas to obtain quasi-fire areas.

In this example, extracting the moving objects in the video frame to obtain the quasi-fire area includes using the mixed Gaussian model modeling method to model the background of the obtained live video frame; updating the parameters in the mixed Gaussian model to obtain the background image; The moving objects in the live video frame are extracted by subtraction from the obtained background image to obtain the quasi-fire area.

In the Gaussian distribution, if the random variable X obeys a Gaussian distribution with a mathematical expectation of μ and a variance of σ^2, denoted as N(μ, σ^2), its probability density function is the expected value of the normal distribution μ determines its position, Its standard deviation σ determines the magnitude of its distribution. What we usually call the standard normal distribution is the normal distribution with μ=0 and σ=1. Figure 4 is an image of a standard normal distribution.

When there is no moving target in the environment, the pixel values at the same position at different times are counted, and it can be found that these values are in a single Gaussian distribution, but the actual environment is usually affected by external factors such as light and wind. Influence, a single Gaussian distribution can no longer satisfy the distribution of pixel values, so several Gaussian distributions can be combined with different weights to describe the statistics of a pixel value at a location, which is the mixture Gaussian model mentioned in this example. The more the number of Gaussian models, the more complex the background can be described, and the higher the accuracy, but the price in exchange is the greater the amount of data computation. In order to achieve satisfactory results and take into account the requirements for computer hardware, the number of Gaussian models in general engineering is appropriate to take 3 to 5.

其中W_jt表示的是第i个像素在t时刻的第j个高斯模型的权值，W的值越大说明该高斯模型与当前图像的像素值越接近，k表示的是采用的高斯模型的个数，并且有

即用于模拟一个像素的所有高斯模型的权值之和为1。

表示的是用于描述第i个像素点在t时刻的第j个高斯模型，

表示的是单个高斯模型。而u_it表示的是该高斯模型的均值，表示的是该高斯模型的方差，在混合高斯模型的建模算法中，主要是通过调整均值和方差的值来达到需要的效果，因些在混合高斯模型的建模算法中，高斯模型的均值以及方差这两个参数的更新方法非常的重要，具体的更新方法后续会介绍。利用混合高斯模型进行背景建模的时候需要根据高斯模型与当前像素之间的相似程序对用于描述同一个像素点的k个单高斯模型进行排序，权值W越大就表示该模型与当前像素的相似度比较高，而

越小就表示该组像素点的变化比较小，比较平稳。所以可以根据

的值来描述这种相似度，

的值越大就表示相似度越高，越有可能是属于背景图像的像素点。把各个高斯模型按照

值从大到小排好序，通常来说运动目标与高斯模型的相似程度都比较小，而背景像素点因为变化小，相似程度比较大。因此可以定义一个阈值T，如果前d个高斯模型的权值之和刚好大于或者等于T，则前d个高斯模型用作背景子集，而剩下的k-d个高斯模型作为前景运动子集。T的取值会直接影响到提取运动前景的效果，当T的取值较小时，则d的值就越小，用于描述背景图像的子集也就越单一，所以一般T的值取0.75。Assuming that the pixel value of pixel i in the image at time t is x _it , its probability density function is:

Among them, W _jt represents the weight of the jth Gaussian model of the ith pixel at time t. The larger the value of W, the closer the Gaussian model is to the pixel value of the current image, and k represents the Gaussian model used. number, and there are

That is, the sum of the weights of all Gaussian models used to simulate a pixel is 1.

Represents the j-th Gaussian model used to describe the i-th pixel at time t,

Represents a single Gaussian model. And u _it represents the mean of the Gaussian model, Represents the variance of the Gaussian model. In the modeling algorithm of the Gaussian mixture model, the desired effect is mainly achieved by adjusting the values of the mean and variance. Therefore, in the modeling algorithm of the Gaussian mixture model, the mean value of the Gaussian model is And the update method of the two parameters of variance is very important, and the specific update method will be introduced later. When using the mixture Gaussian model for background modeling, it is necessary to sort the k single Gaussian models used to describe the same pixel point according to the similar program between the Gaussian model and the current pixel. The similarity of pixels is relatively high, while

The smaller the value, the smaller the change of the group of pixels, and the more stable it is. So according to

to describe this similarity,

The larger the value of , the higher the similarity, and the more likely it is a pixel that belongs to the background image. Put each Gaussian model according to

The values are sorted from large to small. Generally speaking, the similarity between the moving target and the Gaussian model is relatively small, and the background pixels have a relatively large similarity because of the small change. Therefore, a threshold T can be defined. If the sum of the weights of the first d Gaussian models is just greater than or equal to T, the first d Gaussian models are used as the background subset, and the remaining kd Gaussian models are used as the foreground motion subset. The value of T will directly affect the effect of extracting the motion foreground. When the value of T is small, the value of d will be smaller, and the subset used to describe the background image will be more single, so the value of T generally takes 0.75 .

就认为该像素与该模型相匹配(为匹配阈值，一般取2.5)。如果X_it与第i个高斯模型匹配，那么该高斯模型的参数就要被更新，更新的方程如下：Next, the update method of each parameter of the mixture Gaussian model will be introduced in detail, so that the background image can be accurately identified according to the update. Before updating each parameter, it is necessary to determine which Gaussian model the pixel is most similar to, generally if the pixel x _it satisfies

It is considered that the pixel matches the model (for the matching threshold, generally 2.5). If X _it matches the ith Gaussian model, then the parameters of the Gaussian model will be updated, and the updated equation is as follows:

Wi _,t+1 =(1-α)wi _,t +αM _it

p _it =αN(x _it , u _it , σ ² )

u _{i, t+1} = (1-p _it )u _it +p _it u _it

Except that the parameters of the Gaussian model need to be updated, the other Gaussian models remain unchanged. Although the mixed Gaussian model is complex and the amount of calculation is relatively large, the effect of the extracted moving objects is better, so it is widely used, as shown in Figure 5, which is the process of extracting moving objects from the current live video frame by using the mixed Gaussian model Schematic. The background of the sample live video frame is modeled according to the method of mixed Gaussian background modeling, and the current scene video frame is subtracted from the current background image to obtain the foreground of the moving target. That is, by subtracting the live video frame from the obtained background image, the moving target in the live video frame can be extracted to obtain a quasi-fire area. After the live video frame is obtained, the background image is obtained by modeling according to the mixed Gaussian model, and the moving object in the current live video frame is obtained by subtracting the live video frame and the background image.

Please review Figure 3. After obtaining the quasi-fire area, it is necessary to conduct fire confirmation on the fire area to determine whether there is fire information in the quasi-fire area, that is, fire area confirmation. Do a second confirmation. In this example, a total of three artificial intelligence algorithms are used for common confirmation at this stage, namely the BP neural network algorithm based on artificial feature engineering, the SSD algorithm based on the deep convolutional neural network, and the Yolo algorithm based on the deep learning convolutional neural network. Whether there is fire information in the current live video frame can be obtained by confirming that the fire area is aligned. That is to say, in this example, the BP neural network algorithm, the SSD algorithm and the Yolo algorithm are respectively used to detect one or more quasi-fire areas. When the BP neural network algorithm is used to detect the fire area, the network fire in the fire sample library is extracted. The pictures and the pictures of the combustion experiment constitute the BP neural network training set; according to the detection, the fire confidence is output respectively; according to the fire confidence, it is judged whether there is fire information. The following describes in detail how to use BP neural network algorithm, SSD algorithm and Yolo algorithm for fire confirmation.

A) The BP neural network algorithm based on artificial feature engineering extracts three features of the regional curvature of the flame, the regional diffusion rate of the flame and the rate of change of the sharp angle of the flame, and these three features are used as the input of the BP neural network algorithm. As shown in Table 2, the training set of the BP neural network randomly selects at least 50% of the pictures from the fire sample library composed of the network fire pictures and the burning experiment pictures. When at least 50% of the pictures are selected, at least half of the network The fire pictures and at least half of the burning experiment pictures are composed of the BP neural network training set. As for the BP neural network algorithm, a four-layer BP neural network is designed, the input layer is 3 units, there are two hidden layers (each layer has 10 units), and the output layer is a unit, which outputs the BP network fire confidence. P, P∈[0,1], P=0 means no fire information, P=1 means there is fire information, the larger the P is, the higher the confidence level is. After a lot of engineering experiments, the fire confidence level output by the BP neural network P has a relatively small probability of missed judgment, and the probability of missed judgment is stable at 0.01%; however, there will be misjudgments, and the probability of misjudgment is stable at 2%. In this application, the judgment of the BP neural network algorithm is only used as one of the factors, and the advantage of the extremely small probability of missed judgment is used as an auxiliary judgment module.

Table 2 Neural network training set composition B) Large target detection based on SSD algorithm and small target detection based on Yolo algorithm, SSD algorithm and Yolo algorithm both belong to target detection algorithm based on deep convolutional neural network, in this application, after a large number of The comparison experiment shows that the SSD algorithm is more sensitive to large fire targets, but it is easy to ignore small fire targets (flames in the smoldering stage or the fire stage); while the Yolo algorithm is just the opposite, more sensitive to small fire targets, and more sensitive to large fire targets. Ease of misjudgment. This application uses the SSD algorithm for large target detection and the Yolo algorithm for small target detection. Finally, the SSD fire confidence P_A and Yolo fire confidence P_B are respectively integrated with the network fire confidence. When P>0.8 and P_A>0.8 In the case of P>0.6 and P_B>0.7, it is concluded that the information of a small fire is generated, that is, the fire is in the stage of fire. Among them, the threshold value of P is related to the two variables P_A and P_B. When it is used for judging large fire information, the threshold value of P is 0.8, and when it is used for judging small fire information, the threshold value of P is 0.6. In addition, P_A=0.8, P_B= 0.7 are based on engineering experiments.

After the confirmation of the fire information is completed, the fire information is classified into fire grades, and corresponding alarm signals are generated according to the fire grades. In this example, BP neural network algorithm, SSD algorithm and Yolo algorithm are used to integrate multiple factors to jointly decide whether there is fire information. And the fire information is distinguished and identified, the big fire target and the small fire target are divided, and the big fire target and the small fire target are further classified at this stage.

As shown in Figure 5, it is a flow chart of fire level division. In this example, four fire levels are designed. If the fire information is judged to be small fire information, the live video frames of more than 15 consecutive frames and below 30 frames are obtained. If there is small fire information in the above 30 or less live video frames, a level 0 fire warning will be generated. This level is in the stage of fire and has not caused any loss of property and personnel. When the fire information is judged to be large fire information, the continuous For live video frames of more than 30 frames but less than 60 frames, when large fire information is generated in the continuous live video frames of more than 30 frames but less than 60 frames, a first-level fire warning will be generated, and this level is in the early stage of fire development. When it is judged that the fire information is the big fire information, the on-site video frames with more than 60 frames and less than 90 frames in a row are obtained. The level is in the rapid development period of the fire; when it is judged that the fire information is a large fire information, it will obtain more than 90 consecutive live video frames. Development to this level indicates that a certain degree of property damage has been caused, and emergency fire rescue is required.

When transplanting the multi-factor flame identification method suitable for embedded platform of the present application to the Hi3519A series chip of HiSilicon for operation, the transplanting steps include building NFS, connecting the HiSilicon development board by serial port, installing the HiSilicon cross compiler, opencv3 .4.5 Cross-compilation, ncnn cross-compilation and project project compilation, output static library based on arm framework. Building NFS includes installing NFS services, writing configuration files, and restarting NFS services. Connecting the HiSilicon development board by serial port includes installing the driver manually or automatically after connecting the serial port cable to the HiSilicon development board, viewing the port number in the way of Computer-Management-Device Manager-Port, and using SecureCRT to access the serial port on the PC. The rate is 115200, and the HiSilicon development board is connected to the virtual machine to share the directory. If the HiSilicon development board does not have an IP assigned, it can be configured manually. When installing the HiSilicon cross-compiler, the host is 64-bit, and the cross-compiler is for 32-bit development boards. You need to supplement the dependency package, and then directly execute the installation script under the installation package, and then test whether the installation is successful.

Compared with traditional fire identification, the multi-factor flame identification method suitable for embedded platform in this example has the following advantages:

1) When the BP neural network algorithm is used for fire information recognition, the number of pictures in the fire sample library of the BP neural network training set reaches 100,000. The pictures have network pictures, and the real scene data accounts for 50%. The fire image sample library currently used in the existing solutions is usually only crawled from the Internet. The fire image sample scenes crawled on the Internet are relatively simple, most of which are serious fire scenes, and lack sample data of the fire stage or smoldering stage. And the number of picture libraries is relatively small (within 10,000 pictures), which is not comprehensive enough for the identification of algorithm models or the research on fire characteristics, resulting in relatively weak algorithm robustness and migration ability, and the performance in the test set is very high. Good, but the recognition rate of real scenes is low. In this application, images are collected from the Internet, international open source libraries, and combustion experiments to build a huge ideal training set covering common life scenarios, which provides an important guarantee for the training effect of the algorithm.

2) Combined with the BP neural network algorithm based on artificially designed feature engineering and the SSD algorithm and YOLO algorithm of the deep convolutional neural network, a multi-factor decision-making scheme is formed to jointly judge fire information, with strong anti-interference ability and strong robustness. At present, the mainstream solutions use traditional digital image processing solutions, and the feature dimensions manually set in the feature engineering process are difficult to characterize the characteristics of all fires. For example, the diffusion characteristics of flames vary greatly in different development stages of the fire, resulting in the anti-interference ability of the algorithm. Relatively weak, easily affected by strong light, weak light, especially night light. The multi-factor flame identification method suitable for embedded platforms of the present application integrates traditional digital image processing schemes and deep learning-based target detection schemes to form a multi-factor fire identification scheme, and designs different algorithms for large fire targets and small fire targets. Scenario-based recognition greatly improves the anti-interference ability of fire recognition, and the accuracy rate is also greatly guaranteed. Experiments show that the accuracy rate can be stabilized at 99.5%.

3) The algorithm of the present application has low complexity, reduces consumption of computing resources, and is suitable for running on a terminal platform. Compared with traditional digital image processing algorithms, the accuracy of deep learning fire detection algorithms has made great progress, but this improvement in accuracy is achieved at the expense of performance. Currently, the mainstream deep learning fire detection models use general-purpose models. The target detection algorithm is trained. The number of neurons in this algorithm is huge, and it usually needs to be recognized in the background on a high-performance CPU server or GPU server. It is necessary to connect the video streams of multiple cameras to the same algorithm server for unified processing. In this way, the server processes multiple video signals at the same time, resulting in huge computational pressure and high hardware costs. Second, the server processing time and signal transmission time are time-consuming, resulting in poor real-time performance. The present application firstly reduces the live video frame, extracts the moving target, and obtains the quasi-fire area, thereby reducing the amount of calculation in the fire detection link. In addition, in this application, the algorithm is converted based on the ncnn lightweight framework and transplanted based on the arm framework, so that the running platform of the algorithm can adapt to the terminal embedded platform of the arm framework, which improves the application ability of the algorithm and reduces the hardware cost.

The above description is merely an embodiment of the present application, and is not intended to limit the present application. Various modifications and variations of this application are possible for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims (10)

1. A multi-factor flame identification method suitable for an embedded platform is characterized by comprising the following steps:

establishing a fire sample library, wherein the fire sample library is from network fire pictures and combustion experiment pictures;

respectively acquiring a plurality of field video frames;

respectively extracting a moving target in each site video frame to obtain one or more quasi-fire areas;

respectively confirming fire in one or more quasi fire areas according to the fire sample library, and judging whether fire information exists in the one or more quasi fire areas;

and if so, dividing the fire hazard information into fire hazard grades, and generating corresponding alarm signals according to the divided fire hazard grades.

2. The multi-factor flame identification method applicable to the embedded platform according to claim 1, wherein the performing fire identification on one or more quasi-fire areas according to the fire sample library and determining whether fire information exists in the one or more quasi-fire areas comprises:

respectively adopting a BP neural network algorithm, an SSD algorithm and a Yolo algorithm to detect one or more quasi-fire areas, wherein when the BP neural network algorithm is adopted to detect the fire areas, network fire pictures and combustion experiment pictures in the fire sample library are extracted to form a BP neural network training set;

respectively outputting fire confidence degrees according to the detection;

and judging whether fire information exists according to the fire confidence coefficient.

3. The multi-factor flame recognition method applicable to the embedded platform according to claim 2, wherein when the BP neural network algorithm is adopted to detect one or more quasi-fire areas, at least half of network fire pictures and at least half of combustion experiment pictures contained in a fire sample library are extracted to form a BP neural network training set.

4. The multi-factor flame identification method suitable for the embedded platform according to claim 2, wherein the determining whether the fire information exists according to the fire confidence coefficient comprises: when the BP neural network algorithm is adopted for detection, the fire confidence coefficient P of the BP network is output, wherein P belongs to [0,1], and whether fire information exists or not is judged according to the fire confidence coefficient P.

5. The multi-factor flame identification method suitable for the embedded platform according to claim 4, wherein the SSD algorithm and the Yolo algorithm are respectively adopted to detect the fire area, and SSD fire confidence level P _ A and Yolo fire confidence level P _ B are respectively output, if P >0.8 and P _ A >0.8, the fire information is big fire information; if P >0.6 and P _ B >0.7, the fire information is small fire information.

6. The multi-factor flame identification method for embedded platforms as claimed in claim 5, wherein the fire rating of the fire information and the generation of the corresponding alarm signal according to the rated fire rating comprises: if the fire information is judged to be large fire information, acquiring continuous field video frames more than 30 frames and less than 60 frames, and dividing the fire information into a first-level fire early warning and generating a first-level warning signal when the large fire information is generated in the continuous field video frames more than 30 frames and less than 60 frames.

7. The multi-factor flame identification method for embedded platforms as claimed in claim 5, wherein the fire rating of the fire information and the generation of the corresponding alarm signal according to the rated fire rating comprises: the fire classification of the fire information and the generation of the corresponding alarm signal according to the classified fire classification comprises: if the fire information is judged to be large fire information, acquiring continuous field video frames more than 60 frames and less than 90 frames, and when the large fire information appears in the continuous field video frames more than 60 frames and less than 90 frames, dividing the fire information into two-stage fire early warning and generating two-stage warning signals.

8. The multi-factor flame identification method for embedded platforms as claimed in claim 5, wherein the fire rating of the fire information and the generation of the corresponding alarm signal according to the rated fire rating comprises: the fire classification of the fire information and the generation of the corresponding alarm signal according to the classified fire classification comprises: if the fire information is judged to be large fire information, acquiring more than 90 continuous field video frames, and if the large fire information appears in the more than 90 continuous field video frames, the fire information is three-level fire early warning and generating three-level alarm signals.

9. The multi-factor flame identification method for embedded platforms as claimed in claim 5, wherein the fire rating of the fire information and the generation of the corresponding alarm signal according to the rated fire rating comprises: if the fire information is judged to be small fire information, acquiring continuous field video frames more than 15 frames and less than 30 frames, and if the small fire information appears in the continuous field video frames more than 15 frames and less than 30 frames, the fire information is a level 0 fire early warning and generates a level 0 warning signal.

10. The multi-factor flame identification method suitable for the embedded platform according to any one of claims 1-9, wherein the extracting of the moving object in the video frame to obtain the quasi-fire area comprises:

performing background modeling on the acquired field video frame by adopting a Gaussian mixture model modeling method;

updating parameters in the Gaussian mixture model to obtain a background image;

and subtracting the obtained background image from the field video frame, and extracting the moving target in the field video frame to obtain the quasi-fire area.

Images