[go: up one dir, main page]

CN116013024B - A mine fire monitoring method based on visible light visual feature fusion - Google Patents

A mine fire monitoring method based on visible light visual feature fusion Download PDF

Info

Publication number
CN116013024B
CN116013024B CN202310015335.XA CN202310015335A CN116013024B CN 116013024 B CN116013024 B CN 116013024B CN 202310015335 A CN202310015335 A CN 202310015335A CN 116013024 B CN116013024 B CN 116013024B
Authority
CN
China
Prior art keywords
fire
area
frame
mine
visible light
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310015335.XA
Other languages
Chinese (zh)
Other versions
CN116013024A (en
Inventor
范伟强
李晓宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inner Mongolia University
Original Assignee
Inner Mongolia University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inner Mongolia University filed Critical Inner Mongolia University
Priority to CN202310015335.XA priority Critical patent/CN116013024B/en
Publication of CN116013024A publication Critical patent/CN116013024A/en
Application granted granted Critical
Publication of CN116013024B publication Critical patent/CN116013024B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Fire-Detection Mechanisms (AREA)

Abstract

The invention discloses a mine external factor fire monitoring method with visible light visual characteristics fused, which mainly comprises the following steps: a visible light camera is installed at the optimal position of the area to be monitored of the mine; dividing a fire disaster area by adopting an improved seed area growth algorithm; calculating static characteristics and dynamic characteristics of a fire area, and establishing a fire sample data set and an identification model; respectively training and testing an identification model; the mine fire monitoring system monitors whether a mine external fire exists or not in real time through the identification model; when the presence is identified, a fire alarm is activated. The mine external fire monitoring method can overcome the problems of the existing monitoring method, and can improve the identification speed, accuracy and reliability of early fire on the basis of realizing underground large-area monitoring, thereby meeting the actual requirements of coal mine safety production.

Description

一种可见光视觉特征融合的矿井外因火灾监测方法A mine fire monitoring method based on visible light visual feature fusion

技术领域Technical Field

本发明涉及一种可见光视觉特征融合的矿井外因火灾监测方法,尤其是基于视觉的目标检测技术、图像分割技术、视觉特征提取技术、以及火灾监测技术。The present invention relates to a mine external cause fire monitoring method integrating visible light visual features, in particular to a vision-based target detection technology, an image segmentation technology, a visual feature extraction technology, and a fire monitoring technology.

背景技术Background Art

近年来,我国煤矿发生的重特大事故统计表明,矿井火灾是煤矿重特大事故中的五大灾害之一。当前相关研究人员对矿井火灾的致灾机理和救灾风险等认识不充分,导致在矿井火灾的感知和判识方面仍缺少有效的技术方法,造成矿井火灾重特大事故时有发生,并引起巨大的经济损失,严重情况的还造成矿井下的人员伤亡。在火灾发生时,通常会产生火焰、烟雾、有毒有害气体等。同时,事故调查表明:在矿井火灾事故中,由于创伤和烧伤造成人员死亡占比不足20%,一氧化碳中毒窒息造成人员死亡占比高于80%。一氧化碳浓度越高、持续时间越长,对人体伤害越重,直至死亡。因此,在煤矿井下,及时发现高温热源或早期火灾、快速定位灾害源位置、准确判识灾害类型、针对性启动应急预案和应急救援,对于煤炭安全生产来说至关重要。In recent years, statistics on major accidents in coal mines in my country show that mine fire is one of the five major disasters in coal mine accidents. At present, relevant researchers have insufficient understanding of the disaster-causing mechanism and disaster relief risks of mine fires, resulting in the lack of effective technical methods in the perception and identification of mine fires, causing major accidents of mine fires to occur from time to time, causing huge economic losses, and in serious cases, causing casualties in the mines. When a fire occurs, flames, smoke, toxic and harmful gases are usually produced. At the same time, accident investigations show that in mine fire accidents, deaths caused by trauma and burns account for less than 20%, and deaths caused by carbon monoxide poisoning and suffocation account for more than 80%. The higher the concentration of carbon monoxide and the longer the duration, the more serious the damage to the human body, until death. Therefore, in coal mines, it is crucial to timely discover high-temperature heat sources or early fires, quickly locate the location of disaster sources, accurately identify disaster types, and launch emergency plans and emergency rescue in a targeted manner for coal production safety.

目前,研究人员为实现矿井外因火灾的监测和预警,对其进行了大量的理论研究、试验分析和现场试验,提出了多种感知和预警方法,取得了大量研究成果。但现有技术均涉及煤炭地质赋存条件,开采方法和工艺、传感器技术等诸多因素的影响,导致现有煤矿火灾预警系统的漏报率和误报率还很高。同时,现有的监测技术无法实现早期火灾预警,也无法对灾害源进行定位与跟踪,难以满足煤矿安全生产需要。此外,在井下恶劣环境中,灾害感知装置所依赖的传感器容易受到环境因素的干扰,且这些传感器仅能够对安装位置处或周围小范围的监测,很难实现对矿井下灾情的大面积监控。At present, researchers have conducted a lot of theoretical research, experimental analysis and field tests to realize the monitoring and early warning of external causes of mine fires, proposed a variety of perception and early warning methods, and achieved a lot of research results. However, the existing technologies all involve the influence of many factors such as coal geological conditions, mining methods and processes, sensor technology, etc., resulting in high omission and false alarm rates of existing coal mine fire early warning systems. At the same time, the existing monitoring technology cannot achieve early fire warning, nor can it locate and track the source of disasters, which makes it difficult to meet the needs of coal mine safety production. In addition, in the harsh environment underground, the sensors relied on by the disaster perception device are easily interfered by environmental factors, and these sensors can only monitor the installation location or a small area around it, making it difficult to achieve large-scale monitoring of disasters in mines.

在矿井外因火灾的视觉监测中,现有基于近红外和远红外波段的矿井外因火灾监测方法存在不同程度的缺陷。如近红外监测能够在较恶劣的环境中获得比可见光波段更清晰的视频图像,但极易受巷道灯、矿灯、车灯等干扰光源的影响;远红外监测具有监视空间大、抗干扰能力强、实时性好等优点,但容易受到吸收性气体、粉尘和环境温度的影响,导致监视距离受限。可见光视觉监测不仅具有监视空间大、几何信息丰富、时空分辨率高等优点,还具有红外监测所不能表达的颜色特征以及远距离监视性能。为此,基于可见光视觉特征的火灾监测已经成为矿井外因火灾监测的重要研究方向之一。但目前的可见光火灾视觉监测法仅适用于失控火源或无干扰光源的特定场景,无法满足矿井外因火灾在早期阶段和复杂环境下的视觉监测需求。In the visual monitoring of external fires in mines, the existing methods for monitoring external fires in mines based on near-infrared and far-infrared bands have defects to varying degrees. For example, near-infrared monitoring can obtain clearer video images than visible light bands in harsh environments, but it is easily affected by interfering light sources such as tunnel lights, mining lamps, and car lights; far-infrared monitoring has the advantages of large monitoring space, strong anti-interference ability, and good real-time performance, but it is easily affected by absorbent gases, dust, and ambient temperature, resulting in limited monitoring distance. Visible light visual monitoring not only has the advantages of large monitoring space, rich geometric information, and high spatiotemporal resolution, but also has color characteristics that cannot be expressed by infrared monitoring and long-distance monitoring performance. For this reason, fire monitoring based on visible light visual features has become one of the important research directions for monitoring external fires in mines. However, the current visible light fire visual monitoring method is only applicable to specific scenarios of out-of-control fire sources or non-interfering light sources, and cannot meet the visual monitoring needs of external fires in mines in the early stages and complex environments.

针对可见光火灾视觉监测中存在的问题,结合煤矿井下特殊环境,本发明利用矿井外因火灾蔓延过程中的视觉特征,实现了一种可见光视觉特征融合的矿井外因火灾监测方法,利用防爆摄像机等外部设备作为视频图像采集装置,可以实时采集监测区域的视频图像,仅在井下重点区域部署少量的摄像机,就能实现煤矿井下的大面积监测;同时,对监控设备采集到的视频图像进行计算机视觉和图像处理技术分析,并结合矿井外因火灾的视觉特征对视频图像中火灾疑似目标进行特征融合识别,以此达到快速、精确、可靠的识别监视区域内的高温热源和早期火灾。该方法与现有的监测预警方法相比,基于本发明的矿井外因火灾判识方法将更为快速、精确和可靠。此外,在矿井应急救援过程中,基于可见光视觉的火灾监测还能够辅助救援人员对现场灾情做出预判并制订对应的应急预案。Aiming at the problems existing in the visual monitoring of visible light fires, combined with the special environment of coal mines, the present invention uses the visual features of the spread of external fires in mines to realize a method for monitoring external fires in mines with the fusion of visible light visual features. By using external equipment such as explosion-proof cameras as video image acquisition devices, video images of the monitoring area can be acquired in real time. Only a small number of cameras are deployed in key areas underground to achieve large-scale monitoring of coal mines. At the same time, the video images collected by the monitoring equipment are analyzed by computer vision and image processing technology, and the visual features of the external fires in mines are combined to identify the suspected fire targets in the video images, so as to achieve rapid, accurate and reliable identification of high-temperature heat sources and early fires in the monitoring area. Compared with the existing monitoring and early warning methods, the mine external fire identification method based on the present invention will be faster, more accurate and more reliable. In addition, in the process of emergency rescue in mines, fire monitoring based on visible light vision can also assist rescue personnel in making predictions on the disaster situation on the scene and formulating corresponding emergency plans.

发明内容Summary of the invention

本发明所要解决的技术问题在于克服上述现有的可见光视觉监测技术进行矿井外因火灾识别时存在的误报和漏报问题,以及在实现井下大面积监测的基础上,提高早期火灾的识别速度、精确度、可靠性,进而满足煤矿安全生产的实际需要。The technical problem to be solved by the present invention is to overcome the problems of false alarm and missed alarm when the above-mentioned existing visible light visual monitoring technology is used to identify external causes of mine fires, and to improve the recognition speed, accuracy and reliability of early fires on the basis of realizing large-area monitoring underground, thereby meeting the actual needs of safe production in coal mines.

本发明具体采用以下技术方案解决上述技术问题:The present invention specifically adopts the following technical solutions to solve the above technical problems:

一种可见光视觉特征融合的矿井外因火灾监测方法,其特征在于:矿井外因火灾监测方法包括以下步骤:A mine fire monitoring method based on visible light visual feature fusion, characterized in that the mine fire monitoring method includes the following steps:

步骤1:结合矿井待监视区域的空间特征、地质条件、煤质特性和采动影响,在待监测区域的最优位置处安装可见光摄像机;Step 1: Based on the spatial characteristics, geological conditions, coal quality characteristics and mining impact of the mine area to be monitored, a visible light camera is installed at the optimal location of the area to be monitored;

步骤2:对采集的可见光视频进行等间隔采样,获取第t帧和t-1帧的可见光图像;采用改进的种子区域生长算法提取出第t帧和t-1帧中的火灾区域;Step 2: Sample the collected visible light video at equal intervals to obtain visible light images of the t-th frame and the t-1-th frame; use the improved seed region growing algorithm to extract the fire area in the t-th frame and the t-1-th frame;

步骤3:计算火灾区域的静态特征和动态特征;所述的静态特征为圆形度、矩形度、偏心率、颜色、纹理和尖角;所述的动态特征为面积变化率、质心移动、周长增长率、相似系数和闪动频率;融合得到的静态特征和动态特征,构建第t帧图像中火灾区域的特征向量;Step 3: Calculate the static features and dynamic features of the fire area; the static features are circularity, rectangularity, eccentricity, color, texture and sharp corners; the dynamic features are area change rate, centroid movement, perimeter growth rate, similarity coefficient and flashing frequency; fuse the static features and dynamic features to construct a feature vector of the fire area in the t-th frame image;

步骤4:通过循环步骤2~步骤3,构建矿井外因火灾的样本数据集;随机抽取样本数据集中的一部分样本,构成神经网络模型的训练数据集,对神经网络模型进行训练;Step 4: By looping steps 2 to 3, a sample data set of external causes of mine fires is constructed; a portion of samples in the sample data set are randomly selected to form a training data set for the neural network model, and the neural network model is trained;

步骤5:将剩余的另一部分样本作为测试数据集,输入训练后的神经网络模型,采用预测值判断是否存在矿井外因火灾,并采用客观评价方法分析所述神经网络模型的综合性能;Step 5: Use the remaining samples as a test data set, input them into the trained neural network model, use the predicted value to determine whether there is an external cause of mine fire, and use an objective evaluation method to analyze the comprehensive performance of the neural network model;

步骤6:采用改进的种子区域生长算法提取监视区域视频流中当前帧和前一帧的火灾疑似区域;通过步骤3构建当前帧中火灾疑似区域的特征向量,并将前帧中火灾疑似区域的特征向量输入训练后的神经网络模型,识别是否存在矿井外因火灾。Step 6: Use the improved seed region growing algorithm to extract the suspected fire area in the current frame and the previous frame in the video stream of the monitored area; construct the feature vector of the suspected fire area in the current frame through step 3, and input the feature vector of the suspected fire area in the previous frame into the trained neural network model to identify whether there is an external cause of fire in the mine.

进一步地,所述的最优位置为距离监控目标区域最近距离不得小于5米,最远距离不得大于50米,安装位置须避开固定干扰光源,或于顺光方向安装,安装于坚固的顶板或巷道侧壁,不得影响井下正常施工作业。Furthermore, the optimal position is that the closest distance to the monitored target area shall not be less than 5 meters, and the farthest distance shall not be greater than 50 meters. The installation position must avoid fixed interfering light sources, or be installed in the direction of light, installed on a solid roof or tunnel side wall, and must not affect normal construction operations underground.

进一步地,所述改进的种子区域生长算法包括:将第t帧和t-1帧图像进行平滑滤波,选取滤波后图像中灰度为1的像素点作为种子点;将所述种子点的3×3邻域作为起始的生长区域,并且进行8邻域扩展;选取邻域像素的灰度均值和方差作为区域生长条件,当邻域像素与种子点的灰度均值和方差小于设定的生长阈值时,将所述种子点相对的像素点归并到已生长区域;所述的灰度均值通过计算得到;所述的方差通过计算得到;所述的区域生长条件为式中fm(x,y)和fv(x,y)分别为种子点f(x,y)对应邻域的均值和方差;seedpoint(x0,y0)为种子点f(x,y)的灰度;T1和T2分别为生长阈值,T1和T2与方差fv(x,y)有关;r为邻域半径,r=1。Furthermore, the improved seed region growing algorithm includes: smoothing and filtering the t-th frame and t-1 frame images, selecting pixels with a grayscale of 1 in the filtered image as seed points; using the 3×3 neighborhood of the seed point as the starting growth area, and performing 8-neighborhood expansion; selecting the grayscale mean and variance of the neighborhood pixels as region growing conditions, and when the grayscale mean and variance of the neighborhood pixels and the seed point are less than a set growth threshold, merging the pixels corresponding to the seed point into the grown area; the grayscale mean is calculated by Calculated; the variance is obtained by Calculated; the regional growth condition is In the formula, fm (x,y) and fv (x,y) are the mean and variance of the neighborhood corresponding to the seed point f(x,y); seedpoint( x0 , y0 ) is the grayscale of the seed point f(x,y); T1 and T2 are the growth thresholds, which are related to the variance fv (x,y); r is the neighborhood radius, r= 1 .

进一步地,所述的纹理特征通过计算火灾区域或火灾疑似区域的图像熵得到,所述的图像熵式中RIE为图像熵;Pi为火灾疑似区域中灰度i出现的概率;L为可见光图像的灰度等级。Furthermore, the texture feature is obtained by calculating the image entropy of the fire area or the suspected fire area. Where R IE is the image entropy; Pi is the probability of gray level i appearing in the suspected fire area; L is the gray level of the visible light image.

进一步地,所述的尖角特征计算过程为:Furthermore, the sharp corner feature calculation process is as follows:

步骤1:采用遍历法寻找火灾疑似区域的左基点和右基点,并通过一阶微分算子求取“左基点-最高点-右基点”连接曲线在每一边缘点处的导数;Step 1: Use the traversal method to find the left base point and the right base point of the suspected fire area, and use the first-order differential operator to obtain the derivative of the "left base point-highest point-right base point" connection curve at each edge point;

步骤2:根据边缘点的导数判别是否存在过零点,若存在过零点且左侧导数为正,右侧导数为负,则判定该边缘点为局部尖角;Step 2: Determine whether there is a zero-crossing point based on the derivative of the edge point. If there is a zero-crossing point and the left derivative is positive and the right derivative is negative, the edge point is determined to be a local sharp corner.

步骤3:查找每个尖角下侧第N行的左边缘点和右边缘点坐标,并由左边缘点和右边缘点坐标计算尖角对应的宽度,若尖角的宽高比小于0.5,则判定该尖角为火焰尖角,否则判定为噪声或毛刺。Step 3: Find the coordinates of the left and right edge points in the Nth row below each sharp corner, and calculate the width corresponding to the sharp corner based on the coordinates of the left and right edge points. If the aspect ratio of the sharp corner is less than 0.5, the sharp corner is judged to be a flame sharp corner, otherwise it is judged to be noise or burrs.

进一步地,所述的闪动频率获取过程为:采用相似系数寻找第t帧和t-1帧中趋于相同的火灾疑似区域,计算趋于相同的火灾疑似区域对应的累计灰度差值变化率,所述的累计灰度差值变化率式中Fj为第j个火灾疑似区域的累计灰度差值变化率;M、N为第j个火灾疑似区域的像素总数;Vt,i为第t帧图像中火灾疑似区域内第i个像素值;Vt-1,i为第t-1帧图像中火灾疑似区域内第i个像素值;Δt为第t-1帧与第t帧的时间间隔。Furthermore, the flash frequency acquisition process is as follows: using the similarity coefficient to find the same suspected fire area in the t-th frame and the t-1 frame, and calculating the cumulative grayscale difference change rate corresponding to the same suspected fire area. Where Fj is the cumulative grayscale difference change rate of the j-th suspected fire area; M and N are the total number of pixels in the j-th suspected fire area; Vt ,i is the i-th pixel value in the suspected fire area in the t-th frame image; Vt -1,i is the i-th pixel value in the suspected fire area in the t-1-th frame image; Δt is the time interval between the t-1-th frame and the t-th frame.

进一步地,所述的相似系数式中μt、μt-1分别为第t帧和t-1帧中火源疑似区域的平均灰度;σt、σt-1分别为第t帧和t-1帧火源疑似区域的标准差;C1,C2为常数,避免分母为0;通常取C1=(K1·L)2,C2=(K2·L)2,一般地K1=0.01,K2=0.03,L为灰度图像的动态范围,L=255。Furthermore, the similarity coefficient In the formula, μ t and μ t-1 are the average grayscale of the suspected fire source area in the t-th frame and t-1 frame respectively; σ t and σ t-1 are the standard deviations of the suspected fire source area in the t-th frame and t-1 frame respectively; C 1 and C 2 are constants to avoid the denominator being 0; usually C 1 = (K 1 ·L) 2 and C 2 = (K 2 ·L) 2 , generally K 1 = 0.01 and K 2 = 0.03, and L is the dynamic range of the grayscale image, L = 255.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

图1本发明的矿井外因火灾监测系统示意图;FIG1 is a schematic diagram of a mine external fire monitoring system according to the present invention;

图2本发明的矿井外因火灾监测方法流程图;FIG2 is a flow chart of a method for monitoring external causes of fire in a mine according to the present invention;

图3本发明的矿井外因火灾火焰尖角提取示意图。FIG3 is a schematic diagram of extracting the flame corner of a mine fire caused externally according to the present invention.

具体实施方式DETAILED DESCRIPTION

为使本发明的目的、技术方案和优点更加清楚,下面结合附图和具体实施方法对本发明做详细、完整地描述,实施例不应被视为限制本发明的使用范围。In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail and completely below in conjunction with the accompanying drawings and specific implementation methods. The embodiments should not be construed as limiting the scope of use of the present invention.

如图1所示,矿井外因火灾监测系统分为井上部分和井下部分,用于实现矿井外因火灾的监测,其主要组成部分包括:As shown in Figure 1, the mine external fire monitoring system is divided into an above-ground part and an underground part, which is used to monitor the mine external fire. Its main components include:

1.信息处理服务器(101):负责存储由可见光摄像机(105)采集的视频图像,并分析采集的视频图像中是否包含矿井外因火灾疑似区域,如果存在火灾疑似区域,则提取火灾疑似区域的视觉特征,输入训练后的BP神经网络,采用预测值判断是否是矿井外因火灾,若是矿井外因火灾则通过通信网络使报警模块(107)发出声、光、振动等报警信号,同时使监视服务器(102)在报警模块发出报警信息的同时,在监视屏幕上进行报警提示和人机交互。1. Information processing server (101): responsible for storing video images collected by the visible light camera (105), and analyzing whether the collected video images contain suspected areas of external fire in the mine. If there is a suspected fire area, the visual features of the suspected fire area are extracted and input into the trained BP neural network. The predicted value is used to determine whether it is a fire caused by external factors in the mine. If it is a fire caused by external factors in the mine, the alarm module (107) sends out sound, light, vibration and other alarm signals through the communication network, and the monitoring server (102) sends out alarm prompts and human-computer interaction on the monitoring screen when the alarm module sends out the alarm information.

2.监视服务器(102):负责对煤矿监测区域的监控数据进行显示服务,同时监视服务器(102)与信息处理服务器(101)相互通信连接,对监测区域的实时图像进行显示,并在报警模块(107)发出报警信息后进行报警提示和人机交互,生产管理人员可通过监视服务器(102)对信息处理服务器(101)存储的历史数据调取查询,且监视服务器(102)通过通信线路连接核心交换机(103)接入矿用通信网络。2. Monitoring server (102): responsible for displaying monitoring data of the coal mine monitoring area. At the same time, the monitoring server (102) and the information processing server (101) are connected to each other in communication, displaying real-time images of the monitoring area, and providing alarm prompts and human-computer interaction after the alarm module (107) issues an alarm message. Production management personnel can retrieve and query historical data stored in the information processing server (101) through the monitoring server (102), and the monitoring server (102) is connected to the core switch (103) through a communication line to access the mine communication network.

3.核心交换机(103):矿用通信网络的核心管理和交换设备,负责所有接入矿用通信网络的设备的管理和数据交换,具有路由功能,通过防火墙连接外部互联网。3. Core switch (103): The core management and switching device of the mine communication network, responsible for the management and data exchange of all devices connected to the mine communication network, with routing function, connected to the external Internet through a firewall.

4.环网交换机(104):矿用通信网络的井下交换设备,安装在井下,多个环网交换机以环网方式连接。4. Ring network switch (104): underground switching equipment of the mine communication network, installed underground, with multiple ring network switches connected in a ring network manner.

5.可见光摄像机(105):即安装于矿井下重点监控区域的图像采集设备,负责采集矿井下运输巷、通风巷、联络巷、掘进巷、开切眼、采煤工作面、掘进工作面、机电硐室、井下变电站等易发生火灾的监测区域的视频图像,视频图像可为彩色图像、灰度图像或伪彩色图像;可见光摄像机(105)采用矿用防爆式摄像机。5. Visible light camera (105): an image acquisition device installed in key monitoring areas in the mine, responsible for collecting video images of monitoring areas prone to fire, such as transportation tunnels, ventilation tunnels, communication tunnels, heading tunnels, cutting eyes, coal mining faces, heading faces, electromechanical chambers, underground substations, etc. in the mine. The video images can be color images, grayscale images or pseudo-color images; the visible light camera (105) adopts a mining explosion-proof camera.

6.通信分站(106):一端与双目视觉像机(105)通信连接,另一端与环网交换机(104)通信连接,并且可通过无线通信网络或有线通信网络与两端设备相连,本示例中采用有线通信方式进行说明。6. Communication substation (106): One end is connected to the binocular vision camera (105) for communication, and the other end is connected to the ring network switch (104) for communication. It can be connected to the devices at both ends via a wireless communication network or a wired communication network. In this example, wired communication is used for illustration.

7.报警模块(107):采用声、光、振动等报警方式,通过已有的通信网络直接与监视服务器(102)通信连接,通过核心交换机(103)与信息处理服务器(101)通信连接;当报警模块(107)接收到信息处理服务器(101)发送的报警信号后,进行声、光、振动报警中的一种或多种报警方式进行报警,提示工作人员对现场进行处置和启动应急预案。7. Alarm module (107): uses sound, light, vibration and other alarm methods, directly communicates with the monitoring server (102) through the existing communication network, and communicates with the information processing server (101) through the core switch (103); when the alarm module (107) receives the alarm signal sent by the information processing server (101), it uses one or more alarm methods such as sound, light, and vibration to alarm, prompting the staff to deal with the scene and activate the emergency plan.

如图2所示的矿井外因火灾监测方法流程,包括:The process of the mine external fire monitoring method as shown in FIG2 includes:

1.初始化(201):结合矿井的空间特征、地质条件、煤质特性和采动影响,在矿井下重点监测区域的最优位置处安装可见光摄像机;所述重点监测区域包括矿井下的运输巷、通风巷、联络巷、掘进巷、开切眼、采煤工作面、掘进工作面、机电硐室、井下变电站等易发生灾害隐患的区域;所述的最优位置为距离监控目标区域最近距离不得小于5米,最远距离不得大于50米,安装位置须避开固定干扰光源,或于顺光方向安装,安装于坚固的顶板或巷道侧壁,不得影响井下正常施工作业。1. Initialization (201): In combination with the spatial characteristics, geological conditions, coal quality characteristics and mining impacts of the mine, a visible light camera is installed at the optimal position of the key monitoring area in the mine; the key monitoring area includes the transportation tunnel, ventilation tunnel, communication tunnel, heading tunnel, opening eye, coal mining face, heading face, electromechanical chamber, underground substation and other areas prone to disaster hazards in the mine; the optimal position is that the closest distance to the monitoring target area shall not be less than 5 meters, and the farthest distance shall not be more than 50 meters. The installation position must avoid fixed interfering light sources, or be installed in the direction of light, installed on a solid roof or tunnel side wall, and must not affect normal construction operations underground.

2.分割图像中火灾区域(202):对可见光视频进行等间隔采样,获取第t帧和t-1帧的可见光图像,采用改进的种子区域生长算法提取第t帧和t-1帧中的火灾区域;所述改进的种子区域生长算法包括:将第t帧和t-1帧图像进行平滑滤波,选取滤波后图像中灰度为1的像素点作为种子点;将所述种子点的3×3邻域作为起始的生长区域,并且进行8邻域扩展;选取邻域像素的灰度均值和方差作为区域生长条件,当邻域像素与种子点的灰度均值和方差小于设定的生长阈值时,将所述种子点相对的像素点归并到已生长区域;2. Segmenting the fire area in the image (202): sampling the visible light video at equal intervals, obtaining the visible light images of the t-th frame and the t-1-th frame, and extracting the fire area in the t-th frame and the t-1-th frame using an improved seed region growing algorithm; the improved seed region growing algorithm comprises: performing smoothing filtering on the t-th frame and the t-1-th frame images, selecting pixels with a grayscale of 1 in the filtered images as seed points; using the 3×3 neighborhood of the seed point as the starting growth area, and performing 8-neighborhood expansion; selecting the grayscale mean and variance of the neighborhood pixels as region growth conditions, and when the grayscale mean and variance of the neighborhood pixels and the seed point are less than a set growth threshold, merging the pixels relative to the seed point into the grown area;

所述的灰度均值通过计算得到;所述的方差通过计算得到;所述的区域生长条件为式中fm(x,y)和fv(x,y)分别为种子点f(x,y)对应邻域的均值和方差;seedpoint(x0,y0)为种子点f(x,y)的灰度;T1和T2分别为生长阈值,T1和T2与方差fv(x,y)有关;r为邻域半径,r=1。The grayscale mean is obtained by Calculated; the variance is obtained by Calculated; the regional growth condition is In the formula, fm (x,y) and fv (x,y) are the mean and variance of the neighborhood corresponding to the seed point f(x,y); seedpoint( x0 , y0 ) is the grayscale of the seed point f(x,y); T1 and T2 are the growth thresholds, which are related to the variance fv (x,y); r is the neighborhood radius, r= 1 .

随着邻域像素不断地并入到已生长区域,导致T1和T2也会随着方差的变化而改变。通过大量实验可知:若区域生长的阈值选择偏小时,火灾疑似区域易出现部分空洞,不利于火灾特征的提取;若区域生长的阈值选择太大时,火灾疑似区域易出现过分割现象。为此,阈值T1和T2分别选择T1=2fv(x,y),T2=fv(x,y)。As the neighboring pixels are continuously incorporated into the grown region, T 1 and T 2 will also change with the variance. Through a large number of experiments, it is known that if the threshold of region growth is too small, the suspected fire region is prone to partial holes, which is not conducive to the extraction of fire features; if the threshold of region growth is too large, the suspected fire region is prone to over-segmentation. For this reason, the thresholds T 1 and T 2 are selected as T 1 = 2f v (x, y) and T 2 = f v (x, y) respectively.

3.计算火灾区域的视觉特征(203):所述视觉特征包括静态特征和动态特征;所述的静态特征为圆形度、矩形度、偏心率、颜色、纹理和尖角;所述的动态特征为面积变化率、质心移动、周长增长率、相似系数和闪动频率;融合得到的静态特征和动态特征,构建第t帧图像中火灾疑似区域的特征向量;3. Calculate the visual features of the fire area (203): the visual features include static features and dynamic features; the static features are circularity, rectangularity, eccentricity, color, texture and sharp corners; the dynamic features are area change rate, centroid movement, perimeter growth rate, similarity coefficient and flashing frequency; the static features and dynamic features are fused to construct a feature vector of the suspected fire area in the t-th frame image;

纹理特征通过计算火灾区域的图像熵得到,图像熵计算公式为式中RIE为图像熵;Pi为火灾疑似区域中灰度i出现的概率;L为可见光图像的灰度等级。The texture feature is obtained by calculating the image entropy of the fire area. The image entropy calculation formula is: Where R IE is the image entropy; Pi is the probability of gray level i appearing in the suspected fire area; L is the gray level of the visible light image.

闪动频率的获取过程主要包括:采用相似系数寻找第t帧和t-1帧中趋于相同的火灾疑似区域,计算趋于相同的火灾疑似区域对应的累计灰度差值变化率,所述的累计灰度差值变化率式中Fj为第j个火灾疑似区域的累计灰度差值变化率;M、N为第j个火灾疑似区域的像素总数;Vt,i为第t帧图像中火灾疑似区域内第i个像素值;Vt-1,i为第t-1帧图像中火灾疑似区域内第i个像素值;Δt为第t-1帧与第t帧的时间间隔。The flash frequency acquisition process mainly includes: using the similarity coefficient to find the same suspected fire area in the t-th frame and the t-1 frame, and calculating the cumulative grayscale difference change rate corresponding to the same suspected fire area. Where Fj is the cumulative grayscale difference change rate of the j-th suspected fire area; M and N are the total number of pixels in the j-th suspected fire area; Vt ,i is the i-th pixel value in the suspected fire area in the t-th frame image; Vt -1,i is the i-th pixel value in the suspected fire area in the t-1-th frame image; Δt is the time interval between the t-1-th frame and the t-th frame.

相似系数式中μt、μt-1分别为第t帧和t-1帧中火源疑似区域的平均灰度;σt、σt-1分别为第t帧和t-1帧火源疑似区域的标准差;C1、C2为常数,避免分母为0;通常取C1=(K1·L)2,C2=(K2·L)2,一般地K1=0.01,K2=0.03,L为灰度图像的动态范围,L=255。Similarity coefficient In the formula, μ t and μ t-1 are the average grayscale of the suspected fire source area in the t-th frame and t-1 frame respectively; σ t and σ t-1 are the standard deviations of the suspected fire source area in the t-th frame and t-1 frame respectively; C 1 and C 2 are constants to avoid the denominator being 0; usually C 1 = (K 1 ·L) 2 , C 2 = (K 2 ·L) 2 , generally K 1 = 0.01, K 2 = 0.03, L is the dynamic range of the grayscale image, L = 255.

4.建立样本数据集和训练识别模型(204):通过循环步骤202~步骤203,获取视频图像中的构建矿井外因火灾的样本数据集;依据所述视觉特征,获取模型结构和模型参数,建立矿井外因火灾的识别模型;所述识别模型根据不同环境下的矿井外因火灾视觉特征,以及矿井外因火灾的演变动态的调整模型结构和模型参数,不断完善识别模型;所述识别模型至少采用神经网络模型和特征匹配模型中的一种;随机抽取样本数据集中60%的样本,构成识别模型的训练数据集,对识别模型进行训练;4. Establishing a sample data set and training a recognition model (204): by looping steps 202 to 203, a sample data set for constructing a mine external cause fire is obtained in the video image; based on the visual features, a model structure and model parameters are obtained to establish a recognition model for the mine external cause fire; the recognition model adjusts the model structure and model parameters according to the visual features of the mine external cause fire in different environments and the evolution dynamics of the mine external cause fire, and continuously improves the recognition model; the recognition model uses at least one of a neural network model and a feature matching model; 60% of the samples in the sample data set are randomly selected to form a training data set for the recognition model, and the recognition model is trained;

5.测试与分析识别模型(205):将剩余40%的样本作为测试数据集,输入训练后的神经网络模型或特征匹配模型,采用预测值判断是否存在矿井外因火灾,并采用客观评价方法分析所述神经网络模型或特征匹配模型的综合性能。所述的客观评价方法采用的指标分别为正确率(Accuracy,ACC)、检测率(True Positive Rate,TPR)、误检率(FalsePositive Rate,FPR)3个指标进行定量分析。其中ACC和TPR越大,FPR越小时,表明所述的识别模型的识别性能越好。式中TP为预测为正的正样本;TN为预测为负的负样本;FP为预测为正的负样本;FN为预测为负的正样本。5. Test and analyze the recognition model (205): The remaining 40% of the samples are used as a test data set and input into the trained neural network model or feature matching model. The predicted value is used to determine whether there is an external cause of fire in the mine, and an objective evaluation method is used to analyze the comprehensive performance of the neural network model or feature matching model. The objective evaluation method uses three indicators, namely, accuracy (ACC), detection rate (TPR), and false positive rate (FPR), for quantitative analysis. The larger the ACC and TPR, and the smaller the FPR, the better the recognition performance of the recognition model. In the formula, TP is the positive sample predicted to be positive; TN is the negative sample predicted to be negative; FP is the negative sample predicted to be positive; FN is the positive sample predicted to be negative.

6.是否存在矿井外因火灾(206):矿井外因火灾监测系统实时采集监控区域内的视频图像,并采用改进的种子区域生长算法提取视频流中当前帧和前一帧的火灾疑似区域;通过步骤203构建当前帧中火灾疑似区域的特征向量,并将前帧中火灾疑似区域的特征向量输入训练后的神经网络模型,识别是否存在矿井外因火灾。当神经网络模型识别到视频图像中存在矿井外因火灾时,矿井外因火灾监测系统顺序执行(207),否则返回执行(205)。6. Whether there is an external cause fire in the mine (206): The external cause fire monitoring system of the mine collects video images in the monitoring area in real time, and uses the improved seed region growing algorithm to extract the suspected fire area in the current frame and the previous frame in the video stream; through step 203, the feature vector of the suspected fire area in the current frame is constructed, and the feature vector of the suspected fire area in the previous frame is input into the trained neural network model to identify whether there is an external cause fire in the mine. When the neural network model recognizes that there is an external cause fire in the video image, the external cause fire monitoring system of the mine executes (207) in sequence, otherwise it returns to execute (205).

7.矿井外因火灾预警(207):当识别到矿井外因火灾时,信息处理服务器向报警模块发送的报警信号,报警模块接收到报警信号后,进行声、光、振动报警中的一种或多种报警方式进行报警,提示工作人员对现场进行处置和启动应急预案。7. Mine external fire warning (207): When a mine external fire is identified, the information processing server sends an alarm signal to the alarm module. After receiving the alarm signal, the alarm module uses one or more of the following alarm methods: sound, light, and vibration to alert the staff to handle the scene and activate the emergency plan.

如图3所示的矿井外因火灾火焰尖角提取示意图,其火焰尖角特征(在火灾图像中,火焰尖角是边缘轮廓曲线的极大值点(最高点),且该最高点对应的曲率较大。)计算过程为:As shown in Figure 3, the flame corner extraction diagram of the external cause fire in the mine, the flame corner feature (in the fire image, the flame corner is the maximum point (highest point) of the edge contour curve, and the curvature corresponding to the highest point is large.) The calculation process is:

步骤1:采用遍历法寻找火灾疑似区域的左基点和右基点,并通过一阶微分算子求取“左基点-最高点-右基点”连接曲线在每一边缘点处的导数;Step 1: Use the traversal method to find the left base point and the right base point of the suspected fire area, and use the first-order differential operator to obtain the derivative of the "left base point-highest point-right base point" connection curve at each edge point;

步骤2:根据边缘点的导数判别是否存在过零点,若存在过零点且左侧导数为正,右侧导数为负,则判定该边缘点为局部最高点(尖角);Step 2: Determine whether there is a zero-crossing point based on the derivative of the edge point. If there is a zero-crossing point and the left derivative is positive and the right derivative is negative, the edge point is determined to be a local highest point (cusp);

步骤3:查找每个尖角下侧第N行的左边缘点和右边缘点坐标,并由左边缘点和右边缘点坐标计算尖角对应的宽度,若尖角的宽高比小于0.5,则判定该尖角为火焰尖角,否则判定为噪声或毛刺。Step 3: Find the coordinates of the left and right edge points in the Nth row below each sharp corner, and calculate the width corresponding to the sharp corner based on the coordinates of the left and right edge points. If the aspect ratio of the sharp corner is less than 0.5, the sharp corner is judged to be a flame sharp corner, otherwise it is judged to be noise or burrs.

Claims (7)

1. The mine external factor fire monitoring method with the visible light visual characteristics fused is characterized by comprising the following steps of:
Step 1: the method comprises the steps that by combining the spatial characteristics, geological conditions, coal quality characteristics and mining influence of a mine area to be monitored, a visible light camera is installed at the optimal position of the area to be monitored;
Step 2: sampling the acquired visible light video at equal intervals to acquire visible light images of a t frame and a t-1 frame; extracting fire areas in the t frame and the t-1 frame by adopting an improved seed area growth algorithm;
Step 3: calculating static characteristics and dynamic characteristics of a fire area; the static characteristics are circularity, rectangularity, eccentricity, color, texture and sharp angle; the dynamic characteristics are area change rate, mass center movement, perimeter growth rate, similarity coefficient and flickering frequency; the obtained static features and dynamic features are fused, and feature vectors of fire areas in the t-th frame image are constructed;
Step 4: through the circulation steps 2 to 3, a sample data set of the mine external fire disaster is constructed; randomly extracting a part of samples in the sample data set to form a training data set of the neural network model, and training the neural network model;
Step 5: taking the rest of samples as a test data set, inputting a trained neural network model, judging whether a mine external fire disaster exists or not by adopting a predicted value, and analyzing the comprehensive performance of the neural network model by adopting an objective evaluation method;
Step 6: extracting fire suspected areas of a current frame and a previous frame in a video stream of a monitoring area by adopting an improved seed area growth algorithm; and 3, constructing a feature vector of a fire suspected area in the current frame, inputting the feature vector of the fire suspected area in the previous frame into a trained neural network model, and identifying whether a mine external fire exists.
2. The method for monitoring mine external fire disaster by fusion of visual characteristics of visible light according to claim 1, wherein the optimal position is that the shortest distance from the target area to be monitored is not less than 5m, the longest distance is not more than 50m, and the installation position is required to avoid fixed interference light sources or be installed in the forward direction and is installed on a firm top plate or a tunnel side wall, so that the underground normal construction operation cannot be influenced.
3. The mine external fire monitoring method of claim 1, wherein said modified seed region growing algorithm comprises: smoothing filtering is carried out on the t-th frame image and the t-1 frame image, and pixel points with gray level of 1 in the filtered images are selected as seed points; taking a 3×3 neighborhood of the seed point as an initial growth area, and performing 8 neighborhood expansion; selecting the gray average value and variance of the neighborhood pixels as region growing conditions, and merging the pixel points opposite to the seed points into a grown region when the gray average value and variance of the neighborhood pixels and the seed points are smaller than a set growing threshold;
the gray level average value passes Calculating to obtain; the variance is passed throughCalculating to obtain; the region growth conditions are as followsWherein f m (x, y) and f v (x, y) are respectively the mean and variance of the corresponding neighborhood of the seed point f (x, y); seedpoint (x 0,y0) is the gray scale of the seed point f (x, y); t 1 and T 2 are growth thresholds, T 1 and T 2 are related to variance f v (x, y), respectively; r is the neighborhood radius, r=1.
4. The mine external factor fire monitoring method of claim 1, wherein the texture features are obtained by calculating image entropy of a fire area, and the image entropy calculation formula is as followsWherein R IE is image entropy; p i is the probability of gray level i in the suspected fire area; l is the gray scale of the visible light image.
5. The mine external factor fire monitoring method based on visible light visual feature fusion according to claim 1, wherein the sharp angle feature calculation process is as follows:
Step 1: searching a left base point and a right base point of a fire suspected region by adopting a traversal method, and solving the derivative of a 'left base point-highest point-right base point' connecting curve at each edge point by a first-order differential operator;
step 2: judging whether zero crossing points exist according to the derivative of the edge points, and judging that the edge points are local sharp angles if the zero crossing points exist, the left derivative is positive, and the right derivative is negative;
step 3: searching the coordinates of a left edge point and a right edge point of the N line below each sharp angle, calculating the width corresponding to the sharp angle according to the coordinates of the left edge point and the right edge point, judging the sharp angle as a flame sharp angle if the aspect ratio of the sharp angle is smaller than 0.5, and judging the sharp angle as noise or burr if the aspect ratio of the sharp angle is smaller than 0.5.
6. The mine external factor fire monitoring method with visible light visual characteristic fusion according to claim 1, wherein the flicker frequency obtaining process is as follows: searching fire suspected areas which tend to be the same in the t frame and the t-1 frame by adopting a similarity coefficient, and calculating the corresponding cumulative gray difference change rate of the fire suspected areas which tend to be the same, wherein the cumulative gray difference change rate is calculatedWherein F j is the cumulative gray level difference change rate of the j fire suspected region; m, N is the total number of pixels in the j-th fire suspected region; v t,i is the ith pixel value in the fire suspected area in the t frame image; v t-1,i is the ith pixel value in the fire suspected area in the t-1 th frame image; Δt is the time interval between the t-1 th frame and the t-th frame.
7. The method for monitoring mine external factor fire disaster by fusing visual characteristics of visible light according to claim 1, wherein the similarity coefficient is as followsMu t、μt-1 in the formula is the average gray scale of the suspected area of the fire source in the t frame and the t-1 frame respectively; sigma t、σt-1 is the standard deviation of the fire source suspected region of the t frame and the t-1 frame respectively; c 1,C2 is a constant, avoiding denominator being 0; taking C 1=(K1·L)2,C2=(K2·L)2 generally K 1=0.01,K2 =0.03, L is the dynamic range of the gray scale image, and l=255.
CN202310015335.XA 2023-01-04 2023-01-04 A mine fire monitoring method based on visible light visual feature fusion Active CN116013024B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310015335.XA CN116013024B (en) 2023-01-04 2023-01-04 A mine fire monitoring method based on visible light visual feature fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310015335.XA CN116013024B (en) 2023-01-04 2023-01-04 A mine fire monitoring method based on visible light visual feature fusion

Publications (2)

Publication Number Publication Date
CN116013024A CN116013024A (en) 2023-04-25
CN116013024B true CN116013024B (en) 2024-08-30

Family

ID=86022852

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310015335.XA Active CN116013024B (en) 2023-01-04 2023-01-04 A mine fire monitoring method based on visible light visual feature fusion

Country Status (1)

Country Link
CN (1) CN116013024B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118396986B (en) * 2024-06-25 2024-09-03 湖南建工交建宏特科技有限公司 Real-time monitoring method for road engineering construction quality

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105788142A (en) * 2016-05-11 2016-07-20 中国计量大学 A fire detection system and detection method based on video image processing
CN109377703A (en) * 2018-12-06 2019-02-22 河池学院 A forest fire warning system and method based on machine vision

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7786877B2 (en) * 2008-06-20 2010-08-31 Billy Hou Multi-wavelength video image fire detecting system
CN113011224A (en) * 2019-12-19 2021-06-22 李韵涵 Coal mine fire early warning method based on video image
CN112906674A (en) * 2021-04-13 2021-06-04 中国矿业大学(北京) Mine fire identification and fire source positioning method based on binocular vision
CN113420600A (en) * 2021-05-25 2021-09-21 杭州申弘智能科技有限公司 Fire-fighting classification early warning algorithm based on visual image recognition
CN113205562B (en) * 2021-05-31 2023-09-15 中国矿业大学(北京) Mine thermodynamic disaster identification and positioning method based on binocular vision
KR102465105B1 (en) * 2022-04-05 2022-11-09 (주)아이아이에스티 Apparatus and method for detecting disaster based on artificial intelligence model using thermal image complex data

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105788142A (en) * 2016-05-11 2016-07-20 中国计量大学 A fire detection system and detection method based on video image processing
CN109377703A (en) * 2018-12-06 2019-02-22 河池学院 A forest fire warning system and method based on machine vision

Also Published As

Publication number Publication date
CN116013024A (en) 2023-04-25

Similar Documents

Publication Publication Date Title
KR102369229B1 (en) Risk prediction system and risk prediction method based on a rail robot specialized in an underground tunnel
CN112686090B (en) Intelligent monitoring system for abnormal behavior in bus
CN112396658B (en) Indoor personnel positioning method and system based on video
WO2020047879A1 (en) Intelligent inspection system for tunnel
KR101953342B1 (en) Multi-sensor fire detection method and system
CN107133564B (en) Tooling cap detection method
CN108227606A (en) A ship security intelligent management system based on multi-source perception
CN108389359B (en) Deep learning-based urban fire alarm method
CN117789394B (en) Early fire smoke detection method based on motion history image
CN111523386B (en) High-speed railway platform door monitoring and protecting method and system based on machine vision
CN112613449A (en) Safety helmet wearing detection and identification method and system based on video face image
CN112530144B (en) A neural network-based warning method and system for illegal behavior of thermal power plants
CN119314229B (en) Method, system and medium for detecting unsafe behaviors of tunneling working face personnel
CN116013024B (en) A mine fire monitoring method based on visible light visual feature fusion
CN112800975A (en) Behavior identification method in security check channel based on image processing
CN119399446A (en) A smart community target recognition and tracking method based on big data
CN116259002A (en) A video-based human risk behavior analysis method
CN112576310B (en) Tunnel security detection method and system based on robot
CN111597919A (en) Human body tracking method in video monitoring scene
CN114332736A (en) Power site fire safety risk analysis method and system
CN119106925A (en) A coal mine safety supervision method and system
Breive et al. Person detection in thermal images: A comparative analysis of YOLOv8 and YOLOv9 models
CN113486872B (en) Fire dangerous case monitoring system for aerial photography of unmanned aerial vehicle
CN107860691B (en) Laser mine coal dust telemetry method based on machine vision technology
CN116757422A (en) Building engineering safety supervision early warning method and monitoring system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant