[go: up one dir, main page]

CN111494853B - A multi-mode visual servo control fire protection system and its working method - Google Patents

A multi-mode visual servo control fire protection system and its working method Download PDF

Info

Publication number
CN111494853B
CN111494853B CN202010279393.XA CN202010279393A CN111494853B CN 111494853 B CN111494853 B CN 111494853B CN 202010279393 A CN202010279393 A CN 202010279393A CN 111494853 B CN111494853 B CN 111494853B
Authority
CN
China
Prior art keywords
fire
vision device
field
monitor
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010279393.XA
Other languages
Chinese (zh)
Other versions
CN111494853A (en
Inventor
李伟
潘禄
张博
李贝贝
刘秀梅
朱劲松
林达
张帅帅
高鸿壮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China University of Mining and Technology Beijing CUMTB
Original Assignee
China University of Mining and Technology Beijing CUMTB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China University of Mining and Technology Beijing CUMTB filed Critical China University of Mining and Technology Beijing CUMTB
Priority to CN202010279393.XA priority Critical patent/CN111494853B/en
Priority to CA3137995A priority patent/CA3137995C/en
Priority to AU2020441493A priority patent/AU2020441493B2/en
Priority to PCT/CN2020/104824 priority patent/WO2021203582A1/en
Publication of CN111494853A publication Critical patent/CN111494853A/en
Application granted granted Critical
Publication of CN111494853B publication Critical patent/CN111494853B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A62LIFE-SAVING; FIRE-FIGHTING
    • A62CFIRE-FIGHTING
    • A62C31/00Delivery of fire-extinguishing material
    • A62C31/02Nozzles specially adapted for fire-extinguishing
    • A62C31/03Nozzles specially adapted for fire-extinguishing adjustable, e.g. from spray to jet or vice versa
    • AHUMAN NECESSITIES
    • A62LIFE-SAVING; FIRE-FIGHTING
    • A62CFIRE-FIGHTING
    • A62C31/00Delivery of fire-extinguishing material
    • A62C31/28Accessories for delivery devices, e.g. supports
    • AHUMAN NECESSITIES
    • A62LIFE-SAVING; FIRE-FIGHTING
    • A62CFIRE-FIGHTING
    • A62C37/00Control of fire-fighting equipment
    • A62C37/04Control of fire-fighting equipment with electrically-controlled release

Landscapes

  • Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Fire-Detection Mechanisms (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

The invention discloses a multi-mode vision servo control fire-fighting system and a working method thereof, wherein the system comprises a fire monitor, an infrared near-field vision device, a binocular vision device and a control system; the infrared near-field vision device comprises an infrared camera, a near-field camera, a support frame, a support beam and a buckle; the binocular vision devices comprise industrial cameras, sensors, industrial cameras, binocular camera boxes and rotatable electric telescopic rods; the control system comprises a processing unit, a controller and a control cabinet. The intelligent fire extinguishing system integrates a vision system with multiple modes, can meet the fire extinguishing requirements under multiple environments, improves the efficiency of fire extinguishing operation, and achieves the purpose of intelligent fire extinguishing.

Description

Multi-mode visual servo control fire-fighting system and working method thereof
Technical Field
The invention relates to a multi-mode visual servo control fire-fighting system and a working method thereof, belonging to the field of fire fighting.
Background
With the rapid development of China, urban buildings are dense and high-rise, and chemical plants and the like have great potential safety hazards and are very likely to generate large-scale fires. Because the fire environments of urban buildings, chemical plants and the like are complex, the fire hazard degree is high, great economic loss and casualties are easily caused, and effective artificial fire extinguishment cannot be adopted. Therefore, the demand for fire fighting technology is becoming higher and higher in the current social development.
For fire-fighting technology, the fire monitor system plays a leading role therein and is a key in the fire-fighting process. The existing fire monitor can be controlled manually or automatically. The automatic control fire water monitor is mostly in a remote control type, and part of intelligent fire water monitors are also provided. The remote control type intelligent fire monitor judges the position of a fire scene through a fireman and manually controls the fire monitor to rotate to aim at the position of the fire scene to extinguish fire; the existing intelligent fire-fighting system mainly comprises fire sensing technologies such as an infrared flame sensor and the like, and the fire sensing technologies play a key role in the fire sensing technologies. The fire extinguishing mode of the fire extinguishing system mainly judges the position of a fire disaster through infrared sensors and the like, and then controls the fire monitor to rotate to aim at the position of the fire disaster to extinguish the fire. However, single infrared positioning is not accurate, only two-dimensional information of a fire scene can be obtained, distance information of the fire scene cannot be obtained, and a large number of infrared sensors are required to be arranged if accurate positioning is required.
With the rapid development of computer technology, digital image processing and other technologies are widely applied to fire monitor. The patent of application number 201711364149.8 proposes a method for positioning a fire monitor based on binocular vision, which uses the characteristics of a binocular camera to obtain the space position of flame and control the fire monitor to extinguish fire, and the patent mainly describes the positioning of a fire scene, but after the position of the fire scene is obtained, how to control the rotation of the fire monitor is not described, besides, if very serious interference exists in the fire scene, such as dense smoke, the accuracy of the binocular vision can be seriously influenced, and how to position the fire monitor under the condition. The patent application No. 201210277033.1 discloses an automatic flame detection system with three-dimensional positioning, which uses at least two high-resolution infrared CCD image type fire detectors for flame three-dimensional positioning, wherein the positioning of the infrared CCD detectors is not specifically described. Application No. 201610089595.1 discloses a water spray fire extinguishing method of self-service aiming at goods source, this method uses hot line imager to fix a position and then obtain the positional information in the scene of a fire at two different positions, then puts out a fire, however, this method needs to remove between 2 positions and can realize the location, if face the occasion that can't remove, for example install on the fire engine or use as fixed type fire monitor, should how again fix a position again. Meanwhile, most of the existing technologies control the fire monitor to rotate and spray the fire extinguishing agent after identifying the fire scene through the visual technology, and the method has the defects that: if the fire extinguishment is interfered, such as strong wind, fire scene transfer and the like, the method cannot be adjusted in time, and only can be used for executing the identification process again and repositioning, so that the fire extinguishment time is prolonged, and the fire extinguishment progress is influenced.
Therefore, the existing fire monitor based on the vision technology has the following defects:
1. the fire scene is identified for the first time, the fire monitor is controlled to rotate to the position of the designated angle after the fire scene is identified, the visual system ends the work, and if strong wind or fire scene transfer occurs in the fire extinguishing process, the jet flow track cannot accurately reach the position of the fire scene, and only can be positioned again.
2. Separate binocular or infrared techniques are often used in the prior art. For the binocular technology, the positioning accuracy depends on the complexity of the external environment, the positioning can be accurately performed under the condition of less interference, and the positioning accuracy can be influenced if a large number of trees or dense smoke exists; the infrared technology mainly discerns the temperature, if there are many conflagrations in the scene of putting out a fire, then, independently rely on infrared technology to be unable to put out a fire to the point that needs put out a fire accurately, simultaneously, infrared camera often is expensive, uses a plurality of infrared cameras can greatly increased cost undoubtedly, it is required to notice that infrared camera's discernment scope is also limited.
Disclosure of Invention
Aiming at the problems, the invention discloses a multi-mode visual servo control fire-fighting system and a working method thereof, provides a plurality of visual servo control methods, different modes can be applied to different fire scenes, and solves the problems of inaccurate positioning, large influence of the fire scenes and the like of the existing intelligent fire monitor.
In order to achieve the technical purpose, the invention adopts the following technical scheme:
a multi-mode visual servocontrol fire-fighting system, comprising:
the fire monitor is used for emitting fire extinguishing agents to a fire scene;
the first binocular vision device is arranged in front of one side of the fire monitor and used for acquiring a fire scene image and acquiring the spatial positions of the fire monitor and the fire scene, and a first angle sensor is mounted on the first binocular vision device;
the second binocular vision device is arranged at the lateral rear part of the other side of the fire monitor and is used for acquiring a fire scene image and acquiring the spatial position of the fire monitor and the fire scene, and a second angle sensor is arranged on the second binocular vision device;
the third angle sensor is arranged on the gun head of the fire monitor;
the infrared near-field vision device is used for acquiring a temperature image of a fire scene and a jet flow track initial section image of the fire monitor;
and the control system is in signal connection with the first binocular vision device, the second binocular vision device, the first angle sensor, the second angle sensor, the third angle sensor, the infrared near-field vision device and the fire monitor.
The first binocular vision device and the second binocular vision device are identical in structure and are oppositely arranged on two sides of the fire monitor;
the first binocular vision apparatus comprises: the camera comprises a first industrial camera, a second industrial camera, a first angle sensor and a first binocular camera box, wherein the first industrial camera, the first sensor and the second industrial camera are installed in the first binocular camera box in parallel, and the bottom of the first binocular camera box is connected with a first rotatable electric telescopic rod;
the second binocular vision apparatus comprises: the camera comprises a third industrial camera, a fourth industrial camera, a second angle sensor and a second binocular camera box, wherein the third industrial camera, the second angle sensor and the fourth industrial camera are arranged in the second binocular camera box in parallel, and the bottom of the second binocular camera box is connected with a second rotatable electric telescopic rod;
the first rotatable electric telescopic rod, the second rotatable electric telescopic rod and the fire monitor horizontal rotating shaft are located on the same straight line.
The control system comprises a processing unit, a controller and a control cabinet.
The infrared near-field vision device includes:
the bottom of the support frame is fixedly installed with a monitor head of the fire monitor through a buckle, an infrared camera is installed on the upper portion of the support frame and located right above the monitor head, and the optical axis of the infrared camera and the axis of a monitor tube of the fire monitor are located on the same vertical plane;
the supporting beam is connected to one side of the supporting frame in parallel, and a near-field camera is fixedly mounted at one end, far away from the supporting frame, of the supporting beam.
The invention further discloses a working method based on the multi-mode visual servo control fire-fighting system,
through binocular vision device acquires the scene of a fire image and combines angle sensor to obtain the spatial position of fire gun and scene of a fire, and then control fire gun puts out a fire, discerns the efflux orbit and judges whether the efflux reachs the scene of a fire through binocular vision device simultaneously, specifically is:
the first binocular vision device or the second binocular vision device obtains the spatial position (x, y, z) of the fire scene relative to the binocular vision device;
a third angle sensor positioned at the gun head and a first angle sensor positioned at the first binocular camera box measure an included angle alpha between the first binocular vision device and the central axis of the gun head;
a third angle sensor positioned at the gun head and a second angle sensor positioned at the second binocular camera box measure an included angle alpha between the second binocular vision device and the central axis of the gun head;
establishing a first coordinate system by taking a first industrial camera positioned on the left side of a binocular camera box in the first binocular vision device as an origin and taking a first industrial camera optical axis as a z-axis, wherein the coordinates of the fire monitor under the coordinate system are (x 1, y1, z1), and the coordinates of a fire scene are (x, y, z); and moving the first coordinate system to the position of the fire monitor to be changed into a second coordinate system with the fire monitor as an original point, wherein the coordinates of the fire scene under the second coordinate system are (x-x1, y-y 1 and z-z1), and the included angle between the connecting line of the fire scene and the fire monitor and the z axis of the second coordinate system is as follows: β = arctan ((x-x 1)/(z-z 1)); therefore, the angle γ by which the fire monitor needs to be horizontally rotated is: γ = β - α;
the distance d between the fire monitor and the fire scene is obtained by the following formula:
d=
Figure 100002_DEST_PATH_IMAGE001
wherein, in the step (A),
h is the height of the fire monitor, and H is the elongation of the rotatable electric telescopic rod;
according to the distance d, the pitching rotation angle of the fire monitor can be obtained;
according to the spatial position of the fire scene relative to the gun head, the control system sends out an instruction to control the horizontal rotation and the pitching rotation of the fire monitor, so that the fire monitor is aligned to the fire scene, and the fire extinguishing agent jet flow is sprayed out;
the binocular vision device identifies the fire extinguishing agent jet flow track, calculates the spatial position of the fire extinguishing agent jet flow track through the control system, and judges whether the fire extinguishing agent jet flow falling point reaches a fire scene; if the fire extinguishing agent jet flow does not reach the fire scene, the control system sends out an instruction to control the horizontal rotation and the pitching rotation of the fire monitor until the fire extinguishing agent jet flow falls to the fire scene;
recognizing temperature distribution in a view field range through an infrared camera in the infrared near-field vision device, and when the highest temperature is not obviously higher than the ambient temperature, sending an instruction by a controller to control the fire monitor to horizontally rotate until the highest temperature is obviously higher than the ambient temperature;
the control system judges whether the highest temperature point appears in the middle area of the infrared camera view field, and if the highest temperature point does not appear in the middle area of the infrared camera view field, the control system sends an instruction to control the fire monitor to horizontally rotate until the highest temperature point appears in the middle area of the infrared camera view field;
acquiring an initial section image of a fire extinguishing agent jet flow track of the fire monitor through a near-field camera in the infrared near-field vision device;
the control system predicts the linear distance between the fire extinguishing agent jet flow falling point and the gun head of the fire monitor based on the initial section image of the fire extinguishing agent jet flow track of the fire monitor acquired by the infrared near-field vision device, and sends an instruction to control the pitching rotation of the fire monitor according to the predicted linear distance between the fire extinguishing agent jet flow falling point and the gun head until the predicted linear distance between the fire extinguishing agent jet flow falling point and a fire scene is zero.
When the two binocular vision devices start working, the two binocular vision devices simultaneously lift the rotatable electric telescopic rods of the two binocular vision devices, acquire images and rotate to align the images with a fire scene, so that the fire scene is positioned in the middle of the images; at the moment, if the fire monitor is not in the image of one binocular vision device, the rotatable electric telescopic rod of the fire monitor is lowered, and the fire monitor stops working.
The beneficial effects of the invention are as follows:
the invention provides a multi-mode vision servo control fire-fighting system which comprises an infrared near-field vision system, a binocular vision system, a fire monitor and a control system, wherein the infrared near-field vision system is connected with the binocular vision system; the binocular vision system acquires fire scene images and obtains the spatial positions of the fire monitor and the fire scene by combining the sensor angle sensor, so that the fire monitor is controlled to extinguish a fire, and simultaneously, jet flow tracks are identified and whether jet flows reach the fire scene or not is judged, so that the fire extinguishing accuracy of the fire monitor is improved, the two binocular vision systems are arranged, dead-angle-free shooting of the surrounding fire scene can be met, and meanwhile, the binocular vision system can also be applied to fire scenes where the infrared vision system is not suitable, for example, the fire scene position is out of the range of an infrared camera, and fire extinguishing needs to be carried out on one of a plurality of fire scenes; the infrared vision system controls the fire monitor to extinguish fire by acquiring the temperature image of the fire scene and the initial section image of the jet flow track, can be applied to complex fire scene environments, and can also be applied to fire scenes with inapplicable binocular vision systems, such as the situation that a large amount of trees and smoke interfere in the fire scene. The infrared near-field vision system and the binocular vision system can work respectively and can also work cooperatively, and therefore, the fire scenes which are not suitable for the infrared near-field vision system and the binocular vision system are complemented. Compared with a system in a single vision mode, the system can meet fire extinguishing requirements in different scenes, so that the applicability of the fire monitor is greatly enhanced, the problem that the application scene of a single vision system is single is solved, the fire extinguishing operation efficiency is improved, and the robustness of the system is greatly improved. Meanwhile, in the fire extinguishing process, the visual system is always in an open state, the fire extinguishing agent jet flow track is monitored in real time, and the rotation angle of the fire monitor is controlled in a closed-loop mode, so that the fire extinguishing agent can accurately reach the fire scene.
Drawings
FIG. 1 is an overall layout of the present invention;
FIG. 2 is a front view of the infrared near field vision apparatus of the present invention;
FIG. 3 is a top view of the structure of the infrared near field vision device of the present invention;
fig. 4 is a schematic view of the binocular vision apparatus of the present invention;
FIG. 5 is a schematic view of a tilt sensor arrangement in the binocular vision apparatus of the present invention;
FIG. 6 is a schematic diagram of the control system architecture of the present invention;
FIG. 7 is a control flow diagram of the present invention for implementing a fire suppression process;
fig. 8 is a schematic diagram of the processing unit acquiring the fire scene location.
Shown in the figure:
1. an infrared near field vision device; 2. a first binocular vision device; 3. a second binocular vision device; 4. fire monitor; 5. an infrared camera; 6. a near field camera; 7. a support frame; 8. a support beam; 9. buckling; 10. a first industrial camera; 11. a first angle sensor; 12. a second industrial camera; 13. a first binocular camera box; 14. a first rotatable electric telescopic rod; 15. a third angle sensor; 16. a gun head; 17. a processing unit; 18. a controller; 19. a control cabinet.
Detailed Description
The technical scheme of the invention is further described in detail by combining the drawings and the specific embodiments in the specification.
As shown in fig. 1-5, the present invention provides a multi-mode visual servo control fire-fighting system and a control method thereof, wherein the system comprises: the system comprises an infrared near-field vision device 1, a first binocular vision device 2, a second binocular vision device 3, a fire monitor 4 and a control system;
the infrared near-field vision device 1 comprises an infrared camera 5, a near-field camera 6, a support frame 7, a support beam 8 and a buckle 9, wherein the infrared camera 5 and the near-field camera 6 are arranged above the fire monitor 4 through the support frame 7, the support beam 8 and the buckle 9;
the first binocular vision device 2 and the second binocular vision device 3 are configured identically, and each of the first binocular vision device and the second binocular vision device comprises two industrial cameras which are arranged in bilateral symmetry, an angle sensor arranged in the middle of the two industrial cameras, a binocular camera box and a rotatable electric telescopic rod, wherein the two industrial cameras and the angle sensor are arranged in the binocular camera box in parallel, the two industrial cameras and the angle sensor are arranged in the front of the side and the rear of the side of the fire monitor 4 through the rotatable electric telescopic rod, and the two binocular vision devices face in opposite directions.
A third angle sensor 15 is arranged on the fire monitor head;
the control system comprises a processing unit 17, a controller 18 and a control cabinet 19.
The respective rotatable electric telescopic rods of the binocular vision devices and the horizontal rotating shaft of the fire monitor 4 are positioned on the same straight line; when the two binocular vision devices start working, the two binocular vision devices respectively lift the electric telescopic rods of the two binocular vision devices, acquire images and rotate to align the images with a fire scene, so that the fire scene is positioned in the middle of the images; at this time, if the fire monitor 4 is not in the image of the binocular vision device, the rotatable electric telescopic rod is lowered down to stop working. The two binocular vision devices respectively and independently obtain the spatial positions (x, y, z) of the fire scene relative to the binocular vision devices;
a third angle sensor 15 positioned at the gun head 16 and an angle sensor positioned at the binocular camera box measure an included angle alpha between the binocular vision device and the central axis of the gun head 16;
the height of the binocular vision device relative to the gun head 16 is calculated through the elongation h of the rotatable electric telescopic rod 14; the height H of the fire monitor 4 is known. The processing unit 17 calculates the spatial position of the fire scene relative to the monitor head 16 and the rotation angle of the fire monitor 4 from the above information. The treatment process is as follows: as shown in fig. 8, a coordinate system one is established with the first industrial camera 10 located in the first binocular camera box 13 in the first binocular vision device as the origin and the optical axis of the first industrial camera 10 as the z-axis, the coordinates of the fire monitor under the coordinate system are (x 1, y1, z1), and the fire scene coordinates are (x, y, z); and moving the first coordinate system to the position of the fire monitor to be a second coordinate system with the fire monitor as an original point, wherein the coordinates of the fire scene under the second coordinate system are (x-x1, y-y 1, z-z 1). Under the second coordinate system, the included angle between the connecting line of the fire scene and the fire monitor and the z axis of the second coordinate system is as follows: β = arctan ((x-x 1)/(z-z 1)); therefore, the angle γ by which the fire monitor needs to be horizontally rotated is: γ = β - α.
The distance d between the fire monitor 4 and the fire scene is as follows: d =
Figure 474880DEST_PATH_IMAGE001
According to the distance d, the pitching rotation angle of the fire monitor 4 can be obtained, according to the spatial position of the fire scene relative to the monitor head 16 and the rotation angle, the controller 18 sends out an instruction to control the horizontal rotation and pitching rotation of the fire monitor, so that the fire monitor 4 is aligned to the fire scene, and the fire extinguishing agent jet flow is sprayed out; the binocular vision device 2 or 3 identifies the fire extinguishing agent jet flow track, calculates the spatial position of the fire extinguishing agent jet flow track through the processing unit 17, and judges whether the fire extinguishing agent jet flow falling point reaches a fire scene; if the fire is not reached, the controller 18 commands the fire monitor to rotate horizontally and vertically until the drop of the fire suppressant jet reaches the fire.
The infrared camera 5 of the infrared near-field vision device 1 identifies the temperature distribution in the field range, and when the highest temperature is not obviously higher than the ambient temperature, the controller 18 sends an instruction to control the fire monitor to horizontally rotate until the highest temperature is obviously higher than the ambient temperature; the processing unit 17 determines whether the highest temperature point appears in the middle area of the field of view of the infrared camera 5, and if the highest temperature point does not appear in the middle area of the field of view of the infrared camera 5, the controller 18 sends an instruction to control the fire monitor to horizontally rotate until the highest temperature point appears in the middle area of the field of view of the infrared camera 5. A near-field camera 6 of the infrared near-field vision device 1 acquires an initial section image of a fire extinguishing agent jet flow track of the fire monitor 4; the processing unit 17 predicts the linear distance between the fire extinguishing agent jet flow falling point of the fire monitor 4 and the monitor head 16 based on the initial segment image; the controller 18 issues instructions to control the fire monitor to rotate in pitch according to the predicted linear distance between the drop point of the fire extinguishing agent jet and the monitor head 16 until the predicted linear distance between the drop point of the fire extinguishing agent jet and the fire scene is zero.
The foregoing is merely a preferred embodiment of the invention and it should be noted that modifications can be made by persons skilled in the art without departing from the principles of the invention and these modifications should also be considered as within the scope of the invention.

Claims (4)

1.一种多模式视觉伺服控制消防系统,其特征在于,包括:1. a multi-mode visual servo control fire protection system, is characterized in that, comprises: 消防炮,用于向火场发射灭火剂;Fire monitor, which is used to fire extinguishing agent to the fire scene; 第一双目视觉装置,设置在所述消防炮一侧的侧前方,用于获取火场图像、以及获取消防炮与火场的空间位置,第一双目视觉装置上安装有第一角度传感器;The first binocular vision device is arranged in front of one side of the fire monitor, and is used to obtain the fire scene image and the spatial position of the fire monitor and the fire scene, and a first angle sensor is installed on the first binocular vision device; 第二双目视觉装置,设置在所述消防炮另一侧的侧后方,用于获取火场图像、以及获取消防炮与火场的空间位置,第二双目视觉装置上安装有第二角度传感器;The second binocular vision device is arranged at the side and rear of the other side of the fire monitor, and is used to obtain the fire scene image and the spatial position of the fire monitor and the fire scene, and a second angle sensor is installed on the second binocular vision device; 第三角度传感器,安装在所述消防炮的炮头上;The third angle sensor is installed on the gun head of the fire monitor; 红外近场视觉装置,用于获取火场温度图像和消防炮的射流轨迹初段图像;Infrared near-field vision device, used to obtain the temperature image of the fire field and the initial image of the jet trajectory of the fire monitor; 控制系统,与所述第一双目视觉装置、第二双目视觉装置、第一角度传感器、第二角度传感器、第三角度传感器、红外近场视觉装置以及消防炮信号连接;a control system, connected with the first binocular vision device, the second binocular vision device, the first angle sensor, the second angle sensor, the third angle sensor, the infrared near-field vision device and the fire monitor signal; 所述第一双目视觉装置和所述第二双目视觉装置结构相同,且朝向相反的安装在所述消防炮的两侧;The first binocular vision device and the second binocular vision device have the same structure and are installed on both sides of the fire monitor in opposite directions; 所述第一双目视觉装置包括:第一工业相机、第二工业相机、第一角度传感器以及第一双目相机盒,其中,所述第一工业相机、第一传感器和第二工业相机平行安装在所述第一双目相机盒内,第一双目相机盒的底部连接有第一可旋转电动伸缩杆;The first binocular vision device includes: a first industrial camera, a second industrial camera, a first angle sensor and a first binocular camera box, wherein the first industrial camera, the first sensor and the second industrial camera are parallel is installed in the first binocular camera box, and the bottom of the first binocular camera box is connected with a first rotatable electric telescopic rod; 所述第二双目视觉装置包括:第三工业相机、第四工业相机、第二角度传感器以及第二双目相机盒,其中,所述第三工业相机、第二角度传感器和第四工业相机平行安装在所述第二双目相机盒内,第二双目相机盒的底部连接有第二可旋转电动伸缩杆;The second binocular vision device includes: a third industrial camera, a fourth industrial camera, a second angle sensor and a second binocular camera box, wherein the third industrial camera, the second angle sensor and the fourth industrial camera It is installed in parallel in the second binocular camera box, and the bottom of the second binocular camera box is connected with a second rotatable electric telescopic rod; 所述第一可旋转电动伸缩杆、第二可旋转电动伸缩杆以及消防炮水平旋转轴三根轴处于同一直线上;The three axes of the first rotatable electric telescopic rod, the second rotatable electric telescopic rod and the horizontal rotation axis of the fire monitor are on the same straight line; 开始工作时,两个双目视觉装置同时升起其可旋转电动伸缩杆,采集图像并旋转对准火场,使火场处于图像中间;此时若消防炮不在其中一个双目视觉装置的图像中,则降下其可旋转电动伸缩杆,停止工作。When starting to work, the two binocular vision devices raise their rotatable electric telescopic rods at the same time, collect images and rotate to align the fire field, so that the fire field is in the middle of the image; at this time, if the fire monitor is not in the image of one of the binocular vision devices, Then lower its rotatable electric telescopic rod and stop working. 2.根据权利要求1所述的多模式视觉伺服控制消防系统,其特征在于,所述控制系统包括处理单元、控制器和控制柜。2 . The multi-mode visual servo control fire protection system according to claim 1 , wherein the control system comprises a processing unit, a controller and a control cabinet. 3 . 3.根据权利要求1所述的多模式视觉伺服控制消防系统,其特征在于,所述红外近场视觉装置包括:3. The multi-mode visual servo control fire protection system according to claim 1, wherein the infrared near-field vision device comprises: 支撑架,所述支撑架的底部通过卡扣与所述消防炮的炮头安装固定,支撑架的上部安装有红外相机,所述红外相机位于炮头的正上方,并且红外相机光轴线与消防炮炮管的轴线处于同一竖直面;A support frame, the bottom of the support frame is installed and fixed with the gun head of the fire monitor through a buckle, an infrared camera is installed on the upper part of the support frame, the infrared camera is located directly above the gun head, and the optical axis of the infrared camera is connected to the fire monitor. The axes of the gun barrels are in the same vertical plane; 支撑横梁,平行连接在所述支撑架一侧,支撑横梁上远离支撑架的一端上固定安装有近场相机。The support beam is connected parallel to one side of the support frame, and a near-field camera is fixedly installed on the end of the support beam away from the support frame. 4.一种基于权利要求1~3中任一所述多模式视觉伺服控制消防系统的工作方法,其特征在于,4. A working method based on the multi-mode visual servo control fire protection system described in any one of claims 1 to 3, characterized in that, 通过所述双目视觉装置获取火场图像并结合角度传感器获得消防炮与火场的空间位置,进而控制消防炮进行灭火,同时通过双目视觉装置识别射流轨迹并判断射流是否到达火场,具体是:The image of the fire field is obtained through the binocular vision device, and the spatial position of the fire monitor and the fire field is obtained in combination with the angle sensor, and then the fire monitor is controlled to extinguish the fire. At the same time, the jet trajectory is identified through the binocular vision device and it is judged whether the jet reaches the fire field, specifically: 第一双目视觉装置或第二双目视觉装置获得火场相对于双目视觉装置的空间位置(x,y,z);The first binocular vision device or the second binocular vision device obtains the spatial position (x, y, z) of the fire field relative to the binocular vision device; 位于炮头的第三角度传感器与位于第一双目相机盒的第一角度传感器测量第一双目视觉装置与炮头中心轴线之间的夹角α;The third angle sensor located in the gun nose and the first angle sensor located in the first binocular camera box measure the included angle α between the first binocular vision device and the central axis of the gun nose; 位于炮头的第三角度传感器与位于第二双目相机盒的第二角度传感器测量第二双目视觉装置与炮头中心轴线之间的夹角α;The third angle sensor located in the gun nose and the second angle sensor located in the second binocular camera box measure the included angle α between the second binocular vision device and the central axis of the gun nose; 以第一双目视觉装置中位于双目相机盒左侧的第一工业相机为原点、第一工业相机光轴线为z轴建立坐标系一,消防炮在坐标系一下的坐标为(x1,y1,z1),火场坐标为(x,y,z);将坐标系一移动至消防炮位置,变为以消防炮为原点的坐标系二,火场在坐标系二下的坐标为(x-x1,y- y1,z-z1),在坐标系二下,火场与消防炮连线与坐标系二的z轴之间的夹角为:β=arctan((x-x1)/(z-z1) );因此,消防炮需要水平转动的角度γ为:γ=β-α;Taking the first industrial camera on the left side of the binocular camera box in the first binocular vision device as the origin, and the optical axis of the first industrial camera as the z-axis to establish coordinate system 1, the coordinates of the fire monitor below the coordinate system are (x1, y1 , z1), the coordinates of the fire field are (x, y, z); move the coordinate system 1 to the position of the fire monitor, and change the coordinate system 2 with the fire monitor as the origin, and the coordinates of the fire field under the coordinate system 2 are (x-x1 , y- y1, z-z1), in the second coordinate system, the angle between the line connecting the fire field and the fire monitor and the z-axis of the second coordinate system is: β=arctan ((x-x1)/(z-z1 ) ); therefore, the angle γ that the fire monitor needs to rotate horizontally is: γ=β-α; 消防炮与火场的距离d通过以下公式获得:The distance d between the fire monitor and the fire field is obtained by the following formula: d=
Figure DEST_PATH_IMAGE001
,其中,
d=
Figure DEST_PATH_IMAGE001
,in,
H为消防炮的高度,h为可旋转电动伸缩杆的伸长度;H is the height of the fire monitor, h is the extension length of the rotatable electric telescopic rod; 根据距离d,即可得到消防炮的俯仰旋转角度;According to the distance d, the pitch rotation angle of the fire monitor can be obtained; 根据火场相对于炮头的空间位置,控制系统发出指令控制消防炮水平旋转和俯仰旋转,使消防炮对准火场,并喷出灭火剂射流;According to the spatial position of the fire field relative to the gun head, the control system issues commands to control the horizontal rotation and pitch rotation of the fire monitor, so that the fire monitor is aimed at the fire field and sprays out the jet of fire extinguishing agent; 双目视觉装置识别灭火剂射流轨迹,通过控制系统计算灭火剂射流轨迹的空间位置,并判断灭火剂射流落点是否到达火场;若未到达火场,控制系统发出指令控制消防炮水平旋转和俯仰旋转,直到灭火剂射流落点到达火场;The binocular vision device recognizes the fire extinguishing agent jet trajectory, calculates the spatial position of the fire extinguishing agent jet trajectory through the control system, and judges whether the fire extinguishing agent jet drop point reaches the fire scene; if it does not reach the fire scene, the control system issues commands to control the horizontal rotation and pitch rotation of the fire monitor , until the jet of fire extinguishing agent reaches the fire scene; 通过所述红外近场视觉装置中的红外相机识别视场范围内的温度分布,具体是:当最高温度未明显高于环境温度时,控制器发出指令控制消防炮水平旋转,直到最高温度明显高于环境温度;The temperature distribution within the field of view is identified by the infrared camera in the infrared near-field vision device, specifically: when the maximum temperature is not significantly higher than the ambient temperature, the controller sends an instruction to control the horizontal rotation of the fire monitor until the maximum temperature is significantly higher at ambient temperature; 控制系统判断最高温度点是否出现在红外相机视场的中部区域,若最高温度点未出现在红外相机视场的中部区域,则控制系统发出指令控制消防炮水平旋转,直至最高温度点出现在红外相机视场的中部区域;The control system determines whether the highest temperature point appears in the central area of the infrared camera's field of view. If the highest temperature point does not appear in the central area of the infrared camera's field of view, the control system sends an instruction to control the horizontal rotation of the fire monitor until the highest temperature point appears in the infrared camera. the central area of the camera's field of view; 通过红外近场视觉装置中的近场相机采集消防炮的灭火剂射流轨迹初段图像;The initial image of the fire extinguishing agent jet trajectory of the fire monitor is collected by the near-field camera in the infrared near-field vision device; 控制系统基于红外近场视觉装置采集的消防炮的灭火剂射流轨迹初段图像预测消防炮灭火剂射流落点与炮头的直线距离,并根据预测的灭火剂射流落点与炮头的直线距离,发出指令控制消防炮俯仰旋转,直到预测的灭火剂射流落点与火场的直线距离为零。The control system predicts the straight-line distance between the fire-extinguishing agent jet drop point and the gun nose of the fire monitor based on the initial image of the fire-extinguishing agent jet trajectory collected by the infrared near-field vision device, and according to the predicted straight-line distance between the fire-extinguishing agent jet drop point and the gun nose, A command is issued to control the pitch and rotation of the fire monitor until the predicted straight-line distance between the drop point of the fire extinguishing agent jet and the fire field is zero.
CN202010279393.XA 2020-04-10 2020-04-10 A multi-mode visual servo control fire protection system and its working method Active CN111494853B (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN202010279393.XA CN111494853B (en) 2020-04-10 2020-04-10 A multi-mode visual servo control fire protection system and its working method
CA3137995A CA3137995C (en) 2020-04-10 2020-07-27 Multi-mode visual servo control fire-fighting system and working method thereof
AU2020441493A AU2020441493B2 (en) 2020-04-10 2020-07-27 Multimode visual servoing firefighting system and operating method therefor
PCT/CN2020/104824 WO2021203582A1 (en) 2020-04-10 2020-07-27 Multimode visual servoing firefighting system and working method therefor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010279393.XA CN111494853B (en) 2020-04-10 2020-04-10 A multi-mode visual servo control fire protection system and its working method

Publications (2)

Publication Number Publication Date
CN111494853A CN111494853A (en) 2020-08-07
CN111494853B true CN111494853B (en) 2021-05-11

Family

ID=71848302

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010279393.XA Active CN111494853B (en) 2020-04-10 2020-04-10 A multi-mode visual servo control fire protection system and its working method

Country Status (4)

Country Link
CN (1) CN111494853B (en)
AU (1) AU2020441493B2 (en)
CA (1) CA3137995C (en)
WO (1) WO2021203582A1 (en)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111931387B (en) * 2020-09-23 2020-12-22 湖南师范大学 A Visual Servo Approaching Method for Moving Columnar Assemblies
CN112090015B (en) * 2020-10-12 2025-06-10 江西省智能产业技术创新研究院 Collaborative scheduling system and scheduling method of fire-fighting robot
CN113041537A (en) * 2021-04-14 2021-06-29 中国矿业大学 Fire monitor system with variable-visual-angle binocular structure and method
CN113274672A (en) * 2021-04-27 2021-08-20 中国矿业大学 Fire monitor hybrid control system and control method based on machine vision
CN113382143B (en) * 2021-06-09 2022-07-29 中国矿业大学 Automatic exposure adjusting method for binocular camera of fire-fighting robot
CN113440763B (en) * 2021-06-15 2022-09-16 中国矿业大学 An intelligent fire protection system for forest fire prevention and its working method
CN114272546A (en) * 2021-12-30 2022-04-05 深圳市无限动力发展有限公司 Fire fighting method, device, equipment and storage medium based on sweeper platform
CN114882663B (en) * 2022-06-08 2025-06-24 徐俊亮 A fire prediction and early warning device for agroforestry-grass composite area and its early warning method
CN115364401A (en) * 2022-08-15 2022-11-22 山东瑞美油气装备技术创新中心有限公司 Method and device for extinguishing fire
CN115337581B (en) * 2022-10-14 2022-12-27 中国矿业大学 Fire-fighting method based on multi-view vision fire-fighting system
CN116115933A (en) * 2023-02-08 2023-05-16 广西北投信创科技投资集团有限公司 Charging pile fire-fighting system and method
CN116712705B (en) * 2023-07-11 2024-07-19 广州城市理工学院 Inspection robot working method with rotatable fire extinguisher bracket
CN118161813A (en) * 2024-05-11 2024-06-11 徐州徐工道金特种机器人技术有限公司 Fire extinguishing control method, device, system, fire fighting equipment and computer products
CN119656523A (en) * 2024-12-24 2025-03-21 湖南中联重科应急装备有限公司 Fire extinguishing control method for elevating fire truck, fire extinguishing system and elevating fire truck

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101574566A (en) * 2009-06-08 2009-11-11 南京航空航天大学 Monocular vision technique based fire monitor control method for adjusting relative positions of fire point and water-drop point
CN103528520A (en) * 2013-10-08 2014-01-22 哈尔滨工业大学 Detection device and method for synchronous operation jacking system based on binocular vision
CN107909615A (en) * 2017-12-18 2018-04-13 广东广业开元科技有限公司 A kind of fire monitor localization method based on binocular vision
CN109331376A (en) * 2018-10-12 2019-02-15 中国矿业大学 Urban main battle fire truck automatic fire extinguishing system and realization method
CN110860057A (en) * 2019-11-18 2020-03-06 燕山大学 A kind of fire reconnaissance robot and reconnaissance method
US10605567B1 (en) * 2018-09-19 2020-03-31 Steven T. Hartman Sighting device for handheld mortar system
CN210212803U (en) * 2019-05-21 2020-03-31 广西电网有限责任公司百色供电局 Three-span line intelligent space distance calculating device

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101574568B (en) * 2009-06-08 2011-08-24 南京航空航天大学 Identification method of fire monitor waterway and waterway end
CN102836514B (en) * 2012-09-28 2015-03-04 章丘市消防器材有限责任公司 Automatic tracking and positioning jet fire extinguishing system
DE102016104349B3 (en) * 2016-03-10 2017-03-02 Albert Orglmeister Method for improving the accuracy of targeted at extinguishing systems controlled by infrared and video early fire detection
CN107192343B (en) * 2017-07-04 2023-08-18 华中科技大学 Six-degree-of-freedom displacement measuring device and method for suspension characteristic test wheel
JP6945423B2 (en) * 2017-11-27 2021-10-06 ホーチキ株式会社 Water discharge type fire extinguishing equipment
KR102033498B1 (en) * 2017-12-28 2019-10-17 (주)아이아이에스티 Fire extinguishing system, method, program and computer readable medium for controlling of the same
CN109303995A (en) * 2018-09-12 2019-02-05 东南大学 Control method of fire-fighting robot fire monitor based on fire source location identification
CN109785574B (en) * 2019-01-21 2021-03-26 五邑大学 Fire detection method based on deep learning
CN110812745A (en) * 2019-11-18 2020-02-21 燕山大学 A mobile intelligent fire-fighting robot and fire-fighting control method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101574566A (en) * 2009-06-08 2009-11-11 南京航空航天大学 Monocular vision technique based fire monitor control method for adjusting relative positions of fire point and water-drop point
CN103528520A (en) * 2013-10-08 2014-01-22 哈尔滨工业大学 Detection device and method for synchronous operation jacking system based on binocular vision
CN107909615A (en) * 2017-12-18 2018-04-13 广东广业开元科技有限公司 A kind of fire monitor localization method based on binocular vision
US10605567B1 (en) * 2018-09-19 2020-03-31 Steven T. Hartman Sighting device for handheld mortar system
CN109331376A (en) * 2018-10-12 2019-02-15 中国矿业大学 Urban main battle fire truck automatic fire extinguishing system and realization method
CN210212803U (en) * 2019-05-21 2020-03-31 广西电网有限责任公司百色供电局 Three-span line intelligent space distance calculating device
CN110860057A (en) * 2019-11-18 2020-03-06 燕山大学 A kind of fire reconnaissance robot and reconnaissance method

Also Published As

Publication number Publication date
CA3137995A1 (en) 2021-10-14
AU2020441493A1 (en) 2021-11-18
WO2021203582A1 (en) 2021-10-14
CA3137995C (en) 2023-09-26
CN111494853A (en) 2020-08-07
AU2020441493B2 (en) 2024-01-25

Similar Documents

Publication Publication Date Title
CN111494853B (en) A multi-mode visual servo control fire protection system and its working method
US8973671B2 (en) Smart compact indoor firefighting robot for extinguishing a fire at an early stage
CN113440763B (en) An intelligent fire protection system for forest fire prevention and its working method
CN105031868B (en) Adaptive extinguishing method based on flame scale
CN103285548B (en) Method and device for positioning ground fire by monocular camera
CN106843240A (en) System is coordinated by fire prevention robot and multimachine based on intelligent video
CN109303995A (en) Control method of fire-fighting robot fire monitor based on fire source location identification
CN104952201A (en) A video fire alarm monitoring system and method
CN106943693A (en) A kind of directing self-extinguishing module
CN114307016B (en) Fire extinguishing system based on unmanned aerial vehicle vision auxiliary fire monitor and control method
WO2022000279A1 (en) Fire extinguishing system, server, fire-fighting robot, and fire extinguishing method
CN113829340A (en) Transformer substation fire fighting method based on inspection robot
CN110180114B (en) Fire-fighting robot cooperative positioning, reconnaissance, fire source identification and aiming fire-extinguishing method
WO2022217671A1 (en) Fire monitor system having variable-viewing-angle binocular structure, and method
CN111388912B (en) Directional intelligent fire extinguishing system for high-speed rail motor train unit
CN113240881B (en) Fire identification system based on multi-feature fusion
CN110837822A (en) Fire-fighting robot injection curve adjusting method and device based on multi-view vision
CN114870313A (en) Intelligent fire-fighting comprehensive monitoring system
RU2411974C1 (en) Method of automatic fire extinguishing and automatic system for its realisation
CN113521616A (en) Fire-fighting robot, scheduling method and fire extinguishing system
CN219275821U (en) A normalized fire inspection, early warning, fire rescue platform based on heterogeneous robot teamwork
CN206788648U (en) System is coordinated by fire prevention robot and multimachine based on intelligent video
CN115337581B (en) Fire-fighting method based on multi-view vision fire-fighting system
KR20200113391A (en) Autonomous driving drone for fire extinguish and method for fire suppression using the same
Gao et al. Design and experimental verification of an intelligent fire-fighting robot

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant