CN116343102A - Underground personnel safety early warning method, device, system and storage medium - Google Patents
Underground personnel safety early warning method, device, system and storage medium Download PDFInfo
- Publication number
- CN116343102A CN116343102A CN202310619391.4A CN202310619391A CN116343102A CN 116343102 A CN116343102 A CN 116343102A CN 202310619391 A CN202310619391 A CN 202310619391A CN 116343102 A CN116343102 A CN 116343102A
- Authority
- CN
- China
- Prior art keywords
- personnel
- underground
- coordinates
- warned
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K17/00—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
- G06K17/0022—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations arrangements or provisions for transferring data to distant stations, e.g. from a sensing device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/02—Agriculture; Fishing; Forestry; Mining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- General Business, Economics & Management (AREA)
- General Health & Medical Sciences (AREA)
- Marketing (AREA)
- Tourism & Hospitality (AREA)
- Entrepreneurship & Innovation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Educational Administration (AREA)
- General Engineering & Computer Science (AREA)
- Game Theory and Decision Science (AREA)
- Human Computer Interaction (AREA)
- Social Psychology (AREA)
- Psychiatry (AREA)
- Operations Research (AREA)
- Development Economics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Agronomy & Crop Science (AREA)
- Animal Husbandry (AREA)
- Marine Sciences & Fisheries (AREA)
- Mining & Mineral Resources (AREA)
- Primary Health Care (AREA)
- Alarm Systems (AREA)
Abstract
The embodiment of the specification discloses a method, a device and a system for pre-warning the safety of underground personnel. The method comprises the following steps: based on the video acquired by visual AI analysis, determining video coordinates of underground personnel in a video picture, and marking personnel to be early-warned and corresponding behavior information which accord with early-warning conditions; based on a wireless positioning identification result, obtaining identity information of the underground personnel and real position coordinates of the underground personnel in a positioning coordinate system; mapping the video coordinates of the underground personnel to the positioning coordinate system, matching the video coordinates with the real position coordinates of the underground personnel, and determining the identity information and the position information of the personnel to be pre-warned; and reporting the identity information, the position information and the behavior information of the personnel to be early-warned. According to the scheme of the embodiment of the specification, the identity, the position and the behavior of the personnel to be early-warned can be accurately identified in a scene of multiple persons under the mine, the safety risk of the personnel under the mine can be timely found, and the purpose of early-warning in advance is achieved.
Description
Technical Field
The specification relates to the technical field of coal mine safety, in particular to a method, a device, a system, equipment and a storage medium for pre-warning of underground personnel safety.
Background
Currently, mine safety supervision mode is gradually changed to a remote, intelligent, visual and 'internet + supervision' mode. The vision-based AI analysis is gradually applied to underground, the AI vision technology is utilized to train various underground operation scenes, intelligent matching is carried out on various illegal scenes, the purposes of early warning in advance and avoiding safety accidents can be achieved, and the important disaster risk prevention and control capability of mines can be improved.
The downhole environment is complex and personnel identity and behavior need to be monitored and identified in order to monitor violations or abnormal behaviors of personnel downhole. CN202210523280.9 proposes to identify the face and voiceprint of the personnel downhole to determine the identity of the personnel. However, under mine illumination conditions, faces cannot be seen clearly in many cases, as shown in fig. 1, and in addition, the working environment is complex, the faces of workers are often polluted seriously by dust, as shown in fig. 2, even in places with serious dust, workers still need to wear mask to work, and under these scenes, identifiable face information is difficult to collect. Meanwhile, since the underground operation is usually accompanied by great noise, and workers do not make sound to perform the operation in most scenes, the identification of personnel by means of voiceprints is also limited. Thus, determining the identity of a person by means of face recognition and voiceprint recognition presents difficulties in an under-mine environment.
Based on the above, a feasible identification method is provided for safety precaution aiming at the identity and behavior of underground personnel.
Disclosure of Invention
The embodiment of the specification provides an underground personnel safety early warning method, which is used for solving the following technical problems: in a multi-person scene under an mine, how to accurately identify the identity of the person and recognize the behavior of the person so as to facilitate safety early warning and personnel management.
In order to solve the above technical problems, the embodiments of the present specification are implemented as follows:
the embodiment of the specification provides an underground personnel safety early warning method, which comprises the following steps:
based on the video acquired by visual AI analysis, determining video coordinates of underground personnel in a video picture, and marking personnel to be early-warned and corresponding behavior information which accord with early-warning conditions;
based on a wireless positioning identification result, obtaining identity information of the underground personnel and real position coordinates of the underground personnel in a positioning coordinate system;
mapping the video coordinates of the underground personnel to the positioning coordinate system, matching the video coordinates with the real position coordinates of the underground personnel, and determining the identity information and the position information of the personnel to be pre-warned;
and reporting the identity information, the position information and the behavior information of the personnel to be early-warned.
According to the method of the embodiment of the present disclosure, the marking the person to be pre-warned and the corresponding behavior information meeting the pre-warning condition includes:
and identifying behaviors of underground personnel according to comparison results of human body features extracted from the acquired video data and a pre-stored human body feature library, determining personnel meeting early warning conditions as personnel to be early warned, and marking behavior information of the personnel to be early warned.
According to the method of the embodiment of the present specification, the early warning condition includes: the personnel under the mine have illegal operation behaviors and/or abnormal physical behaviors.
According to the method of the embodiment of the present disclosure, based on the wireless positioning recognition result, obtaining the identity information of the underground personnel and the actual position coordinates of the underground personnel in the positioning coordinate system includes:
based on information interaction between the UWB base station and the UWB tag card carried by the underground personnel, identification information of the underground personnel carrying the tag card and real position coordinates of the underground personnel in a positioning coordinate system are identified.
According to the method of the embodiment of the present disclosure, mapping the video coordinates of the underground personnel to the positioning coordinate system, and matching with the real position coordinates of the underground personnel, to determine the identity information and the position information of the personnel to be pre-warned, includes:
mapping video coordinates marked as a person to be pre-warned in the underground personnel into the positioning coordinate system to obtain projection coordinates of the person to be pre-warned in the positioning coordinate system, determining a real position coordinate closest to the projection coordinates as position information of the person to be pre-warned, and determining identity information of the person corresponding to the real position coordinate as identity information of the person to be pre-warned.
According to the method of the embodiment of the present disclosure, mapping the video coordinates of the underground personnel to the positioning coordinate system, and matching with the real position coordinates of the underground personnel, to determine the identity information and the position information of the personnel to be pre-warned, includes:
mapping the video coordinates of the underground personnel to the positioning coordinate system to obtain projection coordinates of the underground personnel in the positioning coordinate system, matching the projection coordinates with the real position coordinates of the underground personnel according to the azimuth relation of the underground personnel, and determining the identity information and the position information of the personnel to be pre-warned.
The embodiment of the specification also provides a personnel safety precaution device under ore deposit, the device includes:
the visual AI analysis module is configured to determine video coordinates of underground personnel in a video picture based on the video acquired by visual AI analysis, and mark the personnel to be early-warned and corresponding behavior information which accord with early-warning conditions;
the wireless identification result acquisition module is configured to acquire the identity information of the underground personnel and the real position coordinates of the underground personnel in a positioning coordinate system based on the wireless positioning identification result;
the information determining module is configured to map the video coordinates of the underground personnel into the positioning coordinate system, match the video coordinates with the real position coordinates of the underground personnel and determine the identity information and the position information of the personnel to be early-warned;
and the early warning reporting module is configured to report the identity information, the position information and the behavior information of the personnel to be early warned.
The embodiment of the specification also provides an underground personnel safety early warning system, which comprises a behavior early warning subsystem, a wireless positioning and identifying subsystem and video acquisition equipment, wherein,
the behavior early warning subsystem comprises the early warning device, and is used for determining identity information, position information and behavior information of a person to be early warned and reporting the identity information, the position information and the behavior information of the person to be early warned;
the wireless positioning and identifying subsystem consists of a plurality of UWB positioning base stations arranged under the mine and UWB tag cards carried by underground personnel, and is used for measuring and calculating the real position coordinates of the underground personnel in a positioning coordinate system and reading the identity information in the UWB tag cards;
the video acquisition equipment is arranged under the mine and is used for acquiring video data of personnel under the mine.
The embodiment of the specification also provides a safety precaution device for underground personnel, comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor;
wherein,,
the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to perform the method as previously described.
The present description also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to implement a method as described above.
The above-mentioned at least one technical scheme that this description embodiment adopted can reach following beneficial effect: the personnel behaviors are identified through visual detection, the personnel identities and the personnel positions identified by the wireless tag card are combined, the personnel behaviors are associated with specific personnel, the identities, the positions and the behaviors of the personnel to be early-warned can be accurately identified in a scene of multiple persons under the mine, the safety risks of the personnel under the mine can be timely found, the purpose of early warning in advance is achieved, the standardized management of the personnel under the mine is facilitated, and then the disaster risk prevention and control capability of a mining area is improved.
Drawings
In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some of the embodiments described in the present description, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a dim light scene under a mine;
FIG. 2 is a face of a worker under a mine;
FIG. 3 is a flow chart of an early warning method according to an embodiment of the present disclosure;
FIG. 4 is a diagram of the visual AI detection result;
FIG. 5 is a schematic illustration of UWB positioning principles;
FIG. 6 is a schematic diagram of a video frame projected onto a positioning plane;
FIG. 7 is a schematic diagram of the relationship between projection coordinates and real position coordinates;
FIG. 8 is a block diagram of an underground personnel safety precaution device provided in an embodiment of the present disclosure;
fig. 9 is a block diagram of an underground personnel safety early warning system according to an embodiment of the present disclosure.
Detailed Description
In order to make the technical solutions in the present specification better understood by those skilled in the art, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.
In order to suppress serious accidents of mines and prevent and resolve great safety risks, in recent years, a new generation of information technology is actively utilized by coal mines and key non-coal mines, real-time identification, monitoring and accurate research and judgment of the important safety risks of key places and key positions of the coal mines and the key non-coal mines are promoted, the mine safety supervision and supervision mode is changed to a remote, intelligent and visual mode and an Internet+supervision mode, the mine safety supervision and supervision law enforcement efficiency is improved, and the mine digital and intelligent safety production level is continuously improved.
Through the video equipment deployed under the mine, the operation environment under the mine can be remotely monitored, and personnel under the mine can be remotely supervised and monitored. Wherein, the scene of supervision to personnel under the ore deposit includes: whether an operator wears protective equipment such as a safety helmet, rubber shoes and the like according to safety regulations, whether a person falls to the ground or calls for help in mine, whether the person passes through a non-safety area in a management and control time period and the like.
When illegal or abnormal behaviors of underground personnel in the monitoring range are found, early warning is sent out, and risk prevention and control capability can be improved to a certain extent. In order to improve the effect of safety supervision, the identity and specific position of the miners with violations or abnormal behaviors need to be identified, and the requirements of safety prevention and control and personnel management of mining areas can be effectively met by combining corresponding management means.
However, the environment under the mine is bad, and the technical means of face recognition which can be implemented on the ground are poor in applicability under the mine. Such as the dim light scene under the mine shown in fig. 1, the acquired person image is difficult to recognize the facial features under such light conditions. As shown in fig. 2, the face of the worker is seriously contaminated by dust, and the worker needs to wear a mask in some areas. In such a case, it is difficult to determine the identity of the mineworker by face recognition through image video.
Therefore, aiming at the requirement of safety supervision under the mine, the inventor provides the technical scheme of the invention by combining the actual situation and combining the wireless positioning technology on the basis of the visual AI recognition technology so as to meet the requirement of omnibearing safety supervision on personnel under the mine.
Embodiments of the technical scheme of the present invention are specifically described below with reference to the accompanying drawings.
Fig. 3 schematically shows a flow chart of a method of underground personnel safety precaution.
As shown in FIG. 3, the method for pre-warning the safety of underground personnel provided in the embodiment of the present disclosure includes operations S310 to S340.
S310, video coordinates of underground personnel in a video picture are determined based on videos acquired by visual AI analysis, and the personnel to be pre-warned and corresponding behaviors which accord with pre-warning conditions are marked.
Vision-based AI detection requires real-time analysis of video data acquired by a video device. And identifying behaviors of underground personnel according to comparison results of human body features extracted from the acquired video data and a pre-stored human body feature library, determining personnel meeting early warning conditions as personnel to be early warned, and marking behavior information of the personnel to be early warned. During detection, human body images are detected from video images by utilizing a pre-trained AI model, human body characteristics are extracted based on analysis of the human body images, and the human body characteristics are compared with data of a pre-stored human body characteristic database to identify behaviors to be pre-warned, such as behaviors without wearing a safety helmet. When the human body image is determined to have the behavior to be pre-warned, the person represented by the human body image is determined to be the person to be pre-warned, and the video coordinates of the person in the video picture are recorded.
Fig. 4 is a schematic diagram of a visual AI detection result, where the AI identifies three miners from the video frame and marks the coordinate positions of the heads of the three miners in the video frame, and the origin of coordinates of the video frame is located in the upper left corner. If the behavior of the miner to be early-warned is identified, the miner is taken as the worker to be early-warned, and in the link, the number of the detected workers to be early-warned in one picture may be multiple. Depending on the setting, the AI may also mark the coordinate position of other body parts of the mineworker, such as the top of the head, the lumbar, the left arm, etc.
The preset early warning conditions comprise: the personnel under the mine have illegal operation behaviors and/or abnormal physical behaviors. The pre-warning conditions are related to specific environmental and management requirements, and in some embodiments, the offending operation behavior is specifically: the worker does not wear the safety helmet according to the safety regulations, does not wear the rubber shoes correctly and does not wear the protective equipment correctly; specific morphological abnormal behaviors are: in the inverted or distress state, entering or traversing unsafe areas during the regulatory period.
S320, based on the wireless positioning recognition result, obtaining the identity information of the underground personnel and the real position coordinates of the underground personnel in a positioning coordinate system.
The wireless tag card can be carried when the miners enter under the mine, the identity information of the miners is written on the tag card, and the identity of the miners can be identified by reading the tag card through the wireless base station arranged under the mine. The wireless tag card may be a UWB tag card, a 5G terminal, zigBee, or other radio frequency tag card. The real position coordinates of the underground personnel can be obtained by combining the wireless tag card for positioning, and the real position coordinates are the positions of the underground personnel in a two-dimensional plane or the positions of the underground personnel in a three-dimensional space.
In the embodiment of the wireless tag card being a UWB tag card, obtaining the identity information of the underground personnel and the real position coordinates of the underground personnel in the positioning coordinate system based on the wireless positioning recognition result includes: based on information interaction between the UWB base station and UWB tag cards carried by underground personnel, identification information of the underground personnel carrying the tag cards and personnel positions of the underground personnel in a positioning coordinate system are identified. The positioning precision of UWB can reach centimeter level, and the positioning result can be used as the actual position of miner. In the embodiment that the wireless tag card is a 5G terminal, the identity and the corresponding position of the underground personnel are obtained through information interaction between the 5G base station arranged under the mine and the 5G terminal carried by the underground personnel.
The basic principle of UWB positioning is that the tag card sends data frames to the positioning base station, calculates the time required by data transmission to the base station, and multiplies the time with the speed of light to calculate the distance between the tag card and the base station. And measuring and calculating the distance by using a plurality of base stations, and obtaining the position of the tag by using the trilateral positioning principle. As shown in fig. 5, the positions of the four UWB base stations are fixed and known, and when the distance between each base station and the UWB tag is measured, the position of the tag in the two-dimensional plane can be determined by trilateration.
In some embodiments, three-dimensional coordinates of the mineworker may also be obtained by means of three-dimensional positioning techniques, such as applying lidar techniques for three-dimensional positioning. The three-dimensional positioning result comprises the height position of the miners, so that more accurate safety early warning can be obtained, and the miners at different heights need to be distinguished to determine the identity in the slope environment.
S330, mapping the video coordinates of the underground personnel to the positioning coordinate system, and matching with the real position coordinates of the underground personnel to determine the identity information and the position information of the personnel to be early-warned.
The environment under the mine is special, and the identity of the person to be early-warned is difficult to confirm through face recognition or other visual detection means. In this embodiment, the wireless identification technology is used to identify and locate the underground personnel. In order to match the identities of the people to be pre-warned one by one, a mapping relation between the video picture and the real position coordinates of the people in the positioning coordinate system is established. In the operation of S330, the video coordinates of the underground personnel in the video frame are projected to the actual space where the underground personnel are located, and the identity information and the actual position of the personnel in the video frame are matched in combination with the wireless positioning recognition result.
Specifically, the identity information and the actual position of the person in the video picture can be matched in two ways.
Firstly, mapping video coordinates marked as a person to be pre-warned in underground personnel into a positioning coordinate system to obtain projection coordinates of the person to be pre-warned in the positioning coordinate system, determining a real position coordinate closest to the projection coordinates as position information of the person to be pre-warned, and determining identity information of the person corresponding to the real position coordinate as identity information of the person to be pre-warned. The method only projects the video coordinates of the personnel to be early-warned into the positioning coordinate system, the calculated amount is small, the matching is also relatively simple through the distance relation between the projection coordinates and the coordinates of the real position coordinates, and an accurate matching result can be obtained in a scene with more scattered personnel.
Secondly, mapping video coordinates of underground personnel into a positioning coordinate system to obtain projection coordinates of the underground personnel in the positioning coordinate system, determining that the position information of the personnel to be pre-warned is matched with the real position coordinates of the underground personnel according to the azimuth relation of the underground personnel, and determining the identity information and the position information of the personnel to be pre-warned. The method needs to project video coordinates of all underground workers into a positioning coordinate system, all underground workers comprise the workers to be pre-warned, the positions of a plurality of underground workers are utilized to construct an azimuth relation, the calculated amount is large, but the video positioning system is insensitive to errors, and the matching accuracy is high.
The principles of video coordinate mapping and person matching will be described below using an embodiment in which wireless location identification is performed using a UWB tag card and a UWB location base station.
As shown in fig. 6, above, there is a video frame 610, below, there is a ground plane 620, i.e., a positioning plane, and the origin O of the video frame 610 is a fixed point, and its projection on the ground plane is O1. The angle α between the video frame 610 and the ground plane 620 is a fixed value in the case of a camera of the video device. For convenience of explanation, three projection areas 621, 622, 623 are respectively obtained by projecting three head image areas 611, 612, 613 of a miner in the video screen 610 onto the ground plane 620 below, and the coordinates of the three projection areas are projection coordinates. If the actual size coordinate of the projector needs to be obtained, the projection coordinate needs to be amplified by k times, and the k value is the ratio of the size of an object in the video picture to the size of the actual object and is related to parameters such as the focal length of the video equipment. In this embodiment, personnel matching can be accomplished without enlarging the projection coordinates.
The projection coordinates reflect the position of the person in the video frame projected in real space (i.e. the positioning coordinate system), and the wireless positioning recognition results in the actual position of the miners, and ideally, the projection coordinates are consistent with the actual position. In practical situations, due to the influences of factors such as video equipment, shooting conditions, practical environments, and different positions of the tag on the miner, errors exist between projection coordinates and practical positions.
As shown in fig. 7, 621, 622, 623 are projection coordinates of three miners, 711, 712, 713 are real position coordinates of three miners obtained by the wireless positioning system, and the projection coordinates are compared with the real position coordinates to determine the real position coordinates of the projection coordinates, so as to obtain identity information thereof. In fig. 7, 621 and 711 are closest to each other, 622 and 712 are closest to each other, 623 and 713 are closest to each other, and the projected coordinates and the actual position coordinates of the person can be matched according to the distance relation, that is, in the first manner described above. In the second method, in fig. 7, since one miner is located on each of the left and right sides of 622 and one miner is located on each of the left and right sides of 712, it can be determined that 622 corresponds to 712, 621 corresponds to 712, and 623 corresponds to 713.
In the case of positioning using a UWB tag, the positioning result is the coordinates of one point. In order to facilitate the comparison with the projection coordinates of the video coordinates, the head image or a part of the image of the human body can be selected for projection, and a position point in the human body can also be selected for projection. In practice, in order to reduce errors, miners may be required to uniformly fix the wireless tag card on the miner's lamp at the head, or uniformly fix the wireless tag card between the waists, and the video coordinates may select an image or a point of the position where the wireless tag card is located to perform projection. That is, mapping video coordinates of the underground personnel to the positioning coordinate system specifically includes: and mapping an image area of the position of the wireless tag card carried by the underground personnel into a positioning coordinate system by taking the image area as a video coordinate. For example, if the UWB tag card is provided on the miner's lamp, the image area of the miner's lamp recognized at the time of video detection is projected. The image area refers to an area containing a plurality of pixels, and covers the case where there is only one image pixel, that is, the case where only one pixel representing a person under mine is projected into a positioning coordinate system.
When only one miner exists in the video picture, the violation or the abnormal behavior of the miner is detected, and meanwhile, the wireless positioning result only identifies one miner, the identification result of the wireless positioning can be directly bound with the miner in the video picture, so that the identity information and the actual position of the miner in the video picture are determined. When a plurality of miners exist in the video picture and the number of the personnel to be early-warned with violations or abnormal behaviors is also a plurality of, the actual position and identity information of each personnel to be early-warned need to be matched from the actual positions of the miners. That is, in a multi-person scene, the video AI detects M people to be pre-warned from N underground people, wherein M is more than or equal to 2, and N is more than or equal to M. The video coordinates are mapped to a positioning coordinate system, and the positions and the identity information of M persons to be pre-warned are matched one by one from the true position coordinates of N persons under the mine.
The coordinate mapping is used for establishing the connection between the personnel in the video picture and the on-site personnel, so that the identity of the personnel to be pre-warned can be accurately associated, and the defect that the identity is difficult to confirm through face recognition under the mine is overcome.
And S340, reporting the identity information, the position information and the behavior information of the personnel to be early-warned.
The early warning results including personnel identity, position and behavior information are obtained through the steps, the information is reported to a server, the information can be displayed to a remote client, and early warning is carried out in parallel with early warning equipment, for example, on-site broadcasting prompt is carried out, or the information is directly prompted to a miner to be early warned through other equipment.
According to the technical scheme of the embodiment of the specification, the personnel behaviors are identified through visual detection, the personnel identities and the personnel positions identified by the wireless tag card are combined, the personnel behaviors are associated with specific personnel, the identities, the positions and the behaviors of the personnel to be early-warned can be accurately identified in a scene of multiple persons under the mine, the safety risks of the personnel under the mine are timely found, the purpose of early warning in advance is achieved, the standard management of the personnel under the mine is facilitated, and then disaster risk prevention and control capability of a mining area is improved.
Based on the same inventive concept, the disclosure also provides an underground personnel safety pre-warning device 800. The warning device 800 will be described with reference to fig. 8.
Fig. 8 schematically illustrates a block diagram of a security early warning device 800 according to an embodiment of the disclosure. The early warning device 800 may be implemented as part or all of an electronic device by software, hardware, or a combination of both.
As shown in fig. 8, the early warning device 800 includes a visual AI analysis module 810, a wireless recognition result acquisition module 820, an information determination module 830, and an early warning reporting module 840.
The visual AI analysis module 810 is configured to determine video coordinates of underground personnel in a video picture based on the video acquired by visual AI analysis, and mark the personnel to be early-warned and corresponding behavior information which accord with early-warning conditions;
a wireless identification result obtaining module 820 configured to obtain identity information of the underground personnel and real position coordinates of the underground personnel in a positioning coordinate system based on a wireless positioning identification result;
the information determining module 830 is configured to map the video coordinates of the underground personnel to the positioning coordinate system, and match with the real position coordinates of the underground personnel to determine the identity information and the position information of the personnel to be pre-warned;
the early warning reporting module 840 is configured to report the identity information, the position information and the behavior information of the person to be early warned.
The underground personnel safety precaution device 800 can execute any safety precaution method as described above, and the description thereof is omitted herein.
According to the safety early warning device, the personnel behaviors are identified through visual detection, the personnel identities and the personnel positions identified by the wireless tag card are combined, the personnel behaviors are associated with specific personnel, the identities, the positions and the behaviors of the personnel to be early warned can be accurately identified in a scene of multiple persons under the mine, the safety risks of the personnel under the mine are timely found, and the purpose of early warning in advance is achieved.
Based on the same inventive concept, the disclosure also provides an underground personnel safety early warning system 900. Fig. 9 schematically illustrates a block diagram of a security early warning system 900 according to an embodiment of the disclosure. The early warning system 900 may be implemented as part or all of an electronic device by software, hardware, or a combination of both.
As shown in fig. 9, the pre-warning system 900 includes a behavior pre-warning subsystem 910, a wireless location identification subsystem 920, and a video acquisition device 930.
The behavior early-warning subsystem 910 includes the early-warning device 800, and is configured to determine identity information, position information, and behavior information of a person to be early-warned and report the information.
The wireless positioning and identifying subsystem 920 is composed of a plurality of UWB positioning base stations arranged under the mine and UWB tag cards carried by underground personnel, and is used for measuring and calculating the real position coordinates of the underground personnel in a positioning coordinate system and reading the identity information in the UWB tag cards.
The video acquisition device 930 is disposed under the mine and is configured to provide the collected video data of the person under the mine to the behavior early-warning subsystem 910.
According to the safety early warning system disclosed by the embodiment of the disclosure, the wireless positioning recognition subsystem is used for providing personnel identity and position information for the behavior early warning subsystem, and the video acquisition equipment is used for providing video data for the behavior early warning subsystem so as to be used for AI detection. The safety early warning system can timely find the safety risk of the personnel under the mine, achieves the purpose of early warning in advance, is beneficial to promoting the standard management of the personnel under the mine, and further improves the disaster risk prevention and control capability of the mine.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus, electronic devices, non-volatile computer storage medium embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to the description of the method embodiments.
The apparatus, the electronic device, the nonvolatile computer storage medium and the method provided in the embodiments of the present disclosure correspond to each other, and therefore, the apparatus, the electronic device, the nonvolatile computer storage medium also have similar beneficial technical effects as those of the corresponding method, and since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, the electronic device, the nonvolatile computer storage medium are not described here again.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing one or more embodiments of the present description.
It will be appreciated by those skilled in the art that the present description may be provided as a method, system, or computer program product. Accordingly, the present specification embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description embodiments may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data optimization device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data optimization device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data optimization device to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data-optimizing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing is merely exemplary embodiments of the present disclosure and is not intended to limit the present disclosure. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.
Claims (10)
1. An underground personnel safety early warning method is characterized by comprising the following steps:
based on the video acquired by visual AI analysis, determining video coordinates of underground personnel in a video picture, and marking personnel to be early-warned and corresponding behavior information which accord with early-warning conditions;
based on a wireless positioning identification result, obtaining identity information of the underground personnel and real position coordinates of the underground personnel in a positioning coordinate system;
mapping the video coordinates of the underground personnel to the positioning coordinate system, matching the video coordinates with the real position coordinates of the underground personnel, and determining the identity information and the position information of the personnel to be pre-warned;
and reporting the identity information, the position information and the behavior information of the personnel to be early-warned.
2. The method of claim 1, wherein the marking the person to be pre-warned and the corresponding behavior information that meet the pre-warning condition comprises:
and identifying behaviors of underground personnel according to comparison results of human body features extracted from the acquired video data and a pre-stored human body feature library, determining personnel meeting early warning conditions as personnel to be early warned, and marking behavior information of the personnel to be early warned.
3. The method of claim 1, wherein the pre-warning condition comprises: the personnel under the mine have illegal operation behaviors and/or abnormal physical behaviors.
4. The method according to claim 1, wherein the obtaining the identity information of the underground person and the actual position coordinates of the underground person in the positioning coordinate system based on the wireless positioning recognition result includes:
based on information interaction between the UWB base station and the UWB tag card carried by the underground personnel, identification information of the underground personnel carrying the tag card and real position coordinates of the underground personnel in a positioning coordinate system are identified.
5. The method according to claim 1, wherein the mapping the video coordinates of the underground personnel to the positioning coordinate system and matching with the real position coordinates of the underground personnel to determine the identity information and the position information of the personnel to be pre-warned includes:
mapping video coordinates marked as a person to be pre-warned in the underground personnel into the positioning coordinate system to obtain projection coordinates of the person to be pre-warned in the positioning coordinate system, determining a real position coordinate closest to the projection coordinates as position information of the person to be pre-warned, and determining identity information of the person corresponding to the real position coordinate as identity information of the person to be pre-warned.
6. The method according to claim 1, wherein the mapping the video coordinates of the underground personnel to the positioning coordinate system and matching with the real position coordinates of the underground personnel to determine the identity information and the position information of the personnel to be pre-warned includes:
mapping the video coordinates of the underground personnel to the positioning coordinate system to obtain projection coordinates of the underground personnel in the positioning coordinate system, matching the projection coordinates with the real position coordinates of the underground personnel according to the azimuth relation of the underground personnel, and determining the identity information and the position information of the personnel to be pre-warned.
7. An underground personnel safety precaution device, characterized in that the device includes:
the visual AI analysis module is configured to determine video coordinates of underground personnel in a video picture based on the video acquired by visual AI analysis, and mark the personnel to be early-warned and corresponding behavior information which accord with early-warning conditions;
the wireless identification result acquisition module is configured to acquire the identity information of the underground personnel and the real position coordinates of the underground personnel in a positioning coordinate system based on the wireless positioning identification result;
the information determining module is configured to map the video coordinates of the underground personnel into the positioning coordinate system, match the video coordinates with the real position coordinates of the underground personnel and determine the identity information and the position information of the personnel to be early-warned;
and the early warning reporting module is configured to report the identity information, the position information and the behavior information of the personnel to be early warned.
8. An underground personnel safety early warning system is characterized by comprising a behavior early warning subsystem, a wireless positioning and identifying subsystem and video acquisition equipment, wherein,
the behavior early warning subsystem comprises the early warning device according to claim 7, and is used for determining the identity information, the position information and the behavior information of the personnel to be early warned and reporting the information;
the wireless positioning and identifying subsystem consists of a plurality of UWB positioning base stations arranged under the mine and UWB tag cards carried by underground personnel, and is used for measuring and calculating the real position coordinates of the underground personnel in a positioning coordinate system and reading the identity information in the UWB tag cards;
the video acquisition equipment is arranged under the mine and is used for acquiring video data of personnel under the mine.
9. An underground personnel safety precaution device, comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor;
wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
10. A computer readable storage medium having stored thereon executable instructions which when executed by a processor cause the processor to implement the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310619391.4A CN116343102A (en) | 2023-05-30 | 2023-05-30 | Underground personnel safety early warning method, device, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310619391.4A CN116343102A (en) | 2023-05-30 | 2023-05-30 | Underground personnel safety early warning method, device, system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116343102A true CN116343102A (en) | 2023-06-27 |
Family
ID=86891591
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310619391.4A Pending CN116343102A (en) | 2023-05-30 | 2023-05-30 | Underground personnel safety early warning method, device, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116343102A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103617699A (en) * | 2013-12-02 | 2014-03-05 | 国家电网公司 | Intelligent safety monitor system of electric power working site |
| CN105072381A (en) * | 2015-07-17 | 2015-11-18 | 上海真灼电子技术有限公司 | Personnel identification method and system combining video identification and UWB positioning technologies |
| CN107437044A (en) * | 2016-05-26 | 2017-12-05 | 中国矿业大学(北京) | A kind of mine movable target following and localization method |
| CN109826668A (en) * | 2018-11-28 | 2019-05-31 | 山西潞安环保能源开发股份有限公司五阳煤矿 | The accurate personnel location system of underground multi-source and method |
| CN114298081A (en) * | 2021-09-29 | 2022-04-08 | 浙江大华技术股份有限公司 | Identification method, device, electronic device and storage medium for target behavior |
| CN114325573A (en) * | 2021-12-16 | 2022-04-12 | 武汉新电电气股份有限公司 | A rapid detection method for the identity and location information of substation operation and maintenance personnel |
| CN115810025A (en) * | 2021-09-13 | 2023-03-17 | 华东师范大学 | A method and system for indoor pedestrian positioning based on UWB and vision |
| CN116129307A (en) * | 2022-12-30 | 2023-05-16 | 江苏长路智造科技有限公司 | Method, device, system and storage medium for managing visual personnel of closed scene |
-
2023
- 2023-05-30 CN CN202310619391.4A patent/CN116343102A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103617699A (en) * | 2013-12-02 | 2014-03-05 | 国家电网公司 | Intelligent safety monitor system of electric power working site |
| CN105072381A (en) * | 2015-07-17 | 2015-11-18 | 上海真灼电子技术有限公司 | Personnel identification method and system combining video identification and UWB positioning technologies |
| CN107437044A (en) * | 2016-05-26 | 2017-12-05 | 中国矿业大学(北京) | A kind of mine movable target following and localization method |
| CN109826668A (en) * | 2018-11-28 | 2019-05-31 | 山西潞安环保能源开发股份有限公司五阳煤矿 | The accurate personnel location system of underground multi-source and method |
| CN115810025A (en) * | 2021-09-13 | 2023-03-17 | 华东师范大学 | A method and system for indoor pedestrian positioning based on UWB and vision |
| CN114298081A (en) * | 2021-09-29 | 2022-04-08 | 浙江大华技术股份有限公司 | Identification method, device, electronic device and storage medium for target behavior |
| CN114325573A (en) * | 2021-12-16 | 2022-04-12 | 武汉新电电气股份有限公司 | A rapid detection method for the identity and location information of substation operation and maintenance personnel |
| CN116129307A (en) * | 2022-12-30 | 2023-05-16 | 江苏长路智造科技有限公司 | Method, device, system and storage medium for managing visual personnel of closed scene |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Park et al. | Hardhat-wearing detection for enhancing on-site safety of construction workers | |
| AU2020270461B2 (en) | Situational Awareness Monitoring | |
| CN111445524A (en) | Recognition method of unsafe behavior of construction site workers based on scene understanding | |
| CN110852183B (en) | A method, system, device and storage medium for identifying a person without a helmet | |
| CN107145851A (en) | Intelligent Identification System of Hazardous Sources in Construction Operation Area | |
| CN111383429A (en) | Method, system, device and storage medium for detecting dress of workers in construction site | |
| KR20160099931A (en) | Disaster preventing and managing method for the disaster harzard and interest area | |
| CN115410354A (en) | Safety early warning method, device and system for industrial plant | |
| CN115700757B (en) | Control method and device for fire water monitor and electronic equipment | |
| CN112656081A (en) | Indoor positioning system based on vector position and attitude measurement technology | |
| CN107016673A (en) | Railway signal cabinet status lamp image analysis method, device and system | |
| CN116343102A (en) | Underground personnel safety early warning method, device, system and storage medium | |
| CN105095891A (en) | Human face capturing method, device and system | |
| KR20150112096A (en) | Kidnapping event detector for intelligent video surveillance system | |
| CN118506450A (en) | Method and device for detecting abnormal behavior in aerial work execution process and processor | |
| CN113887310A (en) | Worker standard dressing identification method, system and medium | |
| CN119064921A (en) | A method and device for safety monitoring of an operation area | |
| CN111597954A (en) | Method and system for identifying vehicle position in monitoring video | |
| CN113674109B (en) | Subway ground construction damage prevention method and device based on GIS and BIM | |
| KR20250019604A (en) | METHOD AND DEVICE FOR NON-PERCEPTUAL PUNCH CLOCK OF CONSTRUCTION PERSONNEL AT THE BOUNDARYLESS WORKFACE OF A HYDROELECTRIC POWER STATION | |
| CN116824471A (en) | Intelligent derrick area operation monitoring method and system | |
| KR20240040532A (en) | Method for infrctious disease prevention based on deep learning, and computer-readable recording medium including the same | |
| CN113989335A (en) | Method for automatically positioning workers in factory building | |
| Jadhav et al. | Realtime safety helmet detection using deep learning | |
| KR102765937B1 (en) | The object detection and motion recognition method through artificial intelligence-based on video big data analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230627 |
|
| RJ01 | Rejection of invention patent application after publication |