CN114554163B

CN114554163B - Coal mine underground operation monitoring system

Info

Publication number: CN114554163B
Application number: CN202210436348.XA
Authority: CN
Inventors: 袁刚
Original assignee: Shenzhen Kuyuan Digital Technology Co ltd
Current assignee: Shenzhen Kuyuan Digital Technology Co ltd
Priority date: 2022-04-25
Filing date: 2022-04-25
Publication date: 2022-08-19
Anticipated expiration: 2042-04-25
Also published as: CN114554163A

Abstract

A coal mine underground operation monitoring system comprises: the NPU unit is located underground, calculates operation data used for indicating an operator to operate operation equipment according to an underground operation real-time image collected by the mining camera, and processes the data of an operation area collected by the operation environment sensor to obtain environment safety data; and the AR equipment is provided with a camera, real images are obtained by the AR equipment through the camera in real time, operation data and environment safety data are obtained from the NPU unit in real time, virtual images are generated in real time according to the operation data, the virtual images and the real images are synthesized into AR images with the operation data, the environment safety data and warning marks in real time, and the AR images are displayed in real time. The invention can avoid accidents in time.

Description

Coal mine underground operation monitoring system

Technical Field

The invention belongs to the technical field of underground coal mines, and particularly relates to an underground coal mine operation monitoring system.

Background

Coal is used as the first large energy in China, the industrial scale of the coal industry is large, the distribution area is wide, the danger coefficient is high, the demand for improving the safe production level is raised to be a major national subject, and the construction requirement of intelligent mines is very urgent.

At present, when carrying out colliery borehole operation, transmit the real-time video of operation to aboveground equipment through the intelligent camera in the pit usually, the personnel in the pit are kept watch on to the operation in the pit according to the video, and such mode has obtained certain guarantee in the aspect of the security, when having at least the accident, can in time arrange the resource search and rescue in the pit, still exist not enoughly: the video data in the well is transmitted to the well, and even if the video data passes through the 5G network under construction, a large delay, even a few seconds, still can cause accidents to happen.

Disclosure of Invention

Based on this, to the technical problem, provide a colliery borehole operation monitoring system.

In order to solve the technical problems, the invention adopts the following technical scheme:

a coal mine underground operation monitoring system comprises:

the NPU unit is located underground, and the NPU unit calculates operation data used for indicating an operator to operate operation equipment according to a real-time image of underground operation acquired by the mining camera, and processes data of an operation area acquired by an operation environment sensor to obtain environment safety data; and

the AR equipment is provided with a camera, real images are obtained by the AR equipment through the camera in real time, operation data and environment safety data are obtained from the NPU unit in real time, virtual images are generated in real time according to the operation data, the virtual images and the real images are synthesized into AR images with the operation data, the environment safety data and warning marks in real time, and the AR images are displayed in real time.

Compared with the prior art, the method and the device do not need to transmit videos to the well, and the operation data and the environment safety data which are obtained by calculation and processing of the underground NPU unit can be quickly transmitted to the AR equipment, so that the real-time performance of the AR image displayed by the AR equipment is high, underground operators can timely and intuitively know the existence of the surrounding safety parameter data and risks through the AR image, accidents are avoided, and meanwhile, the current operation progress and state can be timely known through the AR image, so that accurate bases are provided for the next operation.

Drawings

The invention is described in detail below with reference to the following figures and embodiments:

FIG. 1 is a schematic structural view of the present invention;

FIG. 2 is a schematic diagram of an AR apparatus of the present invention;

FIG. 3 is a diagram illustrating an AR image according to the present invention;

FIG. 4 is a diagram illustrating an AR image according to a second embodiment of the present invention.

Detailed Description

As shown in fig. 1, an embodiment of the present specification provides a coal mine underground operation monitoring system, which includes a mining camera 110, an operation environment sensor 120, an edge computing device 130, a cloud platform 140, and an AR device 150.

Mining camera 110 sets up in the pit, can face towards the operation area and gather the image.

Work environment sensors 120 are used to collect data for the work area.

The mining camera 110 is provided with an NPU (neural-network processing units) unit, and the NPU unit calculates operation data for instructing an operator to operate the operation device according to a real-time image of the downhole operation acquired by the mining camera 110, and processes data of an operation area acquired by the operation environment sensor 120 to obtain environment safety data.

Taking the drilling operation as an example, the operation data includes the length of the drill bit exposed outside the coal wall and the length of the drill bit drilled into the coal wall, and the NPU unit calculates the length of the drill bit exposed outside the coal wall and the length of the drill bit drilled into the coal wall through a pre-trained target detection and tracking model.

For drilling operation, the working environment sensor 120 includes a mining stress sensor and a gas concentration sensor, the data acquired by the mining stress sensor and the gas concentration sensor are analog signals, the analog signals need to be processed by an NPU unit, the analog signals are converted into digital signals, and the digital signals are converted into environment safety data which can be seen by operators, namely, impact energy parameters and gas concentration.

The calculation power of the NPU unit in the mining camera 110 is generally about 1-2T, and the timeliness of data transmission is high when the mining camera 110 is used as an AI calculation power device closest to an operator.

It should be noted that when the computing power of the mining camera 110 is insufficient, centralized operation may be performed by the NPU unit in the downhole edge computing device 130, and then the operation result may be transmitted to the mining camera 110. The calculation power of the NPU unit in the edge calculation device 130 is generally about 10 times that of the mining camera 110, the edge calculation device 130 is a device having the NPU unit in the downhole local area network, such as a router, a server and the like in the downhole local area network, and the mining camera 110, the working environment sensor 120 and the edge calculation device 130 are all connected to the downhole local area network in a wired or wireless manner.

Furthermore, the NPU unit in the cloud platform 140 may also provide an auxiliary computing power, the cloud platform 140 is connected to the local area network downhole, the computing power of the NPU unit in the cloud platform 140 is typically tens of times or even hundreds of times that of the edge computing device 130, and a common and basic big data operation scene, such as the calculation of the whole real environment image, may be processed in a centralized manner, and this part of calculation is used as the basis of the operation, and the updating frequency is relatively low.

The NPU units in the mining camera 110 and the edge computing device 130 are used for computing local images before the eyes of the operator, and the local images are relatively high in updating frequency and high in requirement on timeliness due to the implementation of the operation, and specifically, the NPU unit in the mining camera 110 or the NPU unit in the edge computing device 130 needs to be set in advance according to actual test conditions, so that the safety and timeliness are higher.

An image within 10 seconds can be taken from the mining camera 110, the time (data generation time) for obtaining the operation data and the environmental safety data through calculation and processing by the NPU unit in the mining camera 110 and the time for transmitting the data to the AR device 150 are tested, the two times are added to obtain the total time, similarly, the total time corresponding to the edge calculation device 130 can also be obtained, the two total times are compared, if the total time of the NPU unit in the mining camera 110 is adopted is shorter, the NPU unit in the mining camera 110 is selected, otherwise, the NPU unit in the edge calculation device 130 is used.

Certainly, although the NPU unit is placed in the mining camera 110, the edge computing device 130 and the cloud platform 140, with the rapid development of science and technology, the cost of the NPU unit is likely to be further reduced in the future, and the NPU unit can be directly integrated into the AR device 150, so that the AR device 150 has basic computing power, can obtain operation data through self-computing and process the environment safety data, and finds the mine and seeks help with the camera 110 and the edge computing device 130 under the condition of insufficient computing power.

As shown in fig. 2, the AR device 150 is worn by an operator, is connected to a downhole local area network, and has a camera 151, the AR device 150 obtains a real image in real time through the camera 151, obtains operation data and environment safety data from an NPU unit in real time, generates a virtual image in real time according to the operation data, synthesizes the virtual image and the real image into an AR image with the operation data, the environment safety data and warning labels in real time, and displays the AR image in real time.

In the present embodiment, the AR device 150 is a miner's lamp integrated with AR glasses 152, which includes, in addition to the camera 151 and the AR glasses 152, an illumination unit 153, an audio unit 154, a communication unit 155, and a processing unit 156, the processing unit 156 being configured to:

real images are acquired in real time through the camera 151, and job data and environmental security data are acquired in real time from the NPU unit through the communication unit 155.

And generating a virtual image in real time according to the operation data, and synthesizing the virtual image and the real image into an AR image with the operation data, the environment safety data and the warning label in real time.

And comparing the environmental safety data with a preset threshold in real time, if the comparison result represents that the environmental safety data exceed the standard, sending out a sound alarm through the audio unit 154, and sending alarm information to the aboveground control equipment through the underground local area network through the communication unit 155.

The AR image is displayed in real time through the AR glasses 152, and the warning annotation in the AR image is shown in fig. 3 and 4.

In fig. 3, the impact energy parameter and the gas concentration are not over-marked, so that "safety" is marked at the back of the impact energy parameter and the gas concentration, and "overall safety" is marked at the top, and the drill bit can advance by 20 meters, where 20 meters is a preset distance, and the current drill bit position + the preset distance = the future drill bit position, the processing unit 156 can predict the impact energy parameter and the gas concentration to obtain predicted values of the impact energy parameter and the gas concentration at the future drill bit position, and if the predicted values are within a safety threshold, the drill bit can advance by 20 meters.

In FIG. 4, the impact energy parameter is out of standard and the gas concentration is not out of standard, so the "yellow warning" is marked behind the impact energy parameter, the "safety" is marked behind the gas concentration, and the "yellow warning" is marked at the top, the risk is higher! "

For the drilling operation, the real image is an image of the drill bit exposed out of the coal wall, the virtual image is an image of the drill bit drilled into the coal wall, the virtual image is an extension of the real image, the position (coordinates) of the drill bit in the real image can be determined through an image processing algorithm, and then the virtual image can be generated according to the known length of the drill bit drilled into the coal wall and the shape of the drill bit.

However, those skilled in the art should recognize that the above-described embodiments are illustrative only, and not limiting, and that changes and modifications can be made to the above-described embodiments without departing from the true spirit and scope of the invention, which is defined by the following claims.

Claims

1. A coal mine underground operation monitoring system is characterized by comprising:

the AR equipment is provided with a camera, real images are obtained by the AR equipment through the camera in real time, operation data and environment safety data are obtained from the NPU unit in real time, virtual images are generated in real time according to the operation data, the virtual images and the real images are synthesized into AR images with the operation data, the environment safety data and warning marks in real time, and the AR images are displayed in real time;

the borehole operation is the operation of driling, the operation data include the length that the drill bit exposes outside the coal wall and the length that the drill bit bored into the coal wall, real image is the image that the drill bit exposes outside the coal wall, virtual image is the image that the drill bit bored into the coal wall, environmental safety data include impact energy parameter and gas concentration, the warning mark includes the warning mark according to impact energy parameter and gas concentration prediction.

2. The coal mine underground operation monitoring system of claim 1, wherein the NPU unit is provided in the AR device.

3. The coal mine underground operation monitoring system of claim 1, wherein the NPU unit is disposed in the mining camera.

4. The coal mine underground operation monitoring system of claim 3, further comprising an edge computing device and a cloud platform, wherein the mining camera, the edge computing device and the AR device are all arranged in an underground local area network, the cloud platform is connected with the underground local area network, and NPU units are arranged in the edge computing device and the cloud platform.

5. The coal mine underground operation monitoring system of claim 4, wherein the operating environment sensors comprise mine stress sensors and gas concentration sensors.

6. The coal mine underground operation monitoring system of claim 3, wherein the NPU calculates the length of the drill exposed out of the coal wall and the length of the drill drilled into the coal wall through a pre-trained target detection and tracking model.

7. The coal mine downhole operation monitoring system of claim 6, wherein the AR device is an AR glasses integrated miner's lamp comprising a lighting unit, a camera, an audio unit, a communication unit, and a processing unit configured to:

real images are obtained in real time through a camera, and the operation data and the environment safety data are obtained in real time through the communication unit;

generating a virtual image in real time according to the operation data, and synthesizing the virtual image and the real image into an AR image with operation data, environment safety data and warning label in real time;

the environmental safety data are compared with a preset threshold value in real time, if the comparison result represents that the environmental safety data exceed the standard, a sound alarm is sent out through the audio unit, and alarm information is sent to the aboveground control equipment through the underground local area network through the communication unit;

and displaying the AR image in real time through AR glasses.