CN115727851B - A coal mine underground tunneling equipment posture detection system and method - Google Patents
A coal mine underground tunneling equipment posture detection system and method Download PDFInfo
- Publication number
- CN115727851B CN115727851B CN202211438343.7A CN202211438343A CN115727851B CN 115727851 B CN115727851 B CN 115727851B CN 202211438343 A CN202211438343 A CN 202211438343A CN 115727851 B CN115727851 B CN 115727851B
- Authority
- CN
- China
- Prior art keywords
- laser
- image
- point
- position coordinates
- line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a system and a method for detecting the pose of underground tunneling equipment of a coal mine, and relates to the technical field of pose detection. The system comprises: the three laser direction indicator comprises a three laser direction indicator module, a data acquisition module and a pose resolving module. The three laser direction indicator modules are positioned in a roadway at the rear of the tunneling equipment to be tested and are used for transmitting combined lasers which comprise three mutually parallel laser lines and are arranged in a triangular mode to the tunneling equipment to be tested; the data acquisition module is positioned on the machine body of the tunneling equipment to be detected, is opposite to the three laser director modules and is used for acquiring characteristic images of combined lasers; the characteristic image is a three laser spot image or a three laser spot three laser line image; the pose resolving module is connected with the data acquisition module and is used for calculating pose information of the tunneling equipment to be detected according to the characteristic image. The invention can solve the problems of unobvious roadway characteristics and even characteristic missing under the working condition of high dust and low illumination, and improve the stability and the anti-interference capability of the pose detection system.
Description
Technical Field
The invention relates to the technical field of pose detection, in particular to a pose detection system and method for underground tunneling equipment of a coal mine.
Background
Tunneling is an important production link of coal mine production, and rapid tunneling of coal mine tunnels is a key technical measure for ensuring high and stable production of a coal mine. Along with the development of fully mechanized mining technology and the demands of less humanization and no humanization, the position and posture detection of tunneling equipment enables tunnel tunneling to become a common and key technology of intelligent application and efficient intensive production of coal mines.
The machine vision pose measurement technology establishes a pose resolving model by using space geometric projection constraint conditions, so that the measurement of the target pose is realized, and the machine vision pose measurement technology has the advantages of non-contact measurement, high precision, good stability and the like, and has been preliminarily applied in underground coal mines. However, in the underground tunnel tunneling of the coal mine, under the working condition of high dust and low illumination, tunnel characteristics are not obvious and even the condition of characteristic missing can occur, if the tunnel environment is taken as the characteristic, stable characteristic extraction is difficult to realize, and stable characteristic matching is also difficult to realize, and the positioning is difficult to realize by adopting a conventional method.
Disclosure of Invention
The invention aims to provide a system and a method for detecting the pose of underground tunneling equipment of a coal mine, which are used for solving the problems of unobvious roadway characteristics and even characteristic missing under the working condition environment of high dust and low illumination and improving the stability and the anti-interference capability of the pose detection system.
In order to achieve the above object, the present invention provides the following solutions:
A coal mine underground tunneling equipment pose detection system, the system comprising:
The three laser director modules are positioned in a roadway behind the tunneling equipment to be tested and are used for transmitting combined laser to the tunneling equipment to be tested; the combined laser comprises three mutually parallel laser lines, and the three laser lines are arranged in a triangle;
The data acquisition module is positioned on the machine body of the tunneling equipment to be detected, is opposite to the three laser director modules and is used for acquiring characteristic images of the combined laser; the characteristic image is a three laser spot image or a three laser spot three laser line image;
The pose resolving module is connected with the data acquisition module and is used for calculating pose information of the tunneling equipment to be detected according to the characteristic image; the pose information includes: the machine body of the tunneling equipment to be tested is horizontally, vertically and longitudinally offset from the center line of the roadway where the machine body of the tunneling equipment to be tested is located relative to the pitch angle, yaw angle and roll angle of the roadway where the machine body of the tunneling equipment to be tested is located.
Optionally, the three laser director module includes:
the laser direction indicator bracket is positioned in a roadway behind the tunneling equipment to be tested;
the three laser directors are respectively positioned on the laser director bracket and are used for transmitting combined laser to the tunneling equipment to be tested; the three laser directors are parallel to each other, and are distributed in a triangle.
Optionally, the data acquisition module includes:
the camera explosion-proof shell is fixed on the machine body of the tunneling equipment to be tested;
and the industrial camera is fixed in the camera explosion-proof shell, and the lens faces the three laser director modules and is used for collecting characteristic images of the combined laser.
Optionally, the pose resolving module includes:
The image characteristic judging unit is connected with the data acquisition module and is used for acquiring a characteristic image of the combined laser and judging whether the characteristic image is a three-laser-point image or a three-laser-point three-laser-line image;
The laser point feature extraction unit is connected with the image feature judgment unit and is used for extracting three laser points in the feature image and determining the position coordinates of each laser point when the feature image is a three laser point image;
the laser line feature extraction unit is connected with the image feature judgment unit and is used for extracting three laser lines in the feature image and determining the position coordinates of each laser line and the position coordinates of the corresponding laser point when the feature image is a three-laser-point three-laser-line image; the number of the laser points is three, and the three laser points respectively correspond to the starting points of the three laser lines;
The three-point model calculation unit is connected with the laser point feature extraction unit and is used for calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser point;
The three-point three-line model calculation unit is connected with the laser line characteristic extraction unit and is used for calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser line and each laser point;
The pose resolving unit is respectively connected with the three-point model computing unit and the three-point three-line model computing unit and is used for computing pose information of the tunneling equipment to be tested according to the position coordinates of the tunneling equipment to be tested.
Optionally, the system further comprises:
and the pose visualization module is connected with the pose resolving module and used for displaying the pose information.
The invention also provides a method for detecting the pose of the underground tunneling equipment of the coal mine, which is applied to the system and comprises the following steps:
acquiring a characteristic image of combined laser, and judging whether the characteristic image is a three-laser-point image or a three-laser-point three-laser-line image; the characteristic image is acquired by a data acquisition module through combined laser emitted by the three laser directors;
When the characteristic image is a three-laser-point image, three laser points in the characteristic image are extracted, and the position coordinates of each laser point are determined; calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser point;
When the characteristic image is a three-laser-point three-laser-line image, extracting three laser lines in the characteristic image, and determining the position coordinates of each laser line and the position coordinates of the corresponding laser point; the number of the laser points is three, and the three laser points respectively correspond to the starting points of the three laser lines; calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser line and each laser point;
And calculating pose information of the tunneling equipment to be detected according to the position coordinates of the tunneling equipment to be detected.
Optionally, the extracting three laser points in the feature image specifically includes:
performing distortion correction, gaussian filtering, morphological gradient conversion, RGB channel separation and gray conversion on the characteristic image to obtain a first binary image;
performing a closing operation on the first binary image and determining each region in the first binary image;
Selecting a region meeting a first set condition in the first binary image to perform ellipse fitting, and removing stray light to obtain a plurality of ellipse light spots; the first setting condition is: the number of the regional boundary points is in a first set range;
Judging whether the number of the elliptical light spots is equal to three; if yes, taking the three elliptical light spots as three laser spots in the characteristic image; if not, returning to the step of acquiring the characteristic image of the combined laser and judging whether the characteristic image is a three-laser point image or a three-laser point three-laser line image.
Optionally, the extracting three laser lines in the feature image specifically includes:
performing distortion correction, gaussian filtering and gray level conversion on the characteristic image to obtain a second binary image;
performing a closing operation on the second binary image, and determining each region in the second binary image;
Determining the minimum circumscribed rectangle of the region meeting the second setting condition in the second binary image, and carrying out rectangle combination and contour point straight line fitting on the minimum circumscribed rectangle meeting the third setting condition to obtain three laser lines in the characteristic image; the second setting condition is: the number of the regional edge points is in a second setting range; the third setting condition is: the height-width ratio range of the minimum circumscribed rectangle is in a third setting range, and the area of the minimum circumscribed rectangle is in a fourth setting range.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
according to the invention, three combined lasers which are parallel to each other and are arranged in a triangle are emitted to the tunneling equipment to be detected by the three laser direction indicator modules, and the characteristic images of the combined lasers are acquired in real time by the data acquisition module, so that the characteristic images with obvious characteristics can be acquired under the working condition of high dust and low illumination, and the pose information of the tunneling equipment to be detected is calculated by the pose calculation module according to the acquired three laser point images or the three laser point three laser line images, so that the stability and the anti-interference capability of the pose detection system are improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a block diagram of a pose detection system of underground coal mine tunneling equipment provided by the invention;
FIG. 2 is a schematic diagram of a model of a three laser pointer module;
FIG. 3 is a schematic illustration of a feature image as a three laser spot image;
FIG. 4 is a schematic illustration of a feature image as a three laser point, three laser line image;
FIG. 5 is a flow chart of a method for detecting the pose of underground coal mine tunneling equipment;
FIG. 6 is a flow chart of pose detection provided by the present invention;
FIG. 7 is a perspective schematic diagram of a gate-shaped tri-wire structure;
FIG. 8 is a schematic diagram of the positional relationship of three points a, b, and c in a three-point three-line model;
FIG. 9 is a flowchart of a three-point model image processing provided by the present invention;
FIG. 10 is a schematic diagram of a three laser point vision pose measurement method;
FIG. 11 is a schematic view of a three-point positioning model projection.
Symbol description:
The device comprises a 1-laser director bracket, a 2-laser director, a 3-industrial personal computer, a 4-explosion-proof control box, a 5-industrial camera and a 6-camera explosion-proof shell.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention aims to provide a system and a method for detecting the pose of underground tunneling equipment of a coal mine, which are used for solving the problems of unobvious roadway characteristics and even characteristic missing under the working condition environment of high dust and low illumination and improving the stability and the anti-interference capability of the pose detection system.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Fig. 1 is a block diagram of a system for detecting the pose of underground coal mine tunneling equipment, which is provided by the invention, as shown in fig. 1, and comprises: the three laser direction indicator comprises a three laser direction indicator module, a data acquisition module and a pose resolving module.
The three laser direction indicator modules are positioned in a roadway behind the tunneling equipment to be tested and are used for emitting combined laser to the tunneling equipment to be tested; the combined laser includes three mutually parallel laser lines, and the three laser lines are arranged in a triangle (as shown by three mutually parallel straight lines in fig. 1). The data acquisition module is positioned on the machine body of the tunneling equipment to be detected, is opposite to the three laser director modules and is used for acquiring characteristic images of the combined laser; the characteristic image is a three laser spot image or a three laser spot three laser line image. The pose resolving module is connected with the data acquisition module and is used for calculating pose information of the tunneling equipment to be detected according to the characteristic image; the pose information includes: the machine body of the tunneling equipment to be tested is horizontally, vertically and longitudinally offset from the center line of the roadway where the machine body of the tunneling equipment to be tested is located relative to the pitch angle, yaw angle and roll angle of the roadway where the machine body of the tunneling equipment to be tested is located.
Further, the system further comprises: a pose visualization module; and the pose visualization module is connected with the pose resolving module and is used for displaying the pose information. The pose visualization module can be arranged on the tunneling equipment body or outside the tunneling equipment body, and can dynamically display the pose information of the tunneling equipment in real time in various forms (such as data, graphs, waveform diagrams and the like), so that tunneling, remote and intelligent control and correction of the tunneling equipment are facilitated.
In this embodiment, the three laser pointer module includes: a laser direction indicator bracket 1 and three laser direction indicators 2. The laser direction indicator bracket 1 is positioned in a roadway behind the tunneling equipment to be tested; the three laser directors 2 are respectively positioned on the laser director bracket 1 and are used for transmitting combined laser to the tunneling equipment to be tested; the three laser directors 2 are parallel to each other, and the three laser directors 2 are distributed in a triangle. Fig. 2 is a schematic diagram of a model of a three laser pointer module. As shown in fig. 2, the three laser directors 2 are respectively located at the point a, the point b and the point c on the laser director bracket 1, and the connection lines between every two laser directors 2 form a triangle. The laser direction indicator 2 is preferably a mining explosion-proof laser direction indicator, and imaging characteristics of the laser direction indicator are used as input of a data acquisition module and used for providing a data source for pose calculation.
The data acquisition module comprises: a camera explosion proof housing 6 and an industrial camera 5. The camera explosion-proof shell 6 is fixed on the machine body of the tunneling equipment to be tested; the industrial camera 5 is fixed in the camera explosion-proof shell 6, and the lens faces the three laser direction indicator modules and is used for collecting characteristic images of the combined laser. The industrial camera 5 is preferably an explosion proof industrial camera. Specifically, the camera explosion-proof housing 6 is installed in the position at the rear of the tunneling equipment fuselage, the industrial camera 5 is fixed in the camera explosion-proof housing 6, the camera lens of the industrial camera faces the back of the tunnel, and the industrial camera is used for collecting three laser direction indicator image information in real time in the running process of the tunneling equipment. Further, the camera explosion-proof housing 6 is fixed on the heading machine platform through bolts, a camera opening faces to the rear, and the industrial camera 5 is installed inside the camera explosion-proof housing 6. In practical application, the data acquisition module further comprises a communication line.
The pose resolving module takes an industrial computer as a computing core, and is preferably an industrial computer 3. The industrial personal computer 3 is arranged at the position of the tunneling equipment to be detected, and can process and pose resolving characteristic images acquired by the explosion-proof industrial camera in the data acquisition module in real time, so that data are provided for the pose visualization module. In order to protect the industrial personal computer 3 from the external environment (such as falling rocks in a roadway), the system further comprises an explosion-proof control box 4; the explosion-proof control box 4 is fixed at the position of the tunneling equipment to be detected, and the industrial personal computer 3 is arranged in the explosion-proof control box 4.
Further, the pose resolving module includes: the device comprises an image feature judging unit, a laser point feature extracting unit, a laser line feature extracting unit, a three-point model calculating unit, a three-point three-line model calculating unit and a pose resolving unit.
The image characteristic judging unit is connected with the data acquisition module and is used for acquiring a characteristic image of the combined laser and judging whether the characteristic image is a three-laser-point image or a three-laser-point three-laser-line image. In addition, the imaging characteristics can be predicted according to the dust condition in the roadway, when no dust or the dust concentration is low, the characteristic image is a three laser point image, and when the dust concentration is high, the characteristic image is a three laser point three laser line image.
The laser point feature extraction unit is connected with the image feature judgment unit and is used for extracting three laser points in the feature image and determining the position coordinates of each laser point when the feature image is the three laser point image. The three-point model calculation unit is connected with the laser point feature extraction unit and is used for calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser point.
The laser line feature extraction unit is connected with the image feature judgment unit and is used for extracting three laser lines in the feature image and determining the position coordinates of each laser line and the position coordinates of the corresponding laser point when the feature image is a three-laser-point three-laser-line image; the number of the laser points is three, and the three laser points respectively correspond to the starting points of the three laser lines. The three-point three-line model calculation unit is connected with the laser line feature extraction unit and is used for calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser line and each laser point.
The pose resolving unit is respectively connected with the three-point model computing unit and the three-point three-line model computing unit and is used for computing pose information of the tunneling equipment to be tested according to position coordinates of the tunneling equipment to be tested.
In specific application, as shown in fig. 1, the laser director bracket 1 is arranged at the top end of a roadway of a tunneling surface and is fixed with an anchor cable at the top of the coal roadway by three nuts, so that the laser director bracket is reliable in installation and convenient in disassembly. The laser director bracket 2 is used for installing the laser director 2. The installation position of the laser direction indicator 2 is in an inverted triangle shape, the front end face of the laser direction indicator 2 is kept in the same plane, the plane is kept perpendicular to the ground, and the laser direction indicators 2 are parallel to each other after being installed, so that the error between the triangle side length formed at the beginning of laser point emission and the triangle side length formed on a certain plane in the air after laser irradiation for a certain distance is not more than 5 cm.
The method comprises the steps of installing three mining explosion-proof laser directors and supports thereof on the top of a coal roadway behind tunneling equipment, and acquiring imaging information of the three mining explosion-proof laser directors in real time through an explosion-proof industrial camera fixed on a tunneling equipment body; when the dust is less, the imaging characteristics of the three mining explosion-proof laser directors are three points, namely, the characteristic images are three laser point images, as shown in fig. 3, so that pose calculation is carried out by using the three laser points; when the dust concentration is large, the imaging characteristics of the three mining explosion-proof laser directors are three lines formed by the emission of three laser points, namely, the characteristic images are three laser line images of the three laser points, as shown in fig. 4, so that pose calculation is performed by using the three lines of the three points; finally, the pitch angle, the yaw angle and the roll angle of the tunneling equipment in the tunneling process are obtained, and the horizontal, vertical and front-back offset distances of the machine body relative to the center line of the tunnel are obtained.
The invention also provides a method for detecting the pose of the underground coal mine tunneling equipment, which is applied to the system, wherein fig. 5 is a flow chart of the method for detecting the pose of the underground coal mine tunneling equipment, and fig. 6 is a flow chart of the pose detection. As shown in fig. 5 and 6, the method includes:
Step S1: acquiring a characteristic image of combined laser, and judging whether the characteristic image is a three-laser-point image or a three-laser-point three-laser-line image; the characteristic image is acquired by the data acquisition module through the combined laser emitted by the three laser directors.
Step S2: when the characteristic image is a three-laser-point image, three laser points in the characteristic image are extracted, and the position coordinates of each laser point are determined; and calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser point.
Step S3: when the characteristic image is a three-laser-point three-laser-line image, extracting three laser lines in the characteristic image, and determining the position coordinates of each laser line and the position coordinates of the corresponding laser point; the number of the laser points is three, and the three laser points respectively correspond to the starting points of the three laser lines; and calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser line and each laser point.
Step S4: and calculating pose information of the tunneling equipment to be detected according to the position coordinates of the tunneling equipment to be detected. The laser direction indicator has three-dimensional coordinate information relative to the roadway, the laser direction indicator has three-dimensional coordinate information relative to the industrial camera, and the industrial camera has three-dimensional coordinate information relative to the tunneling equipment body. Therefore, the pitch angle, yaw angle and roll angle of the tunneling equipment body relative to the tunnel and the horizontal, vertical and front-back offset distances from the center line of the tunnel can be calculated according to the position coordinates of the tunneling equipment to be detected.
The specific implementation steps of the invention are as follows:
Step one: arranging a heading machine (namely heading equipment to be detected) in a heading roadway, and installing three laser directors at the rear part of a heading machine body, wherein the three laser directors are installed at the top of the roadway and are not far away from an industrial camera installed at the rear part of the heading machine; an explosion-proof camera shell is arranged at the rear part of the tunneling machine body, an industrial camera is placed inside.
Step two: acquiring image information by using an explosion-proof industrial camera, calibrating the camera, and determining calibration parameters; the calibration parameters include internal parameters of the camera (i.e., industrial camera) including focal length of the camera, main pixel point coordinates, and image distortion coefficients.
Step three: in the tunneling process, an industrial computer arranged in the pose resolving module is arranged on the tunneling machine body to process image information (namely characteristic images) acquired by an industrial camera in real time, and the pose of tunneling equipment is resolved in real time by using three laser point models or three laser points and three laser line models.
Step four: in the tunneling process, the pose resolving module converts the machine body information into the tunnel coordinate system to obtain the tunneling equipment pose through the coordinates of the feature points in the tunnel coordinate system, the coordinates of the feature points in the camera coordinate system and the camera coordinates in the machine body coordinate system. Specifically, when the feature image is a three-laser-spot image, the feature points are three of the laser spots; when the characteristic image is a three-laser-point three-laser-line image, the characteristic points are two laser points and a point obtained by a perspective principle.
Step five: in the tunneling process, the pose visualization module is used for displaying the tunneling equipment pose information of the pose resolving module in real time, so that the tunneling equipment can be controlled remotely and intelligently and correct the deviation conveniently.
The above steps are discussed in detail below.
In step S3, the extracting three laser lines in the feature image specifically includes:
step S3.1: and carrying out distortion correction, gaussian filtering and gray level conversion on the characteristic image to obtain a second binary image.
Step S3.2: and performing a closing operation on the second binary image, and determining each region in the second binary image.
Step S3.3: determining the minimum circumscribed rectangle of the region meeting the second setting condition in the second binary image, and carrying out rectangle combination and contour point straight line fitting on the minimum circumscribed rectangle meeting the third setting condition to obtain three laser lines in the characteristic image; the second setting condition is: the number of the regional edge points is in a second setting range; the third setting condition is: the height-width ratio range of the minimum circumscribed rectangle is in a third setting range, and the area of the minimum circumscribed rectangle is in a fourth setting range.
As a specific embodiment, when the feature image is a three laser point three laser line image, the image processing method is as follows:
(1) And carrying out distortion correction on the original image by utilizing the focal length, the main pixel point coordinates and the image distortion coefficient, and outputting the corrected image.
(2) And carrying out Gaussian filtering on the corrected image to eliminate Gaussian noise.
(3) And carrying out gray level conversion on the Gaussian filtered image, and outputting a binarized image (namely a second binary image). Wherein the binarization threshold is an empirical value.
(4) All contours in the binary image are detected, and the detection method comprises the following steps: and performing a closed operation on the binarized image, then obtaining the edge of each region, finding out the minimum circumscribed rectangle for the region with the number of the edge points of the region being in the interval (a, b), and judging the straight line profile meeting the condition according to the aspect ratio range (c, d) of the minimum circumscribed rectangle and the area range [ e, f ] of the minimum circumscribed rectangle. The aspect ratio ranges (c, d) and the area ranges [ e, f ] are empirical values. Recording the coordinates of each contour point; the region outline meeting the requirement is screened for the first time, and a relatively large number of small regions on the line are solved; rectangular merging is carried out for the second time, and the areas are merged with similar circumscribed rectangular angles; and finally, screening the polymerized large area meeting the linear proportion, and providing a basis for straight line fitting.
(5) The contour points are processed into a straight line through least square fitting. The other two straight lines are fitted in turn by the method described above. And respectively acquiring the slope and intercept of the three straight lines, substituting the slope and the intercept into a three-point three-line model, and calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser line and each laser point.
Calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser line and each laser point, wherein the method specifically comprises the following steps: determining at least one gate-shaped three-wire structure according to the position coordinates of each laser line and each laser point; the gate-shaped three-wire structure is composed of connecting lines of any two laser points and two corresponding laser lines, and the connecting lines of three laser points and the three laser lines jointly form three gate-shaped three-wire structures; and solving the gate-shaped three-wire structure based on the gate-shaped three-wire structure perspective principle to obtain the position coordinates of the tunneling equipment to be detected.
In addition, when the dust concentration is large, the imaging characteristic is three-point three-line (namely, the characteristic image is three-laser-point three-laser-line image), and at the moment, if one point and one line are shielded due to the shielding of an object, pose calculation can be performed by utilizing two points and two lines based on the door-shaped three-line structure perspective principle.
In specific application, firstly, a two-point two-line heading machine pose resolving model is used as a basis, laser points and lines generated by the laser points are extracted, the slope and intercept of a straight line obtained through image processing are substituted into the model, position coordinates of heading equipment are obtained, and then the pose of a machine body in a roadway is obtained through coordinate conversion.
Fig. 7 is a perspective schematic diagram of a gantry three-line structure, as shown in fig. 7, according to the three-line perspective characteristic of the gantry structure, when two parallel straight lines are not parallel to the camera image plane, there is a unique solution for pose calculation between the camera coordinate system and the laser measurement coordinate system. The invention provides a three-point three-line vision measurement model, which has the advantages that the accuracy of the two-point two-line vision measurement model is not high to be improved, and the specific method is to add a new laser, namely, the original two laser beams are changed into three laser beams, and the laser characteristics are abstracted from the original two light spots and two lines to be changed into three light spots and three lines.
The three-point three-line model is based on a three-line perspective geometric structure model of a portal structure, except that the portal three-line structure model is calculated three times in the invention, and three different point line combination modes of ac, ab and bc are respectively adopted, and the position relationship of the three points a, b and c is shown in figure 8.
xg＝t1*xac+t2*xab+t3*xbc (1)
yg＝t1*yac+t2*yab+t3*ybc (2)
zg＝t1*zac+t2*zab+t3*zbc (3)
1＝t1+t2+t3 (4)
Wherein: x g,yg,zg is a coordinate after combined positioning (namely a three-dimensional point coordinate of tunneling equipment to be detected), t is a weighted value of any one of three combination modes, wherein t 1 is a weighted value of an ac combination, t 2 is a weighted value of three combinations ab, and t 3 is a weighted value of a bc combination; x ac,yac,zac is the tunneling equipment position coordinate calculated by the two points ac, x ab,yab,zab is the tunneling equipment position coordinate calculated by the two points ab, and x bc,ybc,zbc is the tunneling equipment position coordinate calculated by the two points bc. The manner in which the coordinates are calculated based on the gate-shaped tri-wire structure is not described in detail herein.
In step S2, the extracting three laser points in the feature image specifically includes:
step S2.1: and carrying out distortion correction, gaussian filtering, morphological gradient conversion, RGB channel separation and gray conversion on the characteristic image to obtain a first binary image.
Step S2.2: and performing a closing operation on the first binary image, and determining each region in the first binary image.
Step S2.3: selecting a region meeting a first set condition in the first binary image to perform ellipse fitting, and removing stray light to obtain a plurality of ellipse light spots; the first setting condition is: the number of the boundary points of the region is within a first set range.
Step S2.4: judging whether the number of the elliptical light spots is equal to three; if yes, taking the three elliptical light spots as three laser spots in the characteristic image; if not, returning to the step of acquiring the characteristic image of the combined laser and judging whether the characteristic image is a three-laser point image or a three-laser point three-laser line image.
As a specific embodiment, when the feature image is a three laser spot image, the image processing method used refers to fig. 9, which includes:
(1) And correcting the distortion of the image by using the internal parameters (namely the focal length, the main pixel point coordinates and the image distortion coefficient) of the industrial camera obtained through calibration, and outputting a corrected image.
(2) And carrying out Gaussian filtering on the corrected image, and carrying out morphological gradient conversion after filtering.
(3) Carrying out R, G, B channel separation on the converted image, separating out an R channel image, then solving the maximum gray level of the image, and determining a binarization threshold range according to the maximum gray level.
(4) The binarized region boundary may contain an unsealed region, a closing operation is needed, then the boundary of each region is obtained, if the number of the region boundaries is more than 3, whether the number of the boundary points is more than a and less than b is judged, if the number of the boundary points is more than a and less than b is met, the minimum rectangle of the region is obtained, and then some parasitic lights are deleted according to the height-width ratio of the minimum rectangle. Wherein the number range (a, b) of boundary points is an empirical value.
(5) And fitting the residual area by using ellipse fitting, obtaining the central coordinates of the light spots, removing part of stray light according to the distance range [ g, h ] between the elliptical light spots, comparing with the image at the previous moment, and removing part of stray light.
(6) Judging the number of points, if the number of the points is not equal to 3, re-acquiring the image, otherwise substituting the image into a three-point positioning model of the heading machine, and calculating the position coordinates of the heading equipment to be detected according to the position coordinates of each laser point.
Calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser point, wherein the method specifically comprises the following steps: determining a rotation matrix of an industrial camera on the machine body of the tunneling equipment to be detected according to the position coordinates of each laser point, and measuring vectors of directions of each laser point in a camera coordinate system; and determining the position coordinates of the tunneling equipment to be tested according to the rotation matrix and the direction measurement vector.
Fig. 10 is a schematic diagram of a three laser point vision pose measurement method. As shown in fig. 10, in a specific application, the three-point positioning model gives the position GPi of the laser spot features f i =1, 2,3 in the reference coordinate system, and the direction measurement vector Cbi of the feature points in the camera coordinate system, with the goal of estimating the rotation matrix and position GpC of the camera. Wherein { C } represents a camera coordinate system, its position is p C, and its azimuth is{ G } represents the reference coordinate system.
From the geometric relationship, obtain
In formula (5): Gpi Representing coordinates of the three laser points in a reference coordinate system, GpC being the position of the camera, Cbi being the directional measurement vector of the feature points in the camera coordinate system. d i denotes the Euclidean distance from the camera to the feature point in the reference coordinate system, and the normal vector of the corresponding surface is projected by subtracting two phases.
The camera coordinates are calculated as follows:
FIG. 11 is a schematic view of a three-point positioning model projection. As shown in fig. 11, for 3 known spatial points P 1、P2 and P 3, the 3 sides of the triangle they make up are A, B and C in length. Imaging points in the camera imaging plane are P c1、Pc2 and P c3 respectively, and unit vectors formed by a camera optical axis center point O and space points P 1、P2 and P 3 are k 1、k2 and k 3.
Let the angle between k 2 and k 3 be α, the angle between k 1 and k 3 be β, and the angle between k 1 and k 2 be γ, then there are:
The distances between the O point and the spatial points P 1、P2 and P 3 are m 1、m2 and m 3, respectively. Equation (8) can be obtained according to the triangle cosine law.
To simplify the calculation, the following equivalent substitutions are made
In the formula (9), p represents a proportional relationship between m 1 and m 3, and q represents a proportional relationship between m 2 and m 3.
Available formula (10) and formula (11)
In the formula (11), a j denotes a coefficient of a polynomial, and j denotes an order of the polynomial.
Wherein,
From the above, the values of p, q, m 1、m2 and m 3 can be found. The position coordinates of each spatial point in the camera are P ci＝miki, i=1, 2,3. The image coordinates of the 2D points are known, the 3 cosine angles are known, the coordinates of the 3D points are known, and only x and y are unknown. From formula (11), a j is known. We can therefore find the values of p and q. The fourth system of equations theoretically has four solutions. Let four pose solutions be (x m,ym), m=1, 2,3,4, and find the minimum two norms.
As in equation (12), the minimum is the pose of the camera (i.e., the heading machine body) relative to the target coordinate system.
The position of the heading machine under the tunnel coordinate system can be obtained by positioning the heading machine body with three laser points and then obtaining the relation between the machine body coordinate system and the tunnel coordinate system according to the known relation between the laser director coordinate system and the tunnel coordinate system.
The invention provides a three-laser-direction-instrument-based system and a three-laser-direction-instrument-based method for detecting the pose of underground coal mine tunneling equipment, which aim to solve the problem that dust is not characteristic when the dust is large in tunneling or overhauling processes, and compared with the prior art, the three-laser-direction-instrument-based system has the following advantages:
(1) The system is complete and stable, and can be preliminarily applied to actual operation.
(2) And three groups of two points and two lines are used for fusion calculation, so that the calculation result is more stable and accurate.
(3) The invention integrates three-point and three-point three-line calculation methods to realize self-adaptive environment switching. The three-point model calculation method is suitable for the situation that the dust concentration in the coal mine is small, and the three-point three-line model calculation method is suitable for the situation that the dust concentration in the coal mine is large, and has good applicability.
(4) The machine vision based on the laser direction indicator is applied to the pose detection of underground tunneling equipment of a coal mine, has the characteristics of low cost, flexible operation, strong anti-interference, high measurement precision, good system stability and the like, and has certain market popularization value.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the core concept of the invention; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (7)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211438343.7A CN115727851B (en) | 2022-11-17 | 2022-11-17 | A coal mine underground tunneling equipment posture detection system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211438343.7A CN115727851B (en) | 2022-11-17 | 2022-11-17 | A coal mine underground tunneling equipment posture detection system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115727851A CN115727851A (en) | 2023-03-03 |
| CN115727851B true CN115727851B (en) | 2024-11-22 |
Family
ID=85296166
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211438343.7A Active CN115727851B (en) | 2022-11-17 | 2022-11-17 | A coal mine underground tunneling equipment posture detection system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115727851B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116735079B (en) * | 2023-08-15 | 2023-11-14 | 山东宇飞传动技术有限公司 | Mining winch balance detection equipment and method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017222558A1 (en) * | 2016-06-24 | 2017-12-28 | Isee, Inc. | Laser-enhanced visual simultaneous localization and mapping (slam) for mobile devices |
| CN112050732B (en) * | 2020-08-21 | 2022-03-15 | 西安科技大学 | Method and system for automatically detecting spatial pose of cantilever type heading machine |
| CN114322960B (en) * | 2022-01-07 | 2023-08-22 | 西安科技大学 | A method and system for intelligent moving station of roadheader based on visual positioning |
-
2022
- 2022-11-17 CN CN202211438343.7A patent/CN115727851B/en active Active
Non-Patent Citations (3)
| Title |
|---|
| 基于三激光束标靶的煤矿井下长距离视觉定位方法研究;杨文娟等;煤炭学报;20220127;全文 * |
| 基于三激光点标靶的掘进机机身视觉定位技术研究;张旭辉等;电子测量与仪器学报;20220613;全文 * |
| 张旭辉等.基于三激光点标靶的掘进机机身视觉定位技术研究.电子测量与仪器学报.2022,全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115727851A (en) | 2023-03-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11673587B2 (en) | Comprehensive inspection vehicle for subway tunnel | |
| CN109100741B (en) | A target detection method based on 3D lidar and image data | |
| CN111855664B (en) | Adjustable three-dimensional tunnel defect detection system | |
| CN110415259B (en) | Street tree point cloud identification method based on laser reflection intensity | |
| JP7233575B2 (en) | MAP GENERATION SYSTEM, MAP GENERATION METHOD AND MAP GENERATION PROGRAM | |
| JP6060682B2 (en) | Road surface image generation system, shadow removal apparatus, method and program | |
| CN113971768A (en) | A three-dimensional dynamic detection method for illegal buildings in transmission lines based on UAV | |
| EP0215948A1 (en) | Vehicle for evaluating properties of road surfaces | |
| CN114066985B (en) | Method for calculating hidden danger distance of power transmission line and terminal | |
| JP2020013573A (en) | Three-dimensional image reconstruction method of vehicle | |
| JP5473383B2 (en) | Section measuring device, section measuring method and section measuring program | |
| CN115727851B (en) | A coal mine underground tunneling equipment posture detection system and method | |
| CN208254424U (en) | A kind of laser blind hole depth detection system | |
| JP3327068B2 (en) | Road surface measurement device | |
| CN114252075A (en) | A path tracking method and system for a cable trench inspection robot | |
| CN110081828B (en) | Machine vision shield tail gap detection image grid characteristic point reliability filtering method | |
| CN118111345A (en) | Tunnel foundation pit surrounding rock displacement, crack and ponding monitoring system | |
| CN114429469B (en) | A method and system for determining the position and posture of a tunnel boring machine body based on three laser point targets | |
| CN116643289A (en) | Underground roadway laser radar SLAM method with attached wire constraint | |
| CN112907674B (en) | Camera calibration method and device for unmanned vehicle | |
| CN113252004A (en) | Tunnel comprehensive detection monitoring system and method | |
| CN113701633A (en) | Position and posture monitoring equipment of development machine | |
| US20240257376A1 (en) | Method and system for detection a line above ground from a helicopter | |
| EP4497653A1 (en) | Information processing method and device for estimating the three-dimensional position of a railroad track | |
| CN112907739B (en) | Method, device and system for acquiring height difference information of well lid |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |