CN113135181B - Mining area automatic driving loading and unloading point accurate parking method based on visual assistance - Google Patents
Mining area automatic driving loading and unloading point accurate parking method based on visual assistance Download PDFInfo
- Publication number
- CN113135181B CN113135181B CN202110688400.6A CN202110688400A CN113135181B CN 113135181 B CN113135181 B CN 113135181B CN 202110688400 A CN202110688400 A CN 202110688400A CN 113135181 B CN113135181 B CN 113135181B
- Authority
- CN
- China
- Prior art keywords
- camera
- parking position
- data point
- expected parking
- expected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005065 mining Methods 0.000 title claims abstract description 56
- 238000000034 method Methods 0.000 title claims abstract description 30
- 230000000007 visual effect Effects 0.000 title claims description 16
- 238000009434 installation Methods 0.000 claims abstract description 51
- 238000004364 calculation method Methods 0.000 claims abstract description 7
- 238000012937 correction Methods 0.000 claims description 19
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000006243 chemical reaction Methods 0.000 claims description 16
- 238000001514 detection method Methods 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 2
- 238000003032 molecular docking Methods 0.000 claims 4
- 230000003993 interaction Effects 0.000 abstract 1
- 238000012545 processing Methods 0.000 description 4
- 239000000463 material Substances 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000003245 coal Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
- B60W30/06—Automatic manoeuvring for parking
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
- B60W60/001—Planning or execution of driving tasks
- B60W60/0025—Planning or execution of driving tasks specially adapted for specific operations
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
- G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
- G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
- G01S19/42—Determining position
- G01S19/45—Determining position by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W2050/0001—Details of the control system
- B60W2050/0043—Signal treatments, identification of variables or parameters, parameter estimation or state estimation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Networks & Wireless Communication (AREA)
- Mathematical Physics (AREA)
- Position Fixing By Use Of Radio Waves (AREA)
- Navigation (AREA)
Abstract
The invention discloses a visual-assistance-based accurate parking method for an automatic driving loading and unloading point in a mining area, which is used for calibrating and finely adjusting a camera arranged in front of an auxiliary operation vehicle; the method has the advantages that the camera picture is displayed in real time through the human-computer interaction interface, the expected parking position is selected, operation and regulation of workers are facilitated, and the loading and unloading efficiency of the mining truck is improved; converting the pixel coordinates of the expected parking position into the transverse and longitudinal distance between the expected parking position and the camera, and acquiring the relative position between the expected parking position and the GPS antenna of the auxiliary operation vehicle according to the transverse and longitudinal distance between the installation position of the camera and the GPS antenna of the auxiliary operation vehicle; calculating the GPS coordinate of the expected parking position by adopting a Vincenty ellipsoid calculation formula in combination with the GPS information of the auxiliary work vehicle; the expected parking position information acquired in the way is accurate, and a basic basis can be provided for accurate parking of the mining truck; the GPS coordinates of the expected stopping location are sent to the planned stopping route for the mining truck.
Description
Technical Field
The invention relates to the technical field of mining area automatic driving, in particular to a mining area automatic driving loading and unloading point accurate parking method based on visual assistance.
Background
Windy, severe cold, dangerous and severe mining environments have extremely high requirements on the skill and experience of drivers, and even drivers with rich driving experience have certain challenges. With the development of the automatic driving technology, the mine area is transported to an inflection point, namely a vehicle driving main body is about to be changed from a human to a machine, and the unmanned driving technology is adopted, so that the harm or threat to the health and safety of drivers can be avoided or reduced, the efficiency can be greatly improved, the cost can be reduced, and the method is more economic, energy-saving and environment-friendly.
A typical workflow for mine transportation can be briefly summarized as "start-load-transport-unload-stop". At present, researches aiming at the loading and unloading process mainly focus on improving the loading and unloading efficiency through cooperative loading, however, the parking position of a loading and unloading point is in dynamic change and cannot be realized through a tracing mode. Mining area roads are different from urban structured roads, and mining area loading and unloading berth district does not have obvious characteristic, hardly passes through the accurate discernment of sensor, realizes that the accurate degree of difficulty of berthing of vehicle at the loading and unloading point is great.
Disclosure of Invention
In view of the above, the invention provides a method for accurately stopping a loading and unloading point of an automatically-driven mining area based on visual assistance, which is used for solving the problem of stopping deviation of the loading and unloading point of an automatically-driven vehicle in the mining area.
The invention provides a visual-assistance-based mining area automatic driving loading and unloading point accurate parking method, which comprises the following steps of:
s1: the camera is installed in front of the auxiliary operation vehicle, and the installation height, the installation pitch angle and the yaw angle of the camera are determined according to a detection area to be covered in the detection range of the camera;
s2: measuring a horizontal distance and a vertical distance between the installation position of the camera and the installation position of the GPS antenna of the auxiliary working vehicle;
s3: placing a checkerboard in front of the camera, and obtaining camera internal parameters, radial distortion coefficients and tangential distortion coefficients by adopting a checkerboard calibration method;
s4: performing radial distortion correction on the image acquired by the camera according to the calibrated radial distortion coefficient, and performing tangential distortion correction on the image acquired by the camera according to the calibrated tangential distortion coefficient;
s5: selecting a plurality of data points from the image after the distortion correction, wherein each data point represents a pixel point in the image, recording the horizontal distance and the vertical distance of each data point and the real ground projection of the camera, storing the picture or video of each data point, and extracting the pixel coordinate of each data point;
s6: determining a conversion matrix from a pixel coordinate system to a world coordinate system according to the installation height, the installation pitch angle and the yaw angle of the camera and calibrated camera internal parameters, and calculating the horizontal distance and the vertical distance between the camera and each data point according to the extracted pixel coordinate of each data point;
s7: comparing the calculated horizontal distance and vertical distance between the camera and each data point with the recorded horizontal distance and vertical distance between each data point and the real ground projection of the camera, finely adjusting the installation height, the installation pitch angle, the yaw angle and the calibrated camera internal parameters of the camera, and updating a conversion matrix from a pixel coordinate system to a world coordinate system;
s8: the auxiliary operation vehicle acquires a picture in front of the auxiliary operation vehicle in real time through an HMI (human machine interface) and selects an expected parking position of the mining truck;
s9: converting the pixel coordinates of the selected expected parking position into the horizontal distance and the vertical distance between the expected parking position and the camera according to the updated conversion matrix, acquiring the relative position of the expected parking position and the GPS antenna of the auxiliary operation vehicle according to the measured horizontal distance and the vertical distance between the camera mounting position and the GPS antenna mounting position of the auxiliary operation vehicle, selecting a Vincenty ellipsoid calculation formula by combining GPS information sent by the auxiliary operation vehicle in real time, and calculating the GPS coordinates of the expected parking position of the mining truck;
s10: and issuing the GPS coordinates of the expected parking position to the mining truck, and planning a parking driving route by the mining truck according to the GPS coordinates of the expected parking position.
In a possible implementation manner, in the method for accurately stopping a loading and unloading point for automatic driving in a mining area based on visual assistance provided by the present invention, in step S3, a checkerboard is placed in front of a camera, and a checkerboard calibration method is used to obtain camera parameters, radial distortion coefficients and tangential distortion coefficients, which specifically includes:
placing a checkerboard in front of the camera, shooting at least 20 pictures containing the checkerboard, inputting the collected images into a calibration tool box of matlab, and obtaining camera internal reference, radial distortion coefficient and tangential distortion coefficient.
In a possible implementation manner, in the above mining area automatic driving loading and unloading point precise parking method based on visual assistance provided by the present invention, in step S4, the radial distortion correction and the tangential distortion correction are performed on the image according to the following formulas:
wherein,
the abscissa of the pixel representing the original position of the distortion point on the camera,
a pixel ordinate representing the original position of the distortion point on the camera,
representing the pixel abscissa of the data point in the image after the distortion correction,
representing the pixel ordinate of the orthorectified data point in the image,
；k 1、k 2andk 3in order to be the radial distortion factor,p 1andp 2is the tangential distortion coefficient.
In a possible implementation mode, the visual assistance-based mining area automatic driving loading and unloading point accurate parking is provided in the inventionIn the method, in step S6, a transformation matrix from a pixel coordinate system to a world coordinate systemTComprises the following steps:
wherein,f x to representxThe direction focal length is set according to the direction,f y to representyThe direction focal length is set according to the direction,u 0andv 0respectively representing the number of horizontal pixels and the number of vertical pixels of the phase difference between the principal point and the origin of the pixel coordinate system;hwhich represents the mounting height of the camera and,
the mounting pitch angle of the camera is indicated,
representing a yaw angle;
the horizontal and vertical distances of the camera from each data point are calculated as follows:
wherein,
representing the horizontal distance of the data point from the camera,
representing the vertical distance of the data point from the camera.
In a possible implementation manner, in the above mining area automatic driving loading and unloading point precise parking method based on visual assistance provided by the present invention, in step S9, the relative positions of the expected parking position and the auxiliary work vehicle GPS antenna are:
wherein,
indicating the horizontal distance of the expected parking location from the auxiliary work vehicle GPS antenna,
indicating the vertical distance of the expected parking location from the auxiliary work vehicle GPS antenna,
indicating the horizontal distance of the installation position of the camera from the installation position of the GPS antenna of the auxiliary work vehicle,
indicating the vertical distance of the installation position of the camera from the installation position of the GPS antenna of the auxiliary work vehicle,
indicating the horizontal distance of the expected parking position from the camera,
indicating the vertical distance of the expected parking position from the camera.
According to the mining area automatic driving loading and unloading point accurate parking method based on visual assistance, a camera is mounted in front of an auxiliary operation vehicle, internal and external parameters and distortion coefficients of the camera are calibrated and finely adjusted, and a conversion matrix from a pixel coordinate system to a world coordinate system is updated; the real-time picture transmitted by the camera is displayed in real time by utilizing the HMI, and the expected stopping position of the mining truck is selected through the HMI, so that the operation and the accurate regulation and control of workers are facilitated, and the loading and unloading efficiency of the mining truck can be improved; converting the pixel coordinates of the selected expected parking position into the horizontal distance and the vertical distance between the expected parking position and the camera according to the updated conversion matrix, and then acquiring the relative position of the expected parking position and the GPS antenna of the auxiliary operation vehicle according to the measured horizontal distance and the vertical distance between the installation position of the camera and the installation position of the GPS antenna of the auxiliary operation vehicle; calculating the GPS coordinate of the expected parking position by combining GPS information sent by the auxiliary operation vehicle in real time and adopting a Vincenty ellipsoid calculation formula; therefore, the pixel coordinates of the expected parking position are converted into the GPS coordinates, the obtained expected parking position information is accurate, and a foundation can be provided for accurate parking of the mining truck; and sending the GPS coordinates of the expected parking position to the mining truck, and automatically planning a parking route by the mining truck according to the GPS coordinates of the vehicle and the GPS coordinates of the expected parking position. The invention is oriented to the mine automatic driving vehicle, the required camera and GPS equipment are provided, and most of the mine automatic driving vehicles are equipped, so that no additional sensor is needed, the processing amount of the vehicle-mounted processing equipment is reduced, and the cost is reduced.
Drawings
Fig. 1 is a flowchart of a method for accurately stopping a loading and unloading point of an automatic driving in a mine area based on visual assistance according to embodiment 1 of the present invention;
fig. 2 is a schematic view showing the selection of the expected stopping position of the mining truck in embodiment 1 of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only illustrative and are not intended to limit the present invention.
The invention provides a visual-assistance-based mining area automatic driving loading and unloading point accurate parking method, which comprises the following steps of:
s1: the camera is installed in front of the auxiliary operation vehicle, and the installation height, the installation pitch angle and the yaw angle of the camera are determined according to a detection area to be covered in the detection range of the camera;
s2: measuring a horizontal distance and a vertical distance between the installation position of the camera and the installation position of the GPS antenna of the auxiliary working vehicle;
s3: placing a checkerboard in front of the camera, and obtaining camera internal parameters, radial distortion coefficients and tangential distortion coefficients by adopting a checkerboard calibration method;
s4: performing radial distortion correction on the image acquired by the camera according to the calibrated radial distortion coefficient, and performing tangential distortion correction on the image acquired by the camera according to the calibrated tangential distortion coefficient;
s5: selecting a plurality of data points from the image after the distortion correction, wherein each data point represents a pixel point in the image, recording the horizontal distance and the vertical distance of each data point and the real ground projection of the camera, storing the picture or video of each data point, and extracting the pixel coordinate of each data point;
s6: determining a conversion matrix from a pixel coordinate system to a world coordinate system according to the installation height, the installation pitch angle and the yaw angle of the camera and calibrated camera internal parameters, and calculating the horizontal distance and the vertical distance between the camera and each data point according to the extracted pixel coordinate of each data point;
s7: comparing the calculated horizontal distance and vertical distance between the camera and each data point with the recorded horizontal distance and vertical distance between each data point and the real ground projection of the camera, finely adjusting the installation height, the installation pitch angle, the yaw angle and the calibrated camera internal parameters of the camera, and updating a conversion matrix from a pixel coordinate system to a world coordinate system;
s8: the auxiliary operation vehicle acquires a picture in front of the auxiliary operation vehicle in real time through an HMI (human machine interface) and selects an expected parking position of the mining truck;
s9: converting the pixel coordinates of the selected expected parking position into the horizontal distance and the vertical distance between the expected parking position and the camera according to the updated conversion matrix, acquiring the relative position of the expected parking position and the GPS antenna of the auxiliary operation vehicle according to the measured horizontal distance and the vertical distance between the camera mounting position and the GPS antenna mounting position of the auxiliary operation vehicle, selecting a Vincenty ellipsoid calculation formula by combining GPS information sent by the auxiliary operation vehicle in real time, and calculating the GPS coordinates of the expected parking position of the mining truck;
s10: and issuing the GPS coordinates of the expected parking position to the mining truck, and planning a parking driving route by the mining truck according to the GPS coordinates of the expected parking position.
The following describes a specific implementation of the above-mentioned visual-assistance-based mining area automatic driving loading and unloading point precise parking method according to a specific embodiment of the present invention in detail.
Example 1:
as shown in fig. 1, the method is divided into two parts, wherein the first part is used for auxiliary work vehicle parameter calibration, and the second part is used for mining truck expected parking position selection.
The first step is as follows: and the auxiliary operation vehicle parameter calibration comprises calibration and fine adjustment of internal and external parameters and distortion coefficients of the camera.
(1) The camera is installed in front of an auxiliary working vehicle (such as an excavator), and the installation height, the installation pitch angle and the yaw angle of the camera are determined according to a detection area to be covered by the detection range of the camera.
(2) Measuring a horizontal distance and a vertical distance between the installation position of the camera and the installation position of the GPS antenna of the auxiliary working vehicle;
(3) a checkerboard is placed in front of the camera, and the internal parameter, the radial distortion coefficient and the tangential distortion coefficient of the camera are obtained by adopting a checkerboard calibration method.
Specifically, a checkerboard is placed in front of the camera, at least 20 pictures containing the checkerboard are shot by rotating the checkerboard front, back, left and right, the rotation angle is not more than 45 degrees, the collected images are input into a calibration tool box of matlab, and camera internal parameters are obtained (the checkerboard is rotated to the front, the back, the left and the right, the collected images are input into a calibration tool box of matlab, the camera internal parameters are acquired (thef x 、f y 、u 0Andv 0) Radial distortion factor (k 1、k 2Andk 3) And tangential distortion coefficient (p 1Andp 2)。
(4) performing radial distortion correction and tangential distortion correction on an image acquired by a camera:
wherein,
indicating deformityThe abscissa of the original position of the change point on the camera,
a ordinate representing the original position of the distortion point on the camera,
representing the pixel abscissa of the data point in the image after the distortion correction,
representing the pixel ordinate of the orthorectified data point in the image,
；k 1、k 2andk 3in order to be the radial distortion factor,p 1andp 2is the tangential distortion coefficient.
(5) Selecting 5-8 data points from the image after distortion correction, wherein each data point represents a pixel point in the image, recording the horizontal distance and the vertical distance of each data point and the real ground projection of the camera, storing the picture or video of each data point, and extracting the pixel coordinates of each data point.
(6) Determining a conversion matrix from the pixel coordinate system to the world coordinate system according to the installation height, the installation pitch angle and the yaw angle of the camera in the step (1) and the camera internal parameters calibrated in the step (3)T：
Wherein,f x to representxThe direction focal length is set according to the direction,f y to representyThe direction focal length is set according to the direction,u 0andv 0respectively representing the number of horizontal pixels and the number of vertical pixels of the phase difference between the principal point and the origin of the pixel coordinate system;hwhich represents the mounting height of the camera and,
the mounting pitch angle of the camera is indicated,
representing a yaw angle;
calculating the horizontal distance and the vertical distance between the camera and each data point according to the pixel coordinates of each data point extracted in the step (5):
wherein,
representing the horizontal distance of the data point from the camera,
representing the vertical distance of the data point from the camera.
(7) And (3) comparing the calculated horizontal distance and vertical distance between the camera and each data point with the horizontal distance and vertical distance of each data point recorded in the step (5) and the real ground projection of the camera, finely adjusting the installation height, the installation pitch angle, the yaw angle and calibrated camera internal parameters of the camera, and updating a conversion matrix from a pixel coordinate system to a world coordinate system so as to reduce the error of manual calibration.
The second step is that: the expected parking positions of the mining trucks are selected, as shown in fig. 2, 1 is a material point (for example, a soil heap or a coal heap in a mining area), 2 is an expected parking position in an actual scene, 3 is the mining truck, 4 is an auxiliary operation vehicle, 5 is an HMI human machine interface, 6 is a material point position under the view angle of a camera on the auxiliary operation vehicle, and 7 is an expected parking position under the view angle of the camera on the auxiliary operation vehicle.
(1) The auxiliary operation vehicle 4 acquires a picture in front of the auxiliary operation vehicle 4 in real time through the HMI 5, and selects an expected parking position 7 of the expected parking position 2 of the mining truck 3 under the view angle of the camera on the auxiliary operation vehicle by combining a material point position 6 under the view angle of the camera on the auxiliary operation vehicle. The HMI 5 supports position selection, information distribution and information display functions.
(2) Converting the pixel coordinates of the selected expected parking position 7 into the horizontal distance and the vertical distance between the expected parking position 2 and the camera according to the updated conversion matrix, and then acquiring the relative positions of the expected parking position 2 and the GPS antenna of the auxiliary work vehicle 4 according to the measured horizontal distance and the vertical distance between the installation position of the camera and the installation position of the GPS antenna of the auxiliary work vehicle:
wherein,
indicating the horizontal distance of the expected parking location from the auxiliary work vehicle GPS antenna,
indicating the vertical distance of the expected parking location from the auxiliary work vehicle GPS antenna,
indicating the horizontal distance of the installation position of the camera from the installation position of the GPS antenna of the auxiliary work vehicle,
indicating the vertical distance of the installation position of the camera from the installation position of the GPS antenna of the auxiliary work vehicle,
indicating the horizontal distance of the expected parking position from the camera,
indicating the vertical distance of the expected parking position from the camera.
(3) The relative positions of the expected parking position 2 and the GPS antenna of the auxiliary work vehicle 4 are combined with the GPS information sent by the auxiliary work vehicle 4 in real time, a vincent ellipsoid calculation formula is selected, and the GPS coordinates of the expected parking position 2 of the mining truck 3 are calculated.
(4) And issuing the GPS coordinates of the expected parking position 2 to the mining truck 3, and planning a parking driving route by the mining truck 3 according to the GPS coordinates of the expected parking position 2.
According to the mining area automatic driving loading and unloading point accurate parking method based on visual assistance, a camera is mounted in front of an auxiliary operation vehicle, internal and external parameters and distortion coefficients of the camera are calibrated and finely adjusted, and a conversion matrix from a pixel coordinate system to a world coordinate system is updated; the real-time picture transmitted by the camera is displayed in real time by utilizing the HMI, and the expected stopping position of the mining truck is selected through the HMI, so that the operation and the accurate regulation and control of workers are facilitated, and the loading and unloading efficiency of the mining truck can be improved; converting the pixel coordinates of the selected expected parking position into the horizontal distance and the vertical distance between the expected parking position and the camera according to the updated conversion matrix, and then acquiring the relative position of the expected parking position and the GPS antenna of the auxiliary operation vehicle according to the measured horizontal distance and the vertical distance between the installation position of the camera and the installation position of the GPS antenna of the auxiliary operation vehicle; calculating the GPS coordinate of the expected parking position by combining GPS information sent by the auxiliary operation vehicle in real time and adopting a Vincenty ellipsoid calculation formula; therefore, the pixel coordinates of the expected parking position are converted into the GPS coordinates, the obtained expected parking position information is accurate, and a foundation can be provided for accurate parking of the mining truck; and sending the GPS coordinates of the expected parking position to the mining truck, and automatically planning a parking route by the mining truck according to the GPS coordinates of the vehicle and the GPS coordinates of the expected parking position. The invention is oriented to the mine automatic driving vehicle, the required camera and GPS equipment are provided, and most of the mine automatic driving vehicles are equipped, so that no additional sensor is needed, the processing amount of the vehicle-mounted processing equipment is reduced, and the cost is reduced.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (5)
1. A mining area automatic driving loading and unloading point accurate parking method based on visual assistance is characterized by comprising the following steps:
s1: the camera is installed in front of the auxiliary operation vehicle, and the installation height, the installation pitch angle and the yaw angle of the camera are determined according to a detection area to be covered in the detection range of the camera;
s2: measuring a horizontal distance and a vertical distance between the installation position of the camera and the installation position of the GPS antenna of the auxiliary working vehicle;
s3: placing a checkerboard in front of the camera, and obtaining camera internal parameters, radial distortion coefficients and tangential distortion coefficients by adopting a checkerboard calibration method;
s4: performing radial distortion correction on the image acquired by the camera according to the calibrated radial distortion coefficient, and performing tangential distortion correction on the image acquired by the camera according to the calibrated tangential distortion coefficient;
s5: selecting a plurality of data points from the image after the distortion correction, wherein each data point represents a pixel point in the image, recording the horizontal distance and the vertical distance of each data point and the real ground projection of the camera, storing the picture or video of each data point, and extracting the pixel coordinate of each data point;
s6: determining a conversion matrix from a pixel coordinate system to a world coordinate system according to the installation height, the installation pitch angle and the yaw angle of the camera and calibrated camera internal parameters, and calculating the horizontal distance and the vertical distance between the camera and each data point according to the extracted pixel coordinate of each data point;
s7: comparing the calculated horizontal distance and vertical distance between the camera and each data point with the recorded horizontal distance and vertical distance between each data point and the real ground projection of the camera, finely adjusting the installation height, the installation pitch angle, the yaw angle and the calibrated camera internal parameters of the camera, and updating a conversion matrix from a pixel coordinate system to a world coordinate system;
s8: the auxiliary operation vehicle acquires a picture in front of the auxiliary operation vehicle in real time through an HMI (human machine interface) and selects an expected parking position of the mining truck;
s9: converting the pixel coordinates of the selected expected parking position into the horizontal distance and the vertical distance between the expected parking position and the camera according to the updated conversion matrix, acquiring the relative position of the expected parking position and the GPS antenna of the auxiliary operation vehicle according to the measured horizontal distance and the vertical distance between the camera mounting position and the GPS antenna mounting position of the auxiliary operation vehicle, selecting a Vincenty ellipsoid calculation formula by combining GPS information sent by the auxiliary operation vehicle in real time, and calculating the GPS coordinates of the expected parking position of the mining truck;
s10: and issuing the GPS coordinates of the expected parking position to the mining truck, and planning a parking driving route by the mining truck according to the GPS coordinates of the expected parking position.
2. The method for accurate parking at a mine automatic driving loading and unloading point based on visual assistance as claimed in claim 1, wherein step S3 is to place a checkerboard in front of the camera and to obtain the camera internal parameter, the radial distortion coefficient and the tangential distortion coefficient by using a checkerboard calibration method, which specifically comprises:
placing a checkerboard in front of the camera, shooting at least 20 pictures containing the checkerboard, inputting the collected images into a calibration tool box of matlab, and obtaining camera internal reference, radial distortion coefficient and tangential distortion coefficient.
3. The method for accurate docking of a loading and unloading point for automatic driving in a mining area based on visual assistance as claimed in claim 1, wherein in step S4, the radial distortion correction and the tangential distortion correction are performed on the image according to the following formulas:
wherein,
indicating distortion pointThe pixel abscissa of the original position on the camera,
a pixel ordinate representing the original position of the distortion point on the camera,
representing the pixel abscissa of the data point in the image after the distortion correction,
representing the pixel ordinate of the orthorectified data point in the image,
；k 1、k 2andk 3in order to be the radial distortion factor,p 1andp 2is the tangential distortion coefficient.
4. The method for accurate docking of a loading and unloading point for automatic driving in mining area based on visual assistance as claimed in claim 3, wherein in step S6, the transformation matrix from pixel coordinate system to world coordinate systemTComprises the following steps:
wherein,f x to representxThe direction focal length is set according to the direction,f y to representyThe direction focal length is set according to the direction,u 0andv 0respectively representing the number of horizontal pixels and the number of vertical pixels of the phase difference between the principal point and the origin of the pixel coordinate system;hwhich represents the mounting height of the camera and,
the mounting pitch angle of the camera is indicated,
representing a yaw angle;
the horizontal and vertical distances of the camera from each data point are calculated as follows:
wherein,
representing the horizontal distance of the data point from the camera,
representing the vertical distance of the data point from the camera.
5. The method for accurate docking of a loading and unloading point for mine automation based on visual assistance as claimed in claim 1, wherein in step S9, the expected docking position and the auxiliary work vehicle GPS antenna relative position are:
wherein,
indicating the horizontal distance of the expected parking location from the auxiliary work vehicle GPS antenna,
indicating the vertical distance of the expected parking location from the auxiliary work vehicle GPS antenna,
indicating the horizontal distance of the installation position of the camera from the installation position of the GPS antenna of the auxiliary work vehicle,
indicating the vertical distance of the installation position of the camera from the installation position of the GPS antenna of the auxiliary work vehicle,
indicating the horizontal distance of the expected parking position from the camera,
indicating the vertical distance of the expected parking position from the camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110688400.6A CN113135181B (en) | 2021-06-22 | 2021-06-22 | Mining area automatic driving loading and unloading point accurate parking method based on visual assistance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110688400.6A CN113135181B (en) | 2021-06-22 | 2021-06-22 | Mining area automatic driving loading and unloading point accurate parking method based on visual assistance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113135181A CN113135181A (en) | 2021-07-20 |
| CN113135181B true CN113135181B (en) | 2021-08-24 |
Family
ID=76815884
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110688400.6A Active CN113135181B (en) | 2021-06-22 | 2021-06-22 | Mining area automatic driving loading and unloading point accurate parking method based on visual assistance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113135181B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108765496A (en) * | 2018-05-24 | 2018-11-06 | 河海大学常州校区 | A kind of multiple views automobile looks around DAS (Driver Assistant System) and method |
| CN108873904A (en) * | 2018-07-04 | 2018-11-23 | 北京踏歌智行科技有限公司 | The unmanned parking scheme of mine vehicle, equipment and readable storage medium storing program for executing |
| CN109343041A (en) * | 2018-09-11 | 2019-02-15 | 昆山星际舟智能科技有限公司 | The monocular distance measuring method driven for high-grade intelligent auxiliary |
| CN111427348A (en) * | 2020-03-24 | 2020-07-17 | 江苏徐工工程机械研究院有限公司 | Automatic drive mining dump truck environmental perception system and mining dump truck |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8233045B2 (en) * | 2007-07-16 | 2012-07-31 | Trw Automotive U.S. Llc | Method and apparatus for distortion correction and image enhancing of a vehicle rear viewing system |
| US9934572B2 (en) * | 2015-12-17 | 2018-04-03 | Ford Global Technologies, Llc | Drawbar scan solution for locating trailer hitch point |
-
2021
- 2021-06-22 CN CN202110688400.6A patent/CN113135181B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108765496A (en) * | 2018-05-24 | 2018-11-06 | 河海大学常州校区 | A kind of multiple views automobile looks around DAS (Driver Assistant System) and method |
| CN108873904A (en) * | 2018-07-04 | 2018-11-23 | 北京踏歌智行科技有限公司 | The unmanned parking scheme of mine vehicle, equipment and readable storage medium storing program for executing |
| CN109343041A (en) * | 2018-09-11 | 2019-02-15 | 昆山星际舟智能科技有限公司 | The monocular distance measuring method driven for high-grade intelligent auxiliary |
| CN111427348A (en) * | 2020-03-24 | 2020-07-17 | 江苏徐工工程机械研究院有限公司 | Automatic drive mining dump truck environmental perception system and mining dump truck |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113135181A (en) | 2021-07-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2395257C (en) | Any aspect passive volumetric image processing method | |
| CN205024577U (en) | Self -walking -type building machine | |
| WO2015029934A1 (en) | Camera calibration device, camera calibration system, and camera calibration method | |
| US20050179801A1 (en) | Camera corrector | |
| US20110164037A1 (en) | Aerial image generating apparatus, aerial image generating method, and storage medium havng aerial image generating program stored therein | |
| CN111815713A (en) | Method and system for automatically calibrating external parameters of camera | |
| JP2009237901A (en) | Method of creating road marker map | |
| CN111145362A (en) | Virtual-real fusion display method and system for airborne comprehensive vision system | |
| CN110555813B (en) | Rapid geometric correction method and system for remote sensing image of unmanned aerial vehicle | |
| JP5281424B2 (en) | Road marking map generation method | |
| CN115235493A (en) | A method and device for automatic driving positioning based on vector map | |
| JP5915268B2 (en) | Parameter calculation method, information processing apparatus, and program | |
| JP2019533875A (en) | Method and system for generating a composite top view image of a road | |
| CN115683128B (en) | Port unmanned integrated card visual characteristic matching and positioning method | |
| CN104268884B (en) | A kind of calibration system and method for the lane departure warning based on car networking | |
| CN113135181B (en) | Mining area automatic driving loading and unloading point accurate parking method based on visual assistance | |
| CN114004949B (en) | Airborne point cloud-assisted mobile measurement system placement parameter checking method and system | |
| US11403770B2 (en) | Road surface area detection device | |
| CN118485719A (en) | Excavator bucket position determining method and device and excavator | |
| JP2974316B1 (en) | Method for restoring two-dimensional position information of local coordinates from bird's-eye view photograph, system for restoring the two-dimensional position information, and computer-readable recording medium recording a program of the method | |
| JP5339753B2 (en) | Road marking map generation method | |
| CN116915929A (en) | Video enhancement display system for immersed tunneling working face | |
| CN115357052A (en) | Method and system for automatically exploring interest points in video picture by unmanned aerial vehicle | |
| CN116907511B (en) | Method for converting pipeline coordinates into image coordinates | |
| CN115147738B (en) | Positioning method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |