Standing tree measuring method based on binocular vision
Technical Field
The invention relates to a standing tree measuring method based on binocular vision.
Background
Forest resource investigation is an important means for checking the quantity, quality and distribution of forest resources. The breast height measurement of standing trees and the tree species are important items in forest resource investigation. The breast diameter measurement is also an important basis for evaluating the site conditions and the growth condition of the forest trees. In conventional resource investigation, a great deal of experience is required from the surveyor to identify the tree species. The traditional breast-height diameter measurement of the standing tree comprises caliper measurement, wheel ruler measurement, diameter tape measurement and the like, the contact measurement methods are low in efficiency and high in labor intensity, and an investigator may not reach the vicinity of the standing tree to perform contact measurement when measuring the breast-height diameter of the tree in a forest land in the field. In order to meet the requirements of intelligent development of forestry in China, researchers develop breast-height measuring methods such as a digital forestry diameter measuring instrument, a digital forestry side height distance measuring instrument, an electronic tree measuring gun, an electronic angle gauge, an electronic theodolite, a binocular fixed-focus camera and the like in sequence. Non-contact measurement becomes a practical requirement, and an intelligent determination method is provided for meeting the requirement: the images are acquired by using a USB binocular camera, and the type identification and the breast diameter measurement of the trees are acquired by performing subsequent processing on the images by using a notebook computer (tx2 development board).
Disclosure of Invention
The invention aims to solve the problems that in the prior art, when forest resources are measured in the field, a measurer needs to be in contact with the forest resources to measure, and the measurer needs a large amount of experience and manpower resource consumption, and provides a standing tree measuring method based on binocular vision.
The invention relates to a binocular vision-based stumpage determination method, which comprises the following steps:
firstly, calibrating a binocular camera to obtain internal and external parameters of the binocular camera;
correcting the binocular camera, acquiring a disparity map, and measuring binocular distance;
step three, identifying tree species;
step four, measuring and calculating real breast diameter data;
the invention has the beneficial effects that: the invention solves the problems that the prior art needs contact measurement and a determinator needs a large amount of experience when measuring the field forest resources, realizes automatic statistical calculation, has the advantage of non-contact measurement, measures in forest lands with complex environment, solves the problem of huge physical consumption of the determinator, and has the advantages of low cost, small volume and convenient carrying, and the measurement result can be processed by a computer in real time.
Drawings
FIG. 1 is an overall flow chart for measuring the breast height of a standing tree;
FIG. 2 is a diagram of the effect of a target detection test on a single picture;
FIG. 3 is a schematic diagram of binocular range finding;
FIG. 4 is a schematic view of a cross-cut of a stumpage;
fig. 5 is a flow chart of target detection.
Detailed Description
The first embodiment is as follows: the diameter at breast height of the tree is the diameter of a trunk at 1.3 meters above the ground, and is an important factor in various applications such as acquisition calculation, decision analysis and the like; with reference to fig. 1, the standing tree measurement method based on binocular vision according to the present embodiment includes the following steps:
firstly, calibrating a binocular camera to obtain internal and external parameters of the binocular camera;
correcting the binocular camera, acquiring a disparity map, and measuring binocular distance;
step three, identifying tree species;
step four, measuring and calculating real breast diameter data;
and step five, recording and storing the tree species and the acquired breast-height diameter data of the trees.
The second embodiment is as follows: the difference between the first embodiment and the second embodiment is that, in the first step, the binocular camera is calibrated to obtain the internal and external parameters of the binocular camera, and the specific process is as follows:
the left camera and the right camera are fixed to be a rigid body, the distance between the left camera and the right camera is 10cm, the distortion degree of each lens of the binocular camera cannot be completely the same, internal and external parameters of the cameras can be obtained through camera calibration, and the internal parameters K of the cameras comprise the focal length f of the cameras, the center distance T between the two cameras, the position of a principal point p (a point where a principal axis intersects with an image plane) and the size proportion of pixels to a real environment; the camera extrinsic parameters comprise a three-dimensional calibration rotation matrix R and a three-dimensional calibration translation vector t; 1. opening a camera 2, synchronously grabbing 3 two camera frame images, placing a chessboard calibration plate in front of the two cameras, grabbing more than 20 views, and changing the angle and distance between the plane of the calibration plate and an imaging plane every time a complete chessboard angular point is successfully detected;
xc1=R1xw+t1
xc2=R2xw+t2
wherein x isc1Presentation Camera C1Non-homogeneous coordinates in a coordinate system, xc2Presentation Camera C2Non-homogeneous coordinates in a coordinate system, xwRepresenting the non-homogeneous coordinate of any point Q in space in a world coordinate system, and a rotation matrix R1And a translation vector t1Is a camera C1External parameter of relative position to world coordinate system, rotation matrix R2And a translation vector t2Is a camera C2External parameters of relative position to the world coordinate system; r represents a stereo calibration rotation matrix, and t represents a stereo calibration translation vector.
Other steps and parameters are the same as those in the first embodiment.
The third concrete implementation mode: the difference between the first embodiment and the second embodiment is that the method comprises the following steps of correcting the binocular camera, acquiring a disparity map, and measuring the binocular distance:
according to monocular internal reference data (center distance T, principal point p and distortion coefficient) and binocular external reference data (rotation matrix and translation vector) obtained after camera calibration, respectively carrying out distortion elimination and row calibration on left and right views, enabling the imaging origin coordinates of the left and right views to be consistent, enabling the optical axes of the two cameras to be parallel, enabling left and right imaging planes to be coplanar, and adjusting the left and right views into an ideal mode of completely parallel alignment;
1. correcting the binocular camera:
(1) acquiring left and right views; (2) eliminating distortion for left and right views; (3) binocular parallel correction; (4) image cutting;
2. and (3) stereo matching calculation disparity map: (stereo matching is mainly to find out the corresponding relation between each pair of images, obtain a disparity map according to the triangulation principle, and obtain the depth information and three-dimensional information of an original image according to a projection model after obtaining disparity information):
(1) and (3) calculating matching cost: carrying out gray level similarity measurement under different parallaxes by adopting a gray level difference square method;
(2) matching cost superposition: the reliability of matching cost is enhanced through window superposition by a region algorithm, and the method is mean square error;
(3) parallax acquisition: selecting a point with the optimal superposition matching cost in a certain range, namely, taking a point with the minimum mean square error as a corresponding matching point;
(4) parallax refinement
3. Binocular ranging:
wherein x islThe abscissa, x, representing the target point P in the left camera planerDenotes the abscissa of the target point P on the right camera plane, T denotes the center-to-center distance (distance between the right and left camera optical centers), Z denotes the vertical distance (as shown in fig. 3, i.e., the distance of OD in fig. 4) of the target point P from the center-to-center distance, and f denotes the focal length of the camera.
Other steps and parameters are the same as those in the first or second embodiment.
The fourth concrete implementation mode: the difference between this embodiment and one of the first to third embodiments is that the identification of tree species in the third step includes the following specific processes:
after the image is obtained, trunk information in the image is extracted by using a target detection technology based on a depth convolution neural network (a target detection process is shown in fig. 5, a convolution network in the image is used for extracting features, an RPN network generates a candidate frame of a target region, a full connection layer is used for classifying a feature map and regressing the target region after ROI pooling is carried out), the category of the tree is obtained, a data set and a label need to be prepared in advance for a model for target detection, a plurality of images of various trees need to be collected as much as possible for labeling in the previous preparation, then a target detection model with high accuracy and robustness is obtained by training, the height of a camera is adjusted to be 1.3 meters, the optical axis of the camera (the ray of which the central point of the camera is vertical to the image plane) is parallel to the ground as much as possible, 3-5 frames of images are extracted from a video of one tree for subsequent processing, and the trunk information in each picture can be predicted and position fitted by the model for, the reliability and location parameters are averaged and output (the effect of the target detection test on a single picture is shown in fig. 2).
Other steps and parameters are the same as those in one of the first to third embodiments.
The fifth concrete implementation mode: the difference between this embodiment and one of the first to fourth embodiments is that the step four calculates the true diameter at breast height data, and the specific process is as follows:
because the left view and the right view of the binocular camera are different, according to the disparity map obtained in the second step, three-dimensional information is obtained through the trigonometry principle, the binocular camera observes the same object from two points to obtain images under different visual angles, the offset between pixels is calculated through the triangulation principle according to the pixel matching relation between the images to obtain the three-dimensional information of the object, the depth of field information of the object is obtained, the distance between the trunk and the camera can be calculated, graying is carried out in the trunk frame extracted by the target detection algorithm, edge detection is carried out through a canny operator, and two-point coordinates (x coordinate) for measuring the diameter of the tree in the image coordinate system are obtained (x is x)A,yA) And (x)B,yB):
Wherein, XARepresenting (x) in the image coordinate systemA,yA) Abscissa, X, of the world coordinate system corresponding to the pointBRepresenting (x) in the image coordinate systemB,yB) The abscissa of the world coordinate system corresponding to the point, Z represents the vertical distance from the target point P to the center distance, f represents the focal length of the camera, u represents the focal length of the camera0Representing the offset, v, of the image plane coordinate system to the x-axis of the image coordinate system0Indicating the offset, Y, of the image plane coordinate system to the Y-axis of the image coordinate systemARepresenting (x) in the image coordinate systemA,yA) Ordinate, Y, of the world coordinate system corresponding to the pointBRepresenting (x) in the image coordinate systemB,yB) The vertical coordinate of the corresponding world coordinate system;
assuming that the trunk is a cylinder, when observing a cylindrical object, the diameter of the object reflected by the light is a bright point straight line (i.e. two lines OA and OB) where the observation light is tangent to the trunk on the chest height plane, as shown in fig. 4, the true breast diameter of the standing tree is further calculated:
wherein α represents the angle between OA and OB, SABRepresenting the distance between A, B points in the world coordinate system, r is the radius of the tree under ideal conditions;
calculating the real breast diameter data D of the standing tree according to the following formula:
sin2α+cos2α=1
D=2r。
other steps and parameters are the same as in one of the first to fourth embodiments.