CN110490903B - Multi-target rapid capturing and tracking method in binocular vision measurement - Google Patents

Abstract

The invention provides a multi-target fast capturing and tracking method in binocular vision measurement, belonging to the field of computer vision measurement. The multi-target fast capture and tracking method includes image preprocessing, target capture, multi-target recognition, and multi-target tracking under the condition of strong clutter background. The present invention can realize real-time processing of massive data, and can stably capture and track multiple beacon points on objects under strong clutter conditions such as sunlight, sky clouds, imaging noise, sea level reflection, etc. Punctuation points and their spatial positions are correspondingly matched, and the invention can improve the stability and robustness of multi-target recognition and tracking in binocular vision measurement.

Description

A method for fast acquisition and tracking of multiple targets in binocular vision measurement

technical field

The invention relates to a method for fast capturing and tracking of multiple weak and small targets in a complex background, in particular to a method for capturing and tracking multiple cooperative target sources (beacons referred to below) in binocular vision measurement. It belongs to the fields of image processing and computer vision measurement.

Background technique

With the development of optoelectronic technology and the integration of cross-disciplines, the use of computer vision measurement methods for non-contact measurement is becoming an important measurement method. Vision measurement technology can not only measure the distance of the object but also measure the three-dimensional space position and attitude angle of the object. Visual measurement technology has the characteristics of low cost and long working distance. As long as the measurement camera can effectively extract a certain number of feature points on the object, it can theoretically measure the position of an object at any distance. Now visual measurement technology is widely used in aerospace, aviation, navigation and other fields, as well as in machinery manufacturing, medicine, biology, architecture and other fields.

To quickly and real-time measure the position and attitude of an object in space relative to a certain reference coordinate system, the primary problem is to quickly capture and accurately track a certain number of cooperative beacon points on the object. By extracting the coordinates of the beacon points in the image and combining the coordinates of the corresponding beacon points on the measured object, the projection mapping equations can be established, and the rotation and translation matrix of the measured object relative to the reference coordinate system can be calculated through the principle of photogrammetry. That is, the position and attitude angle of the measured object in space are measured.

In the process of realizing the present invention, the inventor finds that the prior art has at least the following deficiencies:

Most of the target capture and tracking devices in binocular vision measurement systems are based on PCs. However, PCs are large in size, high in power consumption, and poor in stability, which is difficult to meet engineering applications. It is urgent to develop a stable high-speed embedded processing platform for dual-camera Real-time processing of large amounts of data; traditional multi-target acquisition and tracking methods mostly process each beacon in isolation, without considering the rigid connection between different beacon points on both sides of the same object in binocular vision measurement, and the effective use of binocular The unique result data of the eye measurement system, that is, the position and attitude information of the measured object, in order to improve the stability of target capture and tracking.

Contents of the invention

The purpose of the present invention is to provide a multi-target fast capture and tracking method in binocular vision measurement, to realize the real-time processing of massive data, and at the same time under the condition of strong clutter (sun illumination, sky clouds, imaging noise, sea level reflection, etc.) Stably capture and track multiple beacon points on the object, and match each tracked image beacon with its physical space position.

A method for capturing and tracking multiple targets in binocular measurement of the present invention, comprising: 1) image preprocessing under strong clutter background conditions; 2) target capturing; 3) multiple target recognition; 4) multiple target tracking, each Introduce step by step.

1) Image preprocessing under strong clutter background conditions. The filter method is used to suppress the interference of background clutter and enhance the target energy.

The background clutter mainly includes sunlight, sky clouds, sea level reflection and the like.

The filtering method refers to using a 5*5 high-pass filter template to convolve the image to enhance beacon points in the image and suppress background clutter.

2) Target capture. By extracting all possible candidate targets in the image, a candidate target set is formed to prepare for the next step of target recognition. The target acquisition includes target adaptive threshold segmentation and target search and region labeling.

The target adaptive threshold segmentation refers to automatically calculating the segmentation threshold according to the characteristics of the global image, and using the threshold to segment the image into a binary image with 1 in the foreground and 0 in the background.

The target search and region marking refers to calculating all connected regions for the divided binary image, and counting the attribute characteristics of each connected region, including mathematical morphology processing; multi-target region marking; target merging and separation; multi-target features extract.

The mathematical morphology processing refers to processing the binary image using the opening operation in mathematical morphology, that is, first corroding the image, and then performing an expansion operation to remove false connected regions with a smaller area and fill in larger areas. Connected area.

The labeling of multi-target regions refers to representing the pixels in the binary image conforming to a certain connectivity rule (4-neighborhood connectivity and 8-neighborhood connectivity) with the same marker, so that the markers of the connected regions are the same.

The target merging and separation means that if two target areas are very close, the two areas are combined to form one target area; otherwise, the two areas are separated.

The multi-object feature extraction refers to calculating the attribute features of each candidate object in the candidate object set, including the area, energy, aspect ratio, and centroid point coordinates of the object.

3) Multi-target recognition. Identify multiple real beacon points from the candidate target set, including target attribute feature recognition; geometric relationship recognition between targets; target motion continuity recognition.

The target attribute feature recognition refers to removing false targets from the candidate target set according to a certain attribute feature of the target, and retaining the target set that is most likely to be a real beacon point.

The certain attribute features refer to the area, energy and aspect ratio of the candidate target. If the area, energy and aspect ratio of the candidate target exceed a reasonable range, it will be considered as a false target and eliminated.

The recognition of the geometric relationship between the targets means that the projections of multiple beacon points installed on each side of the measured object all present an approximate square relationship on the image, and the geometric position relationship between multiple beacon points can be used to select from the candidate target set Find real multiple beacon points.

The recognition of target motion continuity refers to analyzing each target motion trajectory in order to eliminate false target points from the candidate target set.

The target motion trajectory is based on the feature information (including target area information and target position information) of each frame image when the target is moving, and the smoothness of the change of the target between adjacent image sequences and the continuity of the change of the target coordinate position through statistics judge.

4) Multi-target tracking. The centroid coordinates of each beacon point are extracted from each frame image to form the trajectory of the target, including initial tracking of multiple targets; trajectory prediction of multiple targets; locking and tracking of multiple targets; and recapture of lost targets.

The multi-target initial tracking is to judge whether the plurality of beacon points to be tracked are real beacon points through the continuity of the target's movement between adjacent image sequences, and to construct the trajectory of each beacon at the same time to prepare for target trajectory prediction , is a complement to the multi-object recognition step.

The multi-target trajectory prediction refers to predicting the displacement of the target at the next moment according to the speed and displacement of the target at the current moment.

The multi-target locking and tracking refers to extracting the centroid coordinates of each beacon at the current moment in real time, and establishing the movement track of each beacon.

The target loss recapture means that when the target is lost, the tracking algorithm can continue to lock and track the remaining target points, and use the remaining target points to perform photogrammetry to calculate the position and attitude of the object, as long as two cameras The sum of the number of remaining beacons in the above can be greater than or equal to four; when the lost target reappears, the target loss recapture method can immediately capture the lost target and add it to the photogrammetry equation group to improve Equation solution accuracy, while improving the calculation accuracy of object position and attitude.

The advantage of the present invention compared with prior art is:

(1) The present invention utilizes the image preprocessing method under the strong clutter background condition to reduce the background clutter, improve the signal-to-noise ratio of the image, and reduce the complexity and processing time of subsequent target capture and tracking;

(2) The present invention performs multi-target recognition in a multi-layer progressive manner. Firstly, the preliminary screening of candidate targets is carried out through target attribute feature recognition; then, the geometric relationship between targets is used for further identification, and a large number of candidate target combinations that do not satisfy the geometric relationship are excluded; finally, the time domain recognition is carried out by using the continuity of target motion. The trajectory continuity of the target motion is used for final discrimination. This progressive recognition method improves the efficiency of recognition on the one hand, reduces the amount of calculation, and can also reduce the false recognition rate;

(3) The present invention can stably track multiple targets, and when a certain target is lost, it can still continue to track the remaining targets, and when the lost target reappears, the target loss recapture method can still capture it and add it to the tracking list without changing the tracking number of the target;

(4) The method of the present invention can be widely applied to a binocular vision measurement system to improve the reliability and stability of target capture and tracking in the system.

Description of drawings

Fig. 1 is a schematic flow chart of multi-target fast capture and tracking method in binocular vision measurement of the present invention;

Fig. 2 is a schematic flow chart of the target capture method of the present invention;

Fig. 3 is a schematic flow chart of the target search and region labeling method of the present invention;

4 is a schematic flow chart of the multi-target recognition method of the present invention;

Fig. 5 is a schematic diagram of multiple beacon point numbers on the image of the present invention;

Fig. 6 is a schematic diagram of the projection relationship of multiple beacon points on the image on the side of the object to be measured according to the present invention;

7 is a schematic flow chart of the multi-target tracking method of the present invention;

Fig. 8 is a schematic diagram of the state transition of different constituent units of the multi-target fast acquisition and tracking method of the present invention;

Fig. 9 is a schematic diagram of the principle of binocular vision measurement in the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the implementation method of the present invention will be further described in detail below in conjunction with the accompanying drawings.

The hardware environment for implementing the present invention is: an image processing platform; two cameras; and a measured object. The image processing platform is composed of four 32-bit floating-point ADSP-TS201 digital signal processors with a main frequency of 600MHz and one ultra-large-scale high-speed programmable logic device XC4VLX80. As shown in FIG. 9 , two cameras 31 and 32 are distributed on both sides of the target to be measured, and four beacon lights are installed on both sides of the target to be measured 33 . The implementation process of the present invention is as follows: firstly, the image data streams containing 4 beacon points on both sides of the object to be measured are simultaneously acquired through two cameras, and the two data streams are transmitted to the image processing platform in real time through optical fibers; the image processing platform performs pre-processing on the images processing, multi-target capture and tracking, extracting the centroid coordinates of each beacon point;

As shown in Fig. 1, the present invention is a multi-target rapid acquisition and tracking method in binocular vision measurement, which is used for rapid acquisition, identification and tracking of targets under strong clutter interference conditions. The specific implementation steps of the method are: 1) image preprocessing under the condition of strong clutter background; 2) target capture; 3) multi-target recognition; 4) multi-target tracking. The detailed description of each step is as follows:

1) Image preprocessing under strong clutter background conditions

For the correct implementation of subsequent target acquisition and tracking, an effective preprocessing method must be selected to preprocess images under strong clutter background conditions to improve the energy of the target and remove or weaken the influence of background clutter. After the analysis of experimental data, it is found that the main clutter interference is sunlight, sky clouds, sea clutter, etc., and the object is within a certain measurement range, and the imaging size of the beacon points on both sides is about 3*3~5*5 pixels , the energy of the beacon points on the image is far less than the energy of the sun's light, and the imaging of the beacon points on the image is an isolated singular point.

In view of this, the present invention provides a 5*5 high-pass filter template F to enhance the beacon points in the image and suppress the background clutter. The high-pass filter template F of the 5*5 is:

By convolving the image with the high-pass filter template F, most of the continuously distributed clutter interference in the image can be removed, such as continuously distributed clouds, sunlight, etc., but the discontinuously distributed cloud layer interference still cannot be removed , so in addition to the useful target point data, the filtered image also contains part of the filtered noise point data, which needs to be processed in the subsequent steps.

2) Target capture

As shown in FIG. 2 , the target acquisition includes target adaptive threshold segmentation, target search and region marking, and each step will be described in detail below.

The target adaptive threshold segmentation refers to segmenting the current grayscale image into a binary image whose foreground is 1 and background is 0, specifically by the following method:

a). The mean value of the image after statistical preprocessing:

是图像均值。Where I(x,y) is the pixel gray value with coordinates (x,y), M and N are the length and width of the image,

is the image mean.

b). The variance of the statistical image:

其中a是调节因子，优选为1.75～2.45之间。where S is the variance value of the image; the segmentation threshold of the image is chosen as:

Wherein a is an adjustment factor, preferably between 1.75 and 2.45.

As shown in FIG. 3 , the target search and region marking includes the following steps: (a) mathematical morphology processing; (b) multi-target region marking; (c) target merging and separation; (d) multi-target feature extraction.

(a) Mathematical Morphological Processing. The morphological opening operation is performed on the binary image after adaptive threshold segmentation, that is, the image is eroded first, and then the dilation operation is performed to remove the false connected regions with a small area and fill the connected regions with a large area. Among them, the structural elements used in erosion and expansion are all 3*3 templates. Since this part is a simple logic operation, the parallelism of the algorithm is relatively high, so this part of the operation is carried out in the field programmable logic device. To increase the speed of operation.

(b) Multi-target region labeling. Pixels in the binary image conforming to certain connectivity rules (4-neighborhood connectivity and 8-neighborhood connectivity) are represented by the same mark, and the present invention uses 8-neighborhood connectivity to mark the image processed by mathematical morphology. In the present invention, the number of image lines is N, and the target segment is defined as a set of pixels in which the pixels in one line of the image are continuously 1, and marked as L[i]={Ms, Me, line}, where Ms and Me represent the target in the line row respectively Where the segment starts and ends. The specific implementation steps of the algorithm are:

Step 0: label is initialized to 0, which records the label number of the connected area, and line is initialized to 0;

Step 1: Find an unmarked target segment in the current line, if there is, record it as {Ms, Me, line}, and add 1 to the label, assign the pixel mark in the target segment to the label, and proceed to the second step ~4 steps; if not, go to step 4;

Step 2: If line>0, find all unmarked target segments connected to the current target segment in the previous line, that is, line-1. If there is, mark the pixels in each target segment as label, and at the same time, Repeat steps 2 to 3 for each target segment; if there is no line or line=0, go to step 3;

Step 3: If line+1<N, find all unmarked target segments connected to the current target segment in the next line, that is, line+1. If there is, mark the pixels in each target segment as label, and at the same time , repeat steps 2 to 3 for each target segment; if not, or line=N-1, do nothing;

Step 4: If the current line ends, add 1 to line (if line=N, end; otherwise, enter step 1); if the current line does not end, enter step 1.

After the above 5 steps, the multi-target area marking can be completed, and each connected target area is given a unique number.

(c) Target merging and separation. On the basis of the above multi-target area marking, that is, if two target areas are very close, the two areas are merged to form one target area; otherwise, the two areas are separated. Where two target areas are adjacent means that the area edges of the two targets are smaller than the threshold T.

The threshold T is preferably between 3 and 5 pixels.

(d) Multi-target feature extraction. Calculate the attribute characteristics of each candidate target in the candidate target set to provide criteria for the next target recognition.

The candidate target set refers to each remaining connected target region after the target merging and separation in the previous step.

The attribute features of the candidate targets include: the area, energy, aspect ratio, and centroid point coordinates of each candidate target. A feature structure is established for each candidate target, so that several feature structures are created for several candidate targets, and finally an array of feature structures is output, which is the attribute feature set of candidate targets.

3) Multi-target recognition

As shown in FIG. 4 , the multi-target recognition includes the following steps: (a) target attribute feature recognition; (b) geometric relationship recognition between targets; (c) target motion continuity recognition.

(a) Target attribute feature recognition. False targets are removed from the candidate target set by the attribute characteristics of the target, and the target set that is most likely to be the real beacon point is retained. The specific method is as follows: first, use the area and energy criteria of the target to judge whether the target in each candidate target set is a false target. According to the change range of the distance of the measured object relative to the camera when it moves in space, and the actual size of the beacon point, the area and energy change range of the beacon point on the image can be approximately judged by the principle of pinhole imaging. If a candidate target If the area or energy exceeds this range, it will be considered as a false target and deleted from the candidate target set; secondly, use the aspect ratio attribute of the target to discriminate the remaining candidate target sets, because the real laser target source is a circular spot , its imaging on the image is also approximately circular, or elliptical, so if the aspect ratio of a candidate target exceeds the threshold T, it is considered to be a false target and deleted from the candidate target set. The threshold T is preferably 1.2 ~1.5.

The above completes the target attribute feature recognition step. Through this operation, a large number of false targets can be removed from the candidate feature set, which reduces the amount of calculation for the next step of identifying the geometric relationship between targets.

(b) Geometric relationship recognition between objects. The geometric relationship between multiple beacon points installed on the measured object is used for identification.

When measuring the position and attitude of an object, at least 4 beacon points on the object need to be provided to obtain the spatial position of the object. In the method of the present invention, 4 beacon points are respectively installed on the left and right sides of the measured object, and there are 8 beacon points in total. In order to ensure the accuracy and robustness of the measurement, it is necessary to match the beacon points extracted on the image with the spatial coordinate points on the object.

As shown in Figure 6, the four beacon points installed on each side of the measured object are in an approximately square relationship, and the four beacon points are also approximately in the same plane. According to the distance of the measured object relative to the camera movement and Angle, as well as knowledge of photographic geometry can determine the geometric relationship of the four beacon points projected on the image plane. Figure 5 shows the projection number of 4 beacons on the image. The beacon point in the upper left corner of the measured object is projected on the image as number I, the upper right corner is number II, the lower left corner is number III, and the lower right corner is number IV. Let the candidate target set be O={o ₁ ,o ₂ ,...,o _n }, where n is the number of candidate targets in the target set, and specifically identify the 4 beacon points corresponding to the 4 beacon points in the measured object space on the image. The steps for each point are as follows:

Step 1: Sort the vertical coordinates of the target centroid coordinates in the candidate target set O={o ₁ ,o ₂ ,...,o _n } from small to large, so as to ensure that the target point above the image field of view appears in the candidate The forefront of the target set, while judging the number n of candidate target sets, if n=4, then turn to the 6th step;

然后转入下一步；Step 2: Select the i-th candidate target point in the candidate target set, where i=1,...,n, and assume it is the beacon point numbered I in Figure 5, and then judge whether there is Beacon points numbered II, III, and IV in Fig. 5, at the same time

Then go to the next step;

Step 3: For j=1,...,n and j≠i, judge whether the following conditions are satisfied:

① x _i +t ₂ ＞ x _j ＞ x _i +t ₁

Wherein, t ₁ is preferably between 10-15 pixels; t ₂ is preferably between 30-35 pixels; t ₃ is preferably between 0.75-0.80. If the jth candidate target point satisfies the above two conditions, it is preliminarily judged to be the beacon point numbered II in Fig. step;

Step 4: For k=1,...,n and k≠i, k≠j, judge whether the following conditions are satisfied:

①y _i +t ₂ ＞y _k ＞y _i +t ₁

If the kth candidate target point satisfies the above two conditions, it is preliminarily judged to be the beacon point numbered III in Fig. step;

Step 5: For l=1,...,n and l≠i, l≠j, l≠k, judge whether the following conditions are satisfied:

①x _k +t ₂ ＞x _l ＞x _k +t ₁

②y _j +t ₂ ＞y _l ＞y _j +t ₁

Among them, t ₄ is preferably between 0.5 and 0.65. If the first candidate target point satisfies the above three conditions, then it is preliminarily judged to be the beacon point numbered IV in Fig. step;

Step 6: Judging the geometric imaging relationship of the 4 candidate target points. According to the calculation of photographic geometry knowledge and the analysis of experimental data, it is found that the imaging of the four beacon points on the image cannot be a concave quadrilateral, only a convex quadrilateral, and the four beacon points are distributed at the center point of the four beacon points as the origin. , the horizontal right direction is the abscissa, and the vertical downward direction is the ordinate within the four quadrants of the coordinate system. The specific discriminative steps are as follows: first calculate the center points of the four target candidate target points, that is,

With the center point as the origin (X ₀ , Y ₀ ), establish a coordinate system, and first find out whether there is a point in the first quadrant among the 4 candidate target points, that is, if the coordinates of a candidate target point satisfy: x _i < X ₀ and y _i <Y ₀ , then this point is the beacon point numbered I in Figure 5; if the coordinates of a candidate target point satisfy: x _i >X ₀ and y _i <Y ₀ , then this point is the 5 is the beacon point II; if the coordinates of a candidate target point satisfy: x _i >X ₀ and y _i >Y ₀ , then the point is the beacon point III in Figure 5; if a candidate target point’s The coordinates satisfy: x _i >X ₀ and y _i >Y ₀ , then this point is the beacon point numbered IV in FIG. 5 . If the 4 candidate target points all correspond to the 4 beacon points in Figure 5, then go to the next step, otherwise return to step 2;

Step 7: Judge the distance between the four candidate beacon points. As shown in Figure 5, four beacon points can be connected in series to form a quadrilateral. By calculating the length and width of the four sides of the quadrilateral, it is possible to further determine whether the four candidate points to be identified are real beacon points. That is, if the length and width of the quadrilateral are less than the minimum threshold and greater than the maximum threshold, then the 4 points are not all real beacon points, and return to step 2, otherwise find 4 real beacon points and end the judgment.

Wherein, the minimum threshold is preferably between 10-15, and the maximum threshold is preferably between 30-35.

(c) Object motion continuity recognition. The trajectory of each target motion is analyzed to remove false target points from the candidate target set. Among them, the trajectory of the target movement is to use the feature information (including target area information and target position information) of each frame image when the target is moving, and through the statistics of the smoothness of the change of the target between adjacent image sequences and the continuity of the change of the target coordinate position sex to judge.

Among them, the method of counting the smoothness of the change of the target area between adjacent image sequences is: set the current moment as t, and count the mean value of the target area in the previous n frames of images:

时，则认为目标面积变化不平滑，将该目标从候选目标集中去除。所述n优选5～7之间。If the target area at the current moment t satisfies:

, it is considered that the change of the target area is not smooth, and the target is removed from the candidate target set. The n is preferably between 5-7.

The method for counting the continuity of the change of the target coordinate position is based on the characteristics of the image data stream collected by the camera in real time, the change of the coordinate position of the target between adjacent image sequences will not produce sudden changes, and at the same time the distance of the target from the camera is different from that on the image. The size of the imaging area will also change. If the target is relatively close to the camera, the target will have a larger imaging area on the image, and the movement distance between adjacent image sequences will also be relatively large; The area is relatively small, and the motion distance between adjacent image sequences is also relatively small. Using this characteristic, the moving distance of the target between adjacent image sequences is divided by the area of the target. If the ratio exceeds the threshold T, it is considered that the target coordinate displacement does not change smoothly, and the target is removed from the candidate target set. The threshold T is preferably between 20% and 30%.

4) Multi-target tracking

The multi-target tracking includes the following steps: (a) multi-target initial tracking; (b) multi-target trajectory prediction; (c) multi-target locking and tracking; (d) reacquisition of lost targets. The specific operation of each step is introduced below.

(a) The multi-target initial tracking is a supplement to the multi-target recognition step, that is, judge whether the 4 beacon points to be tracked are real beacon points through the continuity of the movement of the target between adjacent image sequences, and according to the object to be measured The real speed of movement in space and the distance of the object to be measured relative to the camera can estimate the movement distance of the object between adjacent sequences. The specific estimation method is as follows:

Among them, f is the focal length of the camera lens; L is the distance of the object to be measured relative to the camera; σ is the pixel size of the camera CCD; d is the moving distance of the beacon between adjacent sequences, that is, the value to be estimated; D is The moving speed of the measured object, P is the frequency of the camera. Since it is an approximate estimate, the motion component of the object relative to the direction of the camera optical axis is not considered. By judging the maximum value of D, the maximum value of d can be estimated. If the initially tracked beacon points move more than the maximum value of d between adjacent sequences, it can be judged that the target is a false target. If the movement distances of beacons in consecutive T frames between adjacent sequences are all within the maximum value, enter the next step, where T is preferably 4.

(b) According to the present invention, the multi-target trajectory prediction refers to predicting the displacement of the target at the next moment according to the speed and displacement of the target at the current moment. Due to the real-time data sampling, the movement displacement of the target at adjacent moments is small, and the linear prediction method can be used to estimate the displacement of the target. The displacement prediction and estimation formula of the target at the next moment k+1 is as follows:

Among them, f(k), f(k-1), f(k-2) are the displacement of the target in the kth, k-1 and k-2 image sequences respectively.

其中S_i为第i信标点的面积。每个信标点的质心由下式计算：(c) According to the present invention, the multi-target locking and tracking refers to extracting the centroid position of each beacon at the current moment in real time, and establishing the movement trajectory of each beacon. The specific calculation process is as follows: take the predicted beacon position of the target trajectory as the center, establish a window area of T*T size, and calculate the centroid position of the area. where T is preferably

where S _i is the area of the i-th beacon point. The centroid of each beacon point is calculated by the following formula:

Among them, I(x,y) is the gray value of the image at position (x,y), and (X _i ,Y _i ) is the centroid of the i-th beacon point.

(d) According to the present invention, the target loss recapture means that when the target is lost, the tracking algorithm can continue to lock and track the remaining target points, and use the remaining target points to perform photogrammetry to calculate the position and Attitude, as long as the sum of the number of remaining beacons in the two cameras is greater than or equal to 4; when the lost target reappears, the target loss recapture method can immediately capture the lost target and add it to the camera In the measured equations, the accuracy of solving the equations is improved and the calculation accuracy of the position and attitude of the object is improved.

The basic principle of target loss reacquisition method is based on projection error minimization. As shown in Figure 6, the four beacon points on one side of the measured object 121 in space can be projected into the image plane 122 through photographic transformation, according to the following transformation formula:

The rotation matrix R ₁ R ₂ and the translation vector R ₁ T ₂ +T of the three-dimensional coordinate points on the measured object relative to the camera coordinate system can be calculated, where R ₁ and T ₁ are the object's own coordinate system relative to the reference coordinate system Rotation and translation matrix; R ₂ and T ₂ are the rotation and translation matrices of the reference coordinate system relative to the camera coordinate system; X _O is the object coordinate system; X _H is the reference coordinate system; X _C is the camera coordinate system. Since R ₁ and T ₁ are the final photogrammetry results, they can be calculated when no more than 4 targets are lost, and R ₂ and T ₂ are the calibration results of the external parameters of the camera, so the three-dimensional space of the measured object can be calculated according to the principle of pinhole imaging. Coordinates {(X ₁ ,Y ₁ ),...,(X ₈ ,Y ₈ )} are transformed into image plane coordinates {(x ₁ ,y ₁ ),...,(x ₈ ,y ₈ )}. If a certain beacon point i is lost during the tracking process, while tracking other beacon points that are not lost, continue to search in the neighborhood where the missing beacon point i may appear, that is, use the projected point of the lost beacon point i ( _xi , y _i ) as the center of the circle, with the threshold T as the radius, search whether the currently lost beacon point i appears again. The threshold T is preferably between 5 and 7 pixels.

Figure 7 shows the flow chart of multi-target tracking. Firstly, the initial tracking of the target is carried out. When the number of initial tracking frames exceeds 4 frames, it is transferred to the target trajectory prediction process. After the target trajectory prediction predicts the position of the target at the next moment, it enters the target locking and tracking link. The predicted position is the center of the circle, and the center of mass of the area is calculated. This point is the tracked target position. At the same time, it is judged whether there is a target loss. If no target is lost, it will be transferred to the target trajectory prediction process. The process is repeated. If all four beacons on one side of the measured object are lost during the movement, the target acquisition will be carried out again.

Fig. 8 is a schematic diagram of the state transition of different constituent units in the multi-target fast acquisition and tracking method. When the image data stream arrives, first capture the target, and then transfer to the target recognition, if the target recognition unit does not find the real beacon point, then return to the target capture unit to continue the target capture of the next frame, otherwise enter the target initial tracking unit . The initial tracking of the target establishes an initial motion trajectory for each beacon point, and judges the continuity of the trajectory. If it is not continuous, it returns to the target acquisition unit, otherwise, it enters the target locking and tracking unit. Target locking and tracking completes the extraction of target centroid points through a series of processes, and judges whether the target is lost. If all targets are lost, it will enter the target acquisition unit again.

Claims (1)

1. a kind of multi-target fast capture and tracking method in binocular vision measurement, it is characterized in that, described method comprises: Step 1. Image preprocessing under strong clutter background conditions; Step 2, target capture, the target capture includes target adaptive threshold segmentation and target search and area marking; The target adaptive threshold segmentation refers to automatically calculating the segmentation threshold according to the characteristics of the global image, and using the threshold to segment the image into a binary image whose foreground is 1 and the background is 0; The target search and region marking refers to calculating all connected regions for the segmented binary image, and counting the attribute characteristics of each connected region, including mathematical morphology processing, multi-target region marking, target merging and separation, and multi-target features. extract; Step 3, multi-target recognition, which includes target attribute feature recognition, geometric relationship recognition between targets, and target motion continuity recognition; The target attribute feature recognition refers to removing false targets from the candidate target set through the target attribute feature, and retaining the target set that is most likely to be a real beacon point. If the area and energy characteristics of the candidate target exceed a reasonable range, it is considered to be a false target. target, to be eliminated; The recognition of the geometric relationship between the targets means that the projections of the four beacon points installed on each side of the measured object all present an approximate square relationship on the image, and the geometric positional relationship between the four beacon points can be used to select from the candidate target set Find the real 4 beacon points; The recognition of target motion continuity refers to analyzing each target motion trajectory so as to remove false target points from the candidate target set; The target motion trajectory is carried out by using the feature information of each frame image when the target is moving, including the target area information and target position information, through the smoothness of the change of the target between adjacent image sequences and the continuity of the change of the target coordinate position judge; Step 4, multi-target tracking, the multi-target tracking includes multi-target initial tracking, multi-target trajectory prediction, multi-target locked tracking, and target loss recapture; The multi-target initial tracking is to judge whether the 4 beacon points to be tracked are real beacon points through the continuity of the target's motion between adjacent image sequences, and to construct the motion track of each beacon at the same time, which is the core of the multi-target recognition step. Replenish; The multi-target trajectory prediction refers to predicting the displacement of the target at the next moment according to the speed and displacement of the target at the current moment; The multi-target locking and tracking refers to extracting the centroid coordinates of each beacon at the current moment in real time, and establishing the trajectory of each beacon; The target loss recapture means that when the target is lost, the tracking algorithm can continue to lock and track the remaining target points, and use the remaining target points to perform photogrammetry to calculate the three-dimensional space position and attitude of the object, as long as two The sum of the number of remaining beacons in the camera is greater than or equal to 4; when the lost beacons reappear in the image, the target loss recapture method can immediately capture the lost beacons and add them to the photogrammetry In the equation group, the accuracy of solving the equation is improved, and the calculation accuracy of the position and attitude of the object is improved at the same time.

Images