CN105550670A - Target object dynamic tracking and measurement positioning method - Google Patents

Abstract

A dynamic tracking and measurement positioning method for a target object, which uses two cameras to collect images of a monitoring area, dynamically updates the background of the monitoring area and extracts the target object, and uses the principle of binocular recognition and positioning to generate a three-dimensional point cloud of the viewing angle area; combined with the target object Extraction and binocular recognition positioning principle, dynamic tracking and positioning of target objects. The present invention combines visual ranging and target tracking in the field of video surveillance, determines the pixel coordinates of the image where the target is located through dynamic tracking of the target, combines the three-dimensional point cloud generated by visual ranging, locks the target object, and determines its three-dimensional coordinates. When the target object enters the warning area, the system can issue an alarm to achieve the purpose of real-time early warning; the captured position information of the target object provides a scientific basis for the actual manipulation of the background staff.

Description

A method for dynamic tracking, measurement and positioning of target objects

technical field

The invention relates to the technical field of video security and human-computer interaction, in particular to a method for dynamic tracking, measurement and positioning of a target object in the field of video surveillance.

Background technique

At present, the application of video surveillance is becoming more and more common, which has brought immeasurable effects to people's work in the field of security. However, the existing surveillance technology is low in intelligence and still relies on a large number of human resources to identify video content to deal with dangers. and emergencies. Most traditional video surveillance systems can only collect video information in the monitoring area. This monitoring method relies on manual continuous work to detect sudden and dangerous situations in the monitoring area, and lacks intelligent early warning of dangerous information in the monitoring area. The actual operation requires investment A large amount of manpower is used for real-time or post-event analysis, and the images returned by this video surveillance method cannot provide accurate location information of the target object. Operators can only guess the approximate location of the target object based on experience, making the tracking and positioning of the target object inaccurate and lacking intelligence.

Contents of the invention

The invention provides a method for dynamic tracking, measurement and positioning of a target object, which makes up for the deficiency that the pictures returned by traditional video monitoring cannot provide the precise position of the target object, improves the current situation of relying on a large number of human resources, and improves the intelligence level of the video monitoring system.

For this reason, the adopted technical scheme is:

A dynamic tracking and measurement positioning method for a target object, which uses two cameras to collect images of a monitoring area, dynamically updates the background of the monitoring area and extracts the target object, and uses the principle of binocular recognition and positioning to generate a three-dimensional point cloud of the viewing angle area; combined with the target object Extraction and binocular recognition positioning principle, dynamic tracking and positioning of target objects.

The specific steps are as follows:

Step 1, target object extraction: dynamically establish a background library and update it in real time, assign different thresholds to backgrounds with different dynamic degrees, and distinguish the foreground and background parts in the current image according to the difference calculation results between the current image and the image in the background library, and Update the background part to the background gallery;

Step 2, binocular ranging:

(1) Eliminate image distortion and camera correction: Use Taylor series expansion and add correction factors to correct the collected image distortion; use 16 12 The checkerboard is used as a calibration object to calibrate the camera, and the principle of distance minimization and projection maximization is used to ensure that the feature points in the checkerboard image are evenly distributed, and the geometric relationship between the checkerboard feature points and image feature points is used to obtain the coordinate point pair equation to solve The internal and external parameters of the camera correct the distorted image through the internal parameters to obtain a more realistic and natural image; adjust the angle and position of the two images relative to the chessboard through the external parameters, and output the corrected image for line alignment;

(2) Image matching: Take multiple images of the target object in different fields of view at the same time, find the same features of the images captured by the left and right cameras at the same time and different fields of view, analyze the differences, and output the pixel coordinates of the same feature point on the left and right images difference;

(3) Reprojection: convert the pixel coordinate difference result of the same feature point in the left and right images into a distance by triangulation, and output the 3D point cloud of the perspective image;

Step 3, target tracking and positioning: make a difference between any current frame image and the corresponding background image in the images captured by the left and right cameras, dynamically lock the target in the image, and extract its pixel coordinates in the current frame, combined with binocular distance measurement The generated 3D point cloud information determines the 3D point cloud of the target and obtains the coordinates of the target object in the world coordinate system.

In the step 1, the mixed Gaussian model is used to weaken the interference factors similar to the shaking of leaves in the image to reduce the mutual interference between the foreground and the background; effectively separate the foreground and background images of the current frame according to the dynamic threshold, and update the background part of the current image to the background gallery; according to the extracted foreground image, determine the pixel coordinates of the image where the foreground is located, and provide a scientific basis for calculating the three-dimensional world coordinates of the foreground image.

The present invention combines visual ranging and target tracking in the field of video surveillance, determines the pixel coordinates of the image where the target is located through dynamic tracking of the target, combines the three-dimensional point cloud generated by visual ranging, locks the target object, and determines its three-dimensional coordinates. When the target object enters the warning area, the system can issue an alarm to achieve the purpose of real-time early warning; the captured position information of the target object provides a scientific basis for the actual manipulation of the background staff.

To sum up, compared with the existing video surveillance, the present invention has the following advantages: (1) Through the established dynamic background gallery model, through image processing, the target objects entering the monitoring area can be dynamically locked, providing real-time early warning in the security field support. (2) Using the principle of binocular ranging, combined with the foreground extraction of the target object, it can accurately obtain the position information of the target object, make up for the lack of accurate position information of the target object that traditional video surveillance cannot provide, and improve the intelligence level of video surveillance.

Description of drawings

Fig. 1 is a schematic diagram of the general principle of the present invention;

Fig. 2 is a flow chart of target object measurement and positioning in the present invention

Figure 3 is a camera imaging model diagram;

Figure 4 is a schematic diagram of the triangulation method.

detailed description

The present invention will be described in further detail below in conjunction with the accompanying drawings.

The overall principle of the present invention is schematically shown in Figure 1. The left and right cameras of the USB interface are used to collect binocular image information, and the ARM11 development board processes the image to capture the position and geometric size information of the target object for automatic early warning and background staff. provide the basis for corresponding measures.

In the present invention, the measurement and positioning process of the target object is shown in Figure 2, and the binocular image information is collected by the left and right cameras, and 16 12 The checkerboard is the calibration object, and the camera is calibrated stereoscopically using the principle of minimizing the distance and maximizing the projection, obtaining the camera parameters and correcting the distorted image, matching the feature points of the left and right images, and generating a 3D point cloud of the image through triangulation; dynamically setting the similarity The threshold is used to accurately extract the target object, obtain the pixel coordinates of the target object, combine the generated 3D point cloud, capture the position and geometric size information of the target object, and provide a basis for human-computer interaction and intelligent early warning. The specific method is as follows:

Step 1, target object extraction: dynamically establish a background library and update it in real time, assign different thresholds to different dynamic backgrounds, and make a difference between the current image and the image in the background library. When the difference result exceeds the set threshold, you can It is determined that the part of the difference between the current image and the background image exceeding the threshold is the background, and the rest is the foreground. The background part of the image needs to be updated into the background gallery.

Step 2, binocular ranging:

(1) Elimination of image distortion and camera correction: The ideal camera imaging model is a pinhole model, as shown in Figure 3, in order to increase the amount of light transmitted during actual production, the camera uses a lens, but the lens will produce errors in manufacturing and installation , resulting in distortion of the image captured by the camera. In order to minimize the influence of image distortion on image analysis, 16 The 12 chessboard is used as a calibration object to calibrate the camera and solve the internal and external parameters of the camera. The distorted image is corrected by internal parameters to make the image more realistic and natural; the angle and position of the two images relative to the checkerboard are adjusted by external parameters, and the output row alignment image is obtained.

(2) Image matching: Take multiple images of the target object in different fields of view at the same time, find the same features of the images captured by the left and right cameras at different fields of view at the same time, and output the pixel coordinate difference of the same feature point on the left and right images.

(3) Reprojection: The difference result of the pixel coordinates of the same feature points in the left and right images is converted into a distance by triangulation, and the 3D point cloud of the perspective image is output.

Step 3, target tracking and positioning: make a difference between any current frame image and the corresponding background image in the images captured by the left and right cameras, dynamically lock the target in the image, and extract its pixel coordinates in the current frame, combined with binocular distance measurement The generated 3D point cloud information determines the 3D point cloud of the target object, and obtains the coordinate value of the target object in the world coordinate system.

(1) Detailed description of the target object extraction in step 1

In video surveillance, dynamic target objects are often the focus of people's attention, and target object extraction is the core step of intelligent surveillance. Based on the background model, it is necessary to analyze the difference between the current frame image and the image in the background gallery in order to extract the foreground part of the current frame image; however, in actual extraction, the background image is often affected by lighting or complex scenes, so that it is used to distinguish the current frame The threshold of the foreground and background parts of the image cannot be fixed. Therefore, the background model needs to be updated in real time, and the threshold for distinguishing the foreground and background parts of the image needs to be adjusted continuously. The present invention introduces a mixed Gaussian model to weaken the interference factors in the image such as shaking leaves, so as to reduce the mutual interference between the foreground and the background. Using the matching result s between the background model and the current frame image, dynamically adjust the matching similarity threshold K. The relationship between the matching result s and the threshold K is as follows:

Among them, a, b, and m are fixed parameters; when the background changes, the threshold K will be adjusted appropriately to adapt to the background disturbance.

(2) Explain in detail about binocular ranging in step 2

Binocular distance measurement involves two important parts: camera calibration and binocular distance measurement.

Before introducing the camera calibration, first introduce the basic principle of binocular ranging. The ideal binocular ranging model is triangulation as shown in Fig. 4. In Figure 4, the optical axes of the two images aligned with the pixel rows are strictly parallel (the optical axis is the ray drawn from the projection center toward the principal point c) and are the left and right projection centers, respectively, and are the focal lengths of the two cameras and are equal, the principal point and With the same pixel coordinates on the left and right images, the imaging points of the feature point X on the left and right images are respectively and , and The horizontal displacements in the respective pixel coordinate system are and , the disparity is: , let f be the focal length of the camera, using the principle of similar triangles, the equation of the distance Z between the object and the camera lens can be deduced as follows:

In order to establish an ideal binocular ranging platform, it is necessary to perform stereo calibration on the camera. The imaging model of the camera is shown in Figure 3. to 16 12 The checkerboard with crossed black and white grids is used as the camera calibration object, and the intersection points of black and white grids are used as feature points, and the checkerboard feature points are established through transformations such as matrix translation and rotation and image feature points The connection between them, the establishment of equations, and the use of algorithms such as least squares to solve the camera's focal length, distortion coefficient and other parameters.

Traditional camera calibration uses 9 6. The checkerboard with crossed black and white grids is used as the calibration object. The checkerboard has a total of 54 correction feature points, and the number of correction feature points is small, resulting in blind spots in the correction of some distorted areas, which affects the accuracy of visual distance measurement. The present invention adopts dense checkerboard (16 12) As a camera calibration object, it adopts the principle of minimizing the distance and maximizing the projection, which has the following advantages: there are more correction feature points in the image per unit area, and the distortion coefficient with higher accuracy can be obtained; the distance is minimized and the projection is maximized The principle of maximizing the screen-to-body ratio of the checkerboard image in the viewing area can ensure the uniform distribution of feature points in the viewing area and improve the calibration accuracy. This short-range multi-point camera calibration method can improve the camera calibration accuracy, improve the fisheye phenomenon of the image, promote the efficiency of binocular matching, and improve the accuracy of visual distance measurement.

The effect of the present invention is further illustrated through specific application scenarios below:

Scenario 1: Real-time early warning in the field of video surveillance and security. The traditional monitoring method requires the staff to watch the monitoring video for a long time to achieve the purpose of real-time monitoring, which relies on a large number of human resources, and the monitoring efficiency and intelligence level are low. The invention dynamically establishes a background model and updates it in real time, extracts a foreground image through image differential calculation, and combines binocular recognition and positioning principles to dynamically track and locate target objects to realize real-time early warning. The invention frees a large number of human resources in traditional video monitoring from actual work, and improves the intelligence level of the monitoring system.

Scenario 2: Visual distance measurement. Common ranging methods include laser ranging, infrared ranging, ultrasonic ranging, radar ranging, etc. Compared with these ranging methods, the visual ranging used in the present invention does not need to send out a signal to the measured object during measurement. Any signal, simple in principle and low in cost, can measure the position of the target object in a complex environment. At the same time, if the feature points in the space are selected by the mouse, the distance and relative position relationship between feature points can be calculated by using the Pythagorean theorem and the sine-cosine theorem, and the geometric size information of the target object can be further calculated.

Scenario 3: Object edge detection. Common image edge detection algorithms often obtain the contour information of objects by analyzing the first-order or second-order derivatives of image grayscale changes. This type of edge detection algorithm cannot effectively extract the contour information of target objects in complex scenes. The invention generates the depth information of the three-dimensional point cloud according to the visual measurement, and can draw the contours of objects with different depths through the drawing function, so as to realize the accurate extraction of the contours of the specified target objects among multiple foreground objects. This method can be used in the fields of autonomous intelligent operation and visual navigation of robots.

Scenario 4: Human-computer interaction. Most traditional video surveillance systems only collect video information in the monitoring area. This monitoring method provides insufficient interactive information for the staff. The staff needs to judge and speculate the approximate location of the target object based on their own experience. The workload is heavy and the accuracy is low. . According to the principle of binocular recognition and positioning, the present invention obtains the three-dimensional point cloud of the viewing angle area, combines with the extraction of the target object, determines the position information of the target object, and provides the basis for the staff's decision-making.

In summary, the present invention simulates the way the human eye handles the scene, partially replaces the human brain to understand and recognize things in nature, and generates a three-dimensional point cloud of the viewing angle area based on the principle of binocular ranging; based on the dynamically updated background gallery model, through the image The differential operation obtains the pixel coordinates of the target object image; combined with the 3D point cloud information of the viewing angle area and the pixel coordinates of the target object image, the dynamic tracking and positioning of the target object is realized.

Claims (3)

1. A method for dynamic tracking and measurement positioning of a target object, characterized in that: the method utilizes two cameras to collect images of the monitoring area, dynamically updates the background of the monitoring area and extracts the target object, and utilizes the principle of binocular recognition and positioning to generate a three-dimensional perspective area Point cloud; combined with the principle of target object extraction and binocular recognition and positioning, it can dynamically track and locate the target object. 2. a kind of target object dynamic tracking and measurement positioning method according to claim 1, the method is specifically as follows: Step 1, target object extraction: dynamically establish a background library and update it in real time, assign different thresholds to backgrounds with different dynamic degrees, and distinguish the foreground and background parts in the current image according to the difference calculation results between the current image and the image in the background library, and Update the background part to the background gallery; Step 2, binocular ranging: (1) Eliminate image distortion and camera correction: use Taylor series expansion and add correction factors to correct the collected image distortion; use 16*12 chessboard as the calibration object to calibrate the camera, and use the principle of minimizing distance and maximizing projection Ensure that the feature points in the checkerboard image are evenly distributed, and use the geometric relationship between the checkerboard feature points and image feature points to obtain the coordinate point pair equation, thereby solving the internal and external parameters of the camera, correcting the distorted image through internal parameters, and obtaining a more realistic and natural image; through The external parameters adjust the angle and position of the two images relative to the checkerboard, and output the corrected image for row alignment; (2) Image matching: Take multiple images of the target object in different fields of view at the same time, find the same features of the images captured by the left and right cameras at the same time and different fields of view, analyze the differences, and output the pixel coordinates of the same feature point on the left and right images difference; (3) Reprojection: convert the pixel coordinate difference result of the same feature point in the left and right images into a distance by triangulation, and output the 3D point cloud of the perspective image; Step 3, target tracking and positioning: make a difference between any current frame image and the corresponding background image in the images captured by the left and right cameras, dynamically lock the target in the image, and extract its pixel coordinates in the current frame, combined with binocular distance measurement The generated 3D point cloud information determines the 3D point cloud of the target and obtains the coordinates of the target object in the world coordinate system. 3. A kind of target object dynamic tracking and measurement location method according to claim 2, it is characterized in that: use mixed Gaussian model in described step 1, weaken the disturbing factor similar to the shaking of leaves in the image, to reduce foreground and background mutual interference; effectively separate the foreground and background images of the current frame according to the dynamic threshold, and update the background part of the current image to the background gallery; determine the pixel coordinates of the image where the foreground is located according to the extracted foreground image, in order to calculate the 3D of the foreground image World coordinates provide scientific basis.