CN1254956C - Calibrating method of pick-up device under condition of traffic monitering - Google Patents

Abstract

A camera calibration method in a traffic surveillance environment, select a group of three parallel sides with known intervals from each other and a straight line intersecting the parallel lines with a known slope from the traffic scene as the required calibration targets, find them in the image Each projected straight line corresponding to , and all intersection coordinates of each projected straight line are obtained, and the focal length, direction and position parameters of the camera are obtained according to these data. The present invention can use the typical features that are easy to extract, such as zebra crossings and lane lines including the edges on both sides of the road, in the urban traffic scene, and determine the relevant parameters of the camera according to the distribution information of many special straight lines on the road surface. The method is simple to implement and is aimed at It has good performance and versatility, and has linear computational complexity. It can be applied to complete camera calibration in many traffic monitoring situations where traditional calibration methods cannot be applied.

Description

Camera Calibration Method in Traffic Surveillance Environment

technical field

The invention relates to a camera calibration method in a traffic monitoring environment, which is mainly used for obtaining camera orientation parameters and focal lengths in a traffic monitoring environment, and provides a basic guarantee for real-time and accurate detection of various traffic information based on video. It belongs to the field of intelligent transportation technology.

Background technique

The automatic collection and processing of traffic information through the traffic intelligent monitoring system is one of the most basic elements in the intelligent transportation system (ITS, Intelligent Transportation System) of various countries and even the world. With the development of sensor technology, communication and network technology, pattern recognition, image processing and With the development of high technology such as computer vision, intelligent traffic information monitoring technology has also made great progress. Generally speaking, traffic status and traffic flow data can be obtained through various types of sensors, such as inductive sensors buried under the road surface and radar, infrared, ultrasonic, microwave sensors, etc. installed on the road surface. However, because the signals obtained by these "point" sensors are not intuitive, the detection accuracy is not high, and the monitoring range and detection parameters are very limited, they cannot provide comprehensive and direct traffic information. In recent years, vehicle recognition and traffic operation status detection technology based on cameras and other visual sensors and computer vision theory has gradually matured, and has begun to be applied to the automatic collection and processing of traffic information, without human intervention or only With little intervention, the detection, segmentation, identification and tracking of dynamic and static vehicles can be realized by analyzing the video sequence recorded by the camera, the behavior of the vehicle can be judged, the traffic status information can be extracted and properly controlled and induced, and the traffic system can be realized. effective management. Camera calibration is a necessary and basic processing step. It can determine the mapping relationship between two-dimensional image coordinates and three-dimensional world coordinates. It is the premise and basic guarantee for the effective extraction of traffic information related to spatial dimensions such as vehicle speed, vehicle type, and accident investigation. .

In the fields of computer vision, industrial measurement, intelligent robot navigation, and quality control, for various specific applications, the commonly used camera calibration methods are as follows: (1) use the calibration block of the three-dimensional structure and the corresponding points of the image points to carry out The method of calibration; (2) utilize the method of vanishing point calibration, this method utilizes the vanishing point that parallel lines form in image and the property of vanishing line to solve camera parameter; (3) utilize the method of plane correspondence matrix calibration; (4 ) A self-calibration method based on corresponding points, which uses the corresponding relationship between the sequence images captured by the camera in two sets of translation or rotation to obtain the internal parameters of the camera and so on. Because these methods do not consider the particularity of traffic scenes and the specific characteristics of traffic scenes, there are certain shortcomings when used in video traffic monitoring occasions, and they lack specific pertinence and versatility. For the camera calibration process in traffic scenes, the convenient and quick extraction and identification of calibration targets is an essential step and requirement. Based on the geometric characteristics of adjacent lane segments on the road, Nelson, Grantham, and George et al. have introduced a new camera calibration method for traffic video surveillance systems (“Anovel camera calibration technique for visual traffic surveillance,” Proc.7 ^th World Congress on Intelligence Transportation Systems, paper no. 3024, 2000). Since in many cases, the quadrilateral formed by two adjacent lane segments parallel and each end point connected clockwise is often a rectangle, this method makes full use of the spatial position relationship of the four vertices of such a rectangular object and their projection on the image plane The image coordinates of the corresponding points can be directly derived to obtain the analytical expressions for the camera focal length and orientation parameters under the premise that the distance between the lanes is known. It is simple to implement, has linear time computational complexity, can complete camera calibration simply and quickly, and is quite pertinent, so it is an efficient calibration method. However, in many traffic situations, the rectangular calibration target is not easy to be found. For example, it is difficult to directly find similar targets in the video surveillance scene of urban traffic intersections. This is a limitation that cannot be ignored. Therefore, this method is used in There is necessarily a lack of versatility in a traffic surveillance environment.

Contents of the invention

The purpose of the present invention is to provide a new camera calibration method in the traffic monitoring environment, which is easy to implement, can fully ensure the pertinence or versatility in the traffic monitoring environment, and meet the needs of intelligent traffic information video. The actual need for accurate extraction of traffic parameters in the monitoring system.

In order to achieve such a purpose, the present invention utilizes some typical features that are easy to extract in the urban traffic scene, and determines camera-related parameters according to the distribution information of many special straight lines on the road surface. As a general new method for calibrating traffic scene cameras, the present invention does not require a regular calibration rectangle, but only needs a set of three parallel sides with known intervals on the road surface and a straight line intersecting the parallel lines with known slope and the distance between them. The projection on the image plane is used to obtain the focal length and orientation parameters of the camera. The distance between the parallel sides of the group needs to be determined in advance, and the selected intersecting straight line and its slope also need to be selected and calculated in advance. In practical applications, lane lines like zebra crossings and the edges on both sides of the road may be a good choice for the required set of parallel edges, and they are also very representative typical features in urban traffic scenes. Straight lines with known slopes and intersecting three parallel sides are also easy to find or preset in traffic scenes, which fully guarantees the pertinence and versatility of the camera calibration method proposed in the present invention in the traffic surveillance environment. To complete the camera calibration, the present invention first selects the required calibration target from the traffic scene, measures the distance between the three parallel lines on the road surface and calculates the spatial slope of the straight line intersecting them, and finds their corresponding projected straight lines in the image And calculate all intersection coordinates of each projected straight line, take these data as the original input, and finally obtain the focal length, direction and position parameters of the camera.

The concrete steps of the inventive method are as follows:

(1) Selection of the calibration target and establishment of the relevant coordinate system: in the traffic scene, select a group of three parallel lines with a known distance from each other on the road surface, and a straight line with a known slope that intersects the three parallel lines as the calibration target, and determine Each projection of the calibration target in the image corresponds to a straight line and a corresponding intersection point. Establish a world coordinate system, set the origin of the world coordinate system at the intersection of a parallel line in the middle and the intersecting straight line, the positive Y axis points forward along the middle parallel line, the positive Z axis is vertical to the ground, and the positive X axis points to the right horizontally . Establish a coordinate system centered on the camera, the coordinate system takes the center of the lens as the origin, the optical axis of the camera is the V axis, and the plane of the U-W coordinate axis is parallel to the image plane, and the distance from the center of the lens is the focal length of the camera.

(2) Calibration input data initialization: Relative to the camera-centered coordinate system, use the least squares method to calculate the coordinates of all intersection points of the calibration targets in the image corresponding to the projections, and at the same time calculate their slopes on the image plane.

(3) Calculation of the vanishing line corresponding to the ground and the vanishing point corresponding to the intersecting straight line: according to the distance between two adjacent straight lines in the three parallel lines, the coordinates of the intersection point between the intersecting straight line and the three parallel lines, and the coordinate system centered on the camera The coordinate parameters of each intersection of the three parallel lines and the intersecting straight lines corresponding to the projected straight lines on the image plane are obtained, and the slope of the corresponding disappearing line on the ground and the general formula equation are obtained, and the intersection coordinates of the projected straight lines and the vanishing lines of the intersecting straight lines on the image plane are obtained , that is to obtain the coordinates of the vanishing point corresponding to the intersecting straight line on the image plane.

(4) Camera parameter calibration: the distance between two adjacent straight lines among the three parallel lines, the coordinates of the intersecting points between the intersecting straight lines and the three parallel lines, the coordinates of the intersection points of the three parallel lines and the intersecting straight lines projected in the image, the slope of the vanishing line, and the intersecting straight line The intersection coordinates of the projected straight line and the disappearing line obtain the rotation angle, deflection angle, pitch angle and camera focal length of the camera, and finally obtain the three-dimensional position coordinates of the camera lens center according to the correlation transformation relationship of the two coordinate systems defined by the present invention, and complete the camera calibration .

The method of the invention is simple to realize, has good pertinence and versatility, and has linear calculation complexity, and can be applied to complete camera calibration in many traffic monitoring occasions where traditional calibration methods cannot be applied.

Description of drawings

FIG. 1 is a schematic diagram of the relationship between the world coordinate system and the calibration target in the present invention.

Fig. 2 is a schematic diagram of the projected image of the calibration target and the vanishing line on the ground in the present invention.

Fig. 3 is the camera coordinate system of the present invention.

Fig. 4 is the original image of the traffic scene used in the embodiment of the present invention.

Fig. 5 is a schematic diagram of a calibration target selected from an original image according to an embodiment of the present invention.

Detailed ways

In order to better understand the technical solution of the present invention, a further detailed description will be made below in conjunction with the accompanying drawings and embodiments.

1) Selection of calibration target and establishment of relevant coordinate system:

In the traffic scene, the calibration target is selected, which is a set of three parallel lines with a known distance from each other and a straight line with a known slope intersecting the set of parallel lines. They are respectively represented as a, b, c, d on the actual road surface, the straight line a, b, c represents three parallel lines, the straight line d intersects them, and the intersection points of the three parallel lines and the straight line d are respectively represented as A, O, B, as shown in Figure 1. Here, the slope of the straight line d has been measured in advance, expressed as m, and the distance between two adjacent parallel lines in the straight lines a, b, c is expressed as d ₁ , d ₂ in sequence. In the present invention, the projection straight lines corresponding to the calibration target in the image are represented as a', b', c', d' respectively, and the intersection point of the straight line d' and the straight line a', b', c' is correspondingly represented as A' , O', B', as shown in Figure 2. It should be noted that the straight line in Figure 2 Indicates the vanishing line corresponding to the road plane in the image. Among them, the point V _D represents the vanishing point corresponding to the parallel line group projection a', b', c', and V _O represents the vanishing point corresponding to the straight line d'.

In order to calibrate the direction and focal length parameters of the camera, two right-handed coordinate systems need to be defined: the world coordinate system and the camera-centered coordinate system. Assuming that the origin of the world coordinate system is located at the intersection point O of straight lines b and d, the positive direction of the Y axis points forward along the line b, the positive direction of the Z axis is vertical to the ground, and the positive direction of the X axis points horizontally to the right, then the intersection points A and O , B coordinates can be expressed as (x _A =-d ₁ , y _A , z _A =0), (x _o =0, y _o =0, z _o =0), (x _B =+d ₂ , y _B , z _B =0), calculate the road surface straight line d slope m=(y _B -y _A )/(d ₂ +d ₁ ).

The camera coordinate system takes the center of the lens as the origin as shown in Figure 3. The V axis is the optical axis of the camera, and the U-W plane located at V=f is parallel to the image plane, where -f represents the camera focal length. Assume that the coordinates of any point on the image plane are represented by (u, w) relative to the camera coordinate system.

2) Calibration input data initialization:

Use the least squares method to calculate the coordinates of the public intersection point V _D of the projected straight lines a', b', c' in the image as (u _D , w _D ), and at the same time obtain the coordinates of their intersection with the straight line d'A',O',B' are (u _A′ , w _A′ ), (u _o′ , w _o′ ), (u _B′ , w _B′ ), assuming that the slopes of the straight lines a′, b′, c′, and d′ are k _{- 1} , k ₀ and k ₁ and k ₂ , then there are

${\mathrm{<span\; class="notranslate">\; k</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; w</span>}}_{{\mathrm{<span\; class="notranslate">\; A</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}}}{{\mathrm{<span\; class="notranslate">\; u</span>}}_{{\mathrm{<span\; class="notranslate">\; A</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="0"\; label="k"\; filenames="C20041001770300102.png"\; state="\{\{state\}\}">k</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; w</span>}}_{{\mathrm{<span\; class="notranslate">\; o</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}}}{{\mathrm{<span\; class="notranslate">\; u</span>}}_{{\mathrm{<span\; class="notranslate">\; o</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; w</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}}}{{\mathrm{<span\; class="notranslate">\; u</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; w</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{{\mathrm{<span\; class="notranslate">\; A</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}}{{\mathrm{<span\; class="notranslate">\; u</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{{\mathrm{<span\; class="notranslate">\; A</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}}<span\; class="notranslate">\; ;</span>$

3) Calculation of vanishing line and vanishing point of intersecting straight line:

Calculate intermediate variable X ₄₅ , X ₅₆ , X ₄₆ , X _456D values are respectively

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 45</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; k</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>$

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 56</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>$

X ₄₆ =X ₄₅ ·X ₅₆

X _456D = u _D X ₅₆ -w _D X ₄₆

Then the equation of the vanishing line corresponding to the projection of the road straight line in the image is

k·u-w+(w _D -k·u _D )=0, where k=-X ₄₅ is the slope of the vanishing line.

The straight line d' passes through the point O'(u _o' , w _o' ), and the obtained slope is k ₂ , its equation can be expressed as

k ₂ ·u-w+(w _o′ -k ₂ ·u _o′ )=0

Then the coordinates (u, w) of the intersection of the vanishing line can be calculated by the equation of the vanishing line and the straight line d';

4) Camera parameter calibration:

Assuming that the center of the camera lens is located at the spatial point (x _c , y _c , z _c ), and the camera’s deflection angle (Pan), pitch angle (Tilt) and rotation angle (Swing) are denoted as θ, , Ψ respectively, then

Calculate the rotation angle (Swing) Ψ. The formula is

Ψ=arctg(X ₄₅ )

Calculate the deflection angle (Pan) θ of the camera as

$<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; \&pi;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; arcsin</span>}<span\; class="notranslate">\; [</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\sqrt{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; m</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; \&pi;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; arctg</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; or</span>}$

$\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; \&pi;</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; arcsin</span>}<span\; class="notranslate">\; [</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\sqrt{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; m</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; \&pi;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; arctg</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>$

k ₁ , k ₂ =...-1, 0, 1, 2..., -π/2≤θ≤π/2}

in $x=x_{456\mathrm{D.}}/[(1+x_{45}^2)\&Center\; Dot;x_{56}\&Center\; Dot;(u-u_{\mathrm{D.}})],$

Calculate the pitch angle (Tilt) φ as: φ=arcsin(tgθ/X _456D )

Calculate the focal length of the camera as: $f=-\frac{w_{\mathrm{D.}}\&Center\; Dot;\cos \mathrm{\&psi;}+u_{\mathrm{D.}}\&Center\; Dot;\sin \mathrm{\&psi;}}{\mathrm{tg\&phi;}}$

Consider a point P(x, y, z) on the ground in the world coordinate system, assume its projection point on the image plane is P'=(u, w), and define

A=cosθcosψ+sinθsinφsinΨ, B=sinθcosΨ-cosθsinφsinψ, C=cosφsinΨ,

D=-sinθcosφ, E=cosθcosφ, F=sinφ,

G=sinθsinφcosΨ-cosθsinΨ, H=-cosθsinφcosψ-sinθsinΨ, I=cosφcosΨ, then the actual coordinates of the camera lens center can be given by the following equation (L.L.Wang, and W.H.Tsai, "Camera Calibration by Vanishing Lines for 3-D Computer Vision, "IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.13, no.4, pp.370-376, April 1991.):

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; uA</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; wxya</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; uC</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; uB</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; wxya</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; uC</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Among them, h is the installation height of the camera.

According to the expression about x _c in formula (1), the installation height of the camera is calculated as

h=(d ₁ +d ₂ )/(ab), where

a=( _uA'A +fD+ _wA'G )/( _uA'C +fF+ _wA'I ),

b=(u _B′ A+fD+w _B′ G)/(u _B′ C+fF+w _B′ I)

Since the coordinates of point O are (x _o = 0, y _o = 0, z _o = 0), the coordinates of the three-dimensional position of the camera lens center are calculated according to formula (1):

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{{\mathrm{<span\; class="notranslate">\; o</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}\mathrm{<span\; class="notranslate">\; A</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{{\mathrm{<span\; class="notranslate">\; o</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{{\mathrm{<span\; class="notranslate">\; o</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{{\mathrm{<span\; class="notranslate">\; o</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{{\mathrm{<span\; class="notranslate">\; o</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{{\mathrm{<span\; class="notranslate">\; o</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{{\mathrm{<span\; class="notranslate">\; o</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{{\mathrm{<span\; class="notranslate">\; o</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; h</span>}\end{array}\right.$

Calculate the value of A, B, C, D, E, F, G, H, I according to the definition of A to I and take the verification formula

$|u_{\mathrm{D.}}-\frac{B\&Center\; Dot;f}{\mathrm{E.}}|+|w_{\mathrm{D.}}-\frac{h\&Center\; Dot;f}{\mathrm{E.}}|<T$ (T is a threshold close to zero)

In order to test the obtained multiple sets of θ, φ, Ψ, f values, and finally select the camera orientation parameters for accurate ungrouping.

In order to verify the effectiveness of the method proposed in the present invention, an embodiment of the present invention uses the actual traffic scene image shown in FIG. 4 , and selects a calibration target in the actual traffic scene image, as shown in FIG. 5 . Select three adjacent lane lines whose common point is at V _D in the image as a set of parallel lines required for calibration, respectively marked as a', b', c', and take the parking line as the intersecting straight line d', which are in the world coordinate system The actual corresponding straight lines are denoted as a, b, c, d respectively, where the slope of the straight line d is m=0, and the distances between the straight lines a and b and the straight lines b and c are also known in advance.

The experimental results show that the calculation of the calibration parameters is closely related to the accuracy of the selection of the four straight lines, but after careful selection, a stable result with an average floating error of about 5% can be obtained, as listed in Table 1 for the focal length f and the camera installation height h The comparison between the calibration value and the actual value (the camera direction data cannot be measured, it is difficult to give the actual value for comparison, so it is omitted), this shows that this method can fully meet the accuracy requirements of the traffic monitoring system, and this experiment proves to a certain extent The effectiveness of the method proposed in the present invention has been confirmed.

Table 1 Experimental calibration results of camera focal length and installation height parameter Calculated actual value Offset (%) f(pixel) -665.43 -700.00 4.94 h (meter) 4.87 5.10 4.51

Claims (1)

1, a camera calibration method under traffic monitoring environment, it is characterized in that comprising the steps: 1) Selection of the calibration target and establishment of the relevant coordinate system: in the traffic scene, select a group of three parallel lines with known intervals on the road surface and a straight line with known slope and intersection with the three parallel lines as the calibration target, and determine the calibration target. Each projection of the target in the image corresponds to the straight line and the corresponding intersection point, establishes the world coordinate system, sets the origin of the world coordinate system at the intersection of a parallel line in the middle and the intersecting straight line, the positive Y axis points forward along the middle parallel line, and the positive Z axis is vertical In the upward direction of the ground, the X-axis is positive and horizontally pointing to the right, and a coordinate system centered on the camera is established. The coordinate system takes the center of the lens as the origin, the optical axis of the camera as the V-axis, and makes the plane of the U-W coordinate axis parallel to the image plane, and the distance from the center of the lens is the focal length of the camera; 2) Calibration input data initialization: With respect to the coordinate system centered on the camera, the least squares method is used to calculate the coordinates of all intersection points of the calibration targets in the image corresponding to the projections of the straight lines, and simultaneously calculate their slopes on the image plane; 3) Calculation of the vanishing line corresponding to the ground and the vanishing point corresponding to the intersecting straight line: According to the distance between two adjacent straight lines in the three parallel lines, the coordinates of the intersecting point between the intersecting straight line and the three parallel lines, and relative to the coordinate system centered on the camera The coordinate parameters of each intersection of the three parallel lines and the intersecting straight lines corresponding to the projected straight lines on the image plane, obtain the corresponding vanishing line slope and the general formula equation on the ground, and obtain the intersection coordinates of the projected straight lines and the vanishing lines of the intersecting straight lines on the image plane, That is, the coordinates of the vanishing point corresponding to the intersecting straight line on the image plane are obtained; 4) Calibration of camera parameters: the distance between two adjacent straight lines in the three parallel lines, the coordinates of the intersecting points between the intersecting straight lines and the three parallel lines, the coordinates of the intersection points of the three parallel lines and the intersecting straight lines projected in the image, the slope of the vanishing line, and the intersecting straight line The coordinates of the intersection point of the projected straight line and the vanishing line are obtained to obtain the camera's rotation angle, deflection angle, pitch angle and camera focal length, and finally the three-dimensional position coordinates of the camera lens center are obtained according to the transformation relationship of the two coordinate systems, and the camera calibration is completed.

Images