CN101118648A

CN101118648A - Road Condition Camera Calibration Method in Traffic Surveillance Environment

Info

Publication number: CN101118648A
Application number: CNA2007100228107A
Authority: CN
Inventors: 陈启美; 李勃; 郭凡; 董蓉
Original assignee: Nanjing University
Current assignee: Nanjing University
Priority date: 2007-05-22
Filing date: 2007-05-22
Publication date: 2008-02-06

Abstract

A road condition video camera demarcating method under the traffic monitoring condition includes the following demarcating steps: (1) visual model describing and relevant coordinate system creating: according to the requirement of the monitoring system performance, a classical Tsai transmission projection model is used for reference, and according to the feature of the road condition imaging, a new visual model is presented after conducting corresponding amendment on the classical Tsai transmission projection model, and three kinds of coordinates are established. (2) demarcating the main point of a video camera and the extension factor: the image monitoring light stream is used as the demarcating basic element; by the extension movement of the video camera, a reference frame forecast image and the difference value of the light stream field in a real time frame sampling image are used as the restriction, a constraint equation is established by adopting the least square method, the main point coordinate of the video camera and the actual magnification coefficient of the video camera can be distinguished by the powell direction family method. (3) Demarcated object selecting and parameter linearity evaluating. (4) Accuracy making on the internal and external parameters of the video camera: according to all the angle points of the monitored image and the corresponding world coordinate point, the Levenberg-Marguardt optimization algorithm is adopted to make accuracy on the video camera model parameter, and then the video camera demarcating is finished.

Description

Road Condition Camera Calibration Method in Traffic Surveillance Environment

技术领域technical field

本发明属于智能交通技术领域，特别涉及交通监视环境下的路况摄像机的标定方法，主要用于交通监视环境下云台摄像机的精确标定。The invention belongs to the technical field of intelligent transportation, in particular to a method for calibrating a road condition camera in a traffic monitoring environment, and is mainly used for the precise calibration of a pan-tilt camera in a traffic monitoring environment.

背景技术Background technique

应用电视视频技术、通信与网络技术、模式识别技术、计算机视觉技术等技术，发展现有交通视频监控技术，实现交通信息流的自动采集与分析，是智能交通技术领域中具有广阔应用前景的研究方向。其目标就是用计算机视觉技术通过分析路况监视图像序列来对车辆、行人等交通目标的运动进行检测定位、识别和跟踪，并对检测跟踪的交通运动目标的交通行为进行分析和判断，从而既完成各种交通流数据的采集又进行与交通管理有关的各种日常管理和控制，形成一个全方位立体化的数字交通监控网，真正实现交通管理智能化。Applying TV video technology, communication and network technology, pattern recognition technology, computer vision technology and other technologies to develop existing traffic video monitoring technology and realize automatic collection and analysis of traffic information flow is a research with broad application prospects in the field of intelligent transportation technology direction. Its goal is to use computer vision technology to detect, locate, identify and track the movement of vehicles, pedestrians and other traffic targets by analyzing the sequence of road monitoring images, and to analyze and judge the traffic behavior of the detected and tracked traffic targets, so as to complete The collection of various traffic flow data and various daily management and control related to traffic management form an all-round three-dimensional digital traffic monitoring network and truly realize the intelligentization of traffic management.

针对监视需求，多数路况摄像机为变焦摄像机且采用云台摄像技术。因此当路况摄像机的视域发生改变时，监控路段与成像平面间映射关系也会相应变化，为此需通过对当前路况图像分析，标定摄像机的当前状态，确定场景点与图像像素间确定对应关系，避免摄像机状态改变为车速、车型、运动距离等参数的计算带来影响。For monitoring needs, most traffic cameras are zoom cameras and use PTZ camera technology. Therefore, when the field of view of the road condition camera changes, the mapping relationship between the monitoring road section and the imaging plane will also change accordingly. To this end, it is necessary to analyze the current road condition image, calibrate the current state of the camera, and determine the corresponding relationship between the scene point and the image pixel. , to avoid the impact of camera state changes on the calculation of parameters such as vehicle speed, vehicle type, and movement distance.

摄像机标定是计算机视觉领域里从二维图像提取三维空间信息的重要步骤，需依据摄像机几何成像原理，对实际摄像装置进行抽象处理，提出表征物空间与成像平面间映射关系的视觉模型。进而，从此视觉模型入手，针对其成像特点，在像平面中拟定合适参照物，分析找出相关图像特征参数，标定视觉模型中各未知参数。Camera calibration is an important step in the field of computer vision to extract three-dimensional spatial information from two-dimensional images. It is necessary to abstract the actual camera device based on the geometric imaging principle of the camera, and propose a visual model that characterizes the mapping relationship between the object space and the imaging plane. Furthermore, starting from this visual model, according to its imaging characteristics, draw up a suitable reference object in the image plane, analyze and find out the relevant image characteristic parameters, and calibrate the unknown parameters in the visual model.

路况摄像机组的标定作为相机标定的重要工程应用，多采用传统标定方法和相机自标定方法。传统标定方法是指用一个结构已知的标定块作为空间参照物，通过建立标定块上一组非退化(非共面)的空间点及其图像投影点间的对应关系来建立摄像机模型参数间的约束，然后通过优化算法来求取这些参数；自标定方法是指直接分析利用获得的多幅图像信息，提取表征摄像机内参数自身存在约束的匹配点，建立基于绝对二次曲线(曲面)的虚拟标定块，从而标定摄像机参数。As an important engineering application of camera calibration, the calibration of road condition cameras mostly adopts traditional calibration methods and camera self-calibration methods. The traditional calibration method refers to using a calibration block with a known structure as a spatial reference object, and establishing the correspondence between a group of non-degenerate (non-coplanar) spatial points on the calibration block and their image projection points to establish the relationship between camera model parameters. constraints, and then obtain these parameters through an optimization algorithm; the self-calibration method refers to directly analyzing and utilizing multiple image information obtained, extracting the matching points that represent the constraints of the internal parameters of the camera itself, and establishing an image based on the absolute quadratic curve (surface) Virtual calibration block to calibrate camera parameters.

传统标定方法标定精度高，但需于现场设置标定物，标定点设置困难，过程繁琐，图像特征量需求大，仅适用于小范围场景，不满足路况大场景监视要求。自标定算法优势在于充分利用监视场景图像信息，但其在标定过程中，需控制监视摄像机做数次刚体运动，并对不同视点下的图像进行特征参数精确匹配，如是，算法依赖于要求极高的图像识别及特征匹配技术，鲁棒性差。The traditional calibration method has high calibration accuracy, but it needs to set up calibration objects on site. It is difficult to set up calibration points, the process is cumbersome, and the demand for image features is large. It is only suitable for small-scale scenes and does not meet the monitoring requirements of large-scale road conditions. The advantage of the self-calibration algorithm is to make full use of the image information of the surveillance scene, but in the calibration process, it is necessary to control the surveillance camera to perform several rigid body movements, and to accurately match the feature parameters of the images under different viewpoints. If so, the algorithm depends on extremely high requirements. Advanced image recognition and feature matching technology, poor robustness.

为简化标定流程，针对交通监视景物具体特点，Nelson，Grantham和George等人引入了基于简单成像模型的交通视频监控系统摄像机标定方法，其直接利用交通场景中的路面分道线角点组成的矩形标定目标，标定摄像机焦距和方位参数。与之类似，发明CN1564581A公布了一种城市交通监视环境下标定摄像机焦距及空间外部参数的自标定方法，其利用交通场景中监控路面上数条特殊直线作为标定目标，完成摄像机标定。此类方法实现简单，具有线性时间计算复杂度，然而其在建立三维映射模型时，需要固化摄像机模型部分内部参数(主点坐标和畸变参数)，降低了标定算法的适用范围，难以满足摄像机高精度标定要求。In order to simplify the calibration process, Nelson, Grantham, and George et al. introduced a camera calibration method for traffic video surveillance systems based on a simple imaging model for the specific characteristics of traffic monitoring scenes, which directly uses the rectangle formed by the corner points of road lanes in the traffic scene. Calibrate the target, calibrate the camera focal length and orientation parameters. Similarly, the invention CN1564581A discloses a self-calibration method for calibrating camera focal length and spatial external parameters in an urban traffic monitoring environment, which uses several special straight lines on the monitoring road surface in traffic scenes as calibration targets to complete camera calibration. This type of method is simple to implement and has linear time computational complexity. However, when establishing a 3D mapping model, it needs to solidify some internal parameters of the camera model (principal point coordinates and distortion parameters), which reduces the scope of application of the calibration algorithm and is difficult to meet the requirements of high-resolution cameras. Accuracy calibration requirements.

发明内容Contents of the invention

本发明的目的在于针对现有摄像机标定技术的不足，提供应用于交通监视环境下的一种新的摄像机多级标定方法。The purpose of the present invention is to provide a new camera multi-level calibration method applied in the traffic monitoring environment to address the shortcomings of the existing camera calibration technology.

针对监视需求，建立高精度的摄像机视觉成像模型，确定物空间与像平面间对应关系；根据监视场景下，不同标称放大系数下场景图像光流特性，建立基于摄像机主点的优化模型，以标定摄像机主点及实际放大系数；选择场景图像上的相邻分道线上角点作为标定目标，利用分道线平行关系及分道线间的基准路宽作为约束条件，建立包含有效焦距及摄像机空间位置参数在内的约束方程，求解视觉模型内、外部参数。According to the monitoring requirements, a high-precision camera visual imaging model is established to determine the corresponding relationship between the object space and the image plane; according to the optical flow characteristics of the scene image under different nominal magnification factors in the monitoring scene, an optimization model based on the principal point of the camera is established to Calibrate the main point of the camera and the actual magnification factor; select the corner points on the adjacent lanes on the scene image as the calibration target, use the parallel relationship between the lanes and the reference road width between the lanes as constraints, and establish an effective focal length and Constraint equations including camera space position parameters are used to solve the internal and external parameters of the visual model.

算法采用复杂的摄像机模型，充分考虑了相机主点变化及镜头径向畸变带来的影响，满足高精度摄像机标定的要求；采用了多级标定方法，分解摄像机内部参数为固定参数、可变参数，简化标定流程；标定过程中，无需复杂的特征匹配，提高算法鲁棒性。The algorithm adopts a complex camera model, which fully considers the influence of the change of the principal point of the camera and the radial distortion of the lens, and meets the requirements of high-precision camera calibration; adopts a multi-level calibration method, and decomposes the internal parameters of the camera into fixed parameters and variable parameters , to simplify the calibration process; during the calibration process, no complex feature matching is required, which improves the robustness of the algorithm.

本发明以下述方法实现：交通监视环境下的路况摄像机标定方法，包含如下标定步骤：The present invention is realized with following method: the road condition camera calibration method under the traffic surveillance environment, comprises following calibration steps:

(1)视觉模型描述和相关坐标系建立：针对监控系统性能要求，沿用经典的Tsai透射投影模型，并针对路况成像特点，对其进行相应修正，提出新的视觉模型，建立三种坐标系：(1) Visual model description and related coordinate system establishment: In view of the performance requirements of the monitoring system, the classic Tsai transmission projection model is used, and according to the characteristics of road condition imaging, it is corrected accordingly, a new visual model is proposed, and three coordinate systems are established:

其中地面坐标系X_w-Y_w-Z_w和摄像机坐标系X_c-Y_c-Z_c用来表征三维空间；图像平面坐标系X_f-Y_f用来表征成像平面。建立世界坐标系，其原点为摄像机光轴与地面交点。Y_w轴正向沿路面方向指向前方，X_w轴正向水平指向右方，Z_w轴正向垂直于地面，方向向上。建立摄像机坐标系，原点为摄像机光心位置，Z_c轴为摄像机光轴方向，X_c-Y_c平面平行于像平面。The ground coordinate system X _w -Y _w -Z _w and the camera coordinate system X _c -Y _c -Z _c are used to represent the three-dimensional space; the image plane coordinate system X _f -Y _f is used to represent the imaging plane. Establish a world coordinate system whose origin is the intersection of the camera optical axis and the ground. The positive direction of the Y _w axis is pointing forward along the road surface, the positive direction of the X _w axis is pointing to the right, and the positive direction of the Z _w axis is perpendicular to the ground, and the direction is upward. The camera coordinate system is established, the origin is the position of the optical center of the camera, the Z _c axis is the direction of the camera optical axis, and the X _c -Y _c plane is parallel to the image plane.

X_c＝(cos(p)cos(s)+sin(t)sin(p)sin(s))X_w X _c = (cos(p)cos(s)+sin(t)sin(p)sin(s)) _Xw

+(sin(p)cos(s)-sin(t)cos(p)sin(s))Y_w +(sin(p)cos(s)-sin(t)cos(p)sin(s))Y _w

Y_c＝(-cos(p)sin(s)+sin(t)sin(p)cos(s))X_w(1)Y _c ＝(-cos(p)sin(s)+sin(t)sin(p)cos(s)) _Xw (1)

+(-sin(p)sin(s)+sin(t)cos(p)cos(s))Y_w +(-sin(p)sin(s)+sin(t)cos(p)cos(s))Y _w

Z_c＝-cos(t)sin(p)X_w+cos(t)cos(p)Y_w+lZ _c ＝-cos(t)sin(p) _Xw +cos(t)cos(p) _Yw +l

$\{\begin{matrix} \frac{11}{M m ((z z))} [[{X x}_{f f} - - {C C}_{x x} ((Z Z))]] ((11 + + k k {r r}^{22})) = = f f \frac{{X x}_{C C}}{{Z Z}_{C C}} \\ \frac{11}{M m ((z z))} [[{Y Y}_{f f} - - {C C}_{y the y} ((Z Z))]] ((11 + + k k {r r}^{22})) = = - - f f \frac{{Y Y}_{C C}}{{Z Z}_{C C}} \end{matrix} - - - - - - ((22))$

建立此视觉模型下，世界坐标系与像平面坐标系下映射关系，如式(3)所示Establish the mapping relationship between the world coordinate system and the image plane coordinate system under this visual model, as shown in formula (3)

${X x}_{f f} = = \frac{f f [\begin{matrix} ((cos cos ((p p)) cos cos ((s the s)) + + sin sin ((t t)) sin sin ((p p)) sin sin ((s the s)))) {X x}_{w w} \\ + + ((sin sin ((p p)) cos cos ((s the s)) - - sin sin ((t t)) cos cos ((p p)) sin sin ((s the s)))) {Y Y}_{w w} \end{matrix}]}{- - cos cos ((t t)) sin sin ((p p)) {X x}_{w w} + + cos cos ((t t)) cos cos ((p p)) {Y Y}_{w w} + + l l}$

${Y Y}_{f f} = = \frac{f f [\begin{matrix} ((- - cos cos ((p p)) sin sin ((s the s)) + + sin sin ((t t)) sin sin ((p p)) cos cos ((s the s)))) {X x}_{w w} \\ + + ((- - sin sin ((p p)) sin sin ((s the s)) + + sin sin ((t t)) cos cos ((p p)) cos cos ((s the s)))) {Y Y}_{w w} \end{matrix}]}{- - cos cos ((t t)) sin sin ((p p)) {X x}_{w w} + + cos cos ((t t)) cos cos ((p p)) {Y Y}_{w w} + + l l} - - - - - - ((33))$

由式(3)出发，以Z_w为已知参数，同样可建立由像平面坐标到世界坐标系下的逆映射关系，如式(4)所示：Starting from formula (3), taking Z _w as a known parameter, the inverse mapping relationship from the image plane coordinates to the world coordinate system can also be established, as shown in formula (4):

${X x}_{w w} = = \frac{[\begin{matrix} sin sin p p ((l l + + {Z Z}_{w w} sin sin t t)) (({x x}_{f f} sin sin s the s + + {y the y}_{f f} cos cos s the s)) \\ + + cos cos p p ((l l sin sin t t + + {Z Z}_{w w})) (({x x}_{f f} cos cos s the s - - {y the y}_{f f} sin sin s the s)) - - {Z Z}_{w w} f f cos cos t t sin sin p p \end{matrix}]}{{x x}_{f f} cos cos t t sin sin s the s + + {y the y}_{f f} cos cos t t cos cos s the s + + f f sin sin t t}$

${Y Y}_{w w} = = \frac{[\begin{matrix} - - cos cos p p ((l l + + {Z Z}_{w w} sin sin t t)) (({x x}_{f f} sin sin s the s + + {y the y}_{f f} cos cos s the s)) \\ + + sin sin p p ((l l sin sin t t + + {Z Z}_{w w})) (({x x}_{f f} cos cos s the s - - {y the y}_{f f} sin sin s the s)) - - {Z Z}_{w w} f f cos cos t t sin sin p p \end{matrix}]}{{x x}_{f f} cos cos t t sin sin s the s + + {y the y}_{f f} cos cos t t cos cos s the s + + f f sin sin t t} - - - - - - ((44))$

式中H为摄像机的垂直安置高度，t，p，s分别为摄像机俯仰角、偏角和旋角，f为广角有效焦距，M(z)为放缩系数，Cx(z)。Cy(z)为图像主点坐标。k为一阶径向畸变。In the formula, H is the vertical installation height of the camera, t, p, s are the pitch angle, deflection angle and rotation angle of the camera respectively, f is the wide-angle effective focal length, M(z) is the scaling factor, and Cx(z). Cy(z) is the coordinates of the principal point of the image. k is the first-order radial distortion.

(2)标定摄像机主点及放缩系数：以监视图像光流作为标定基元。通过摄像机作放缩运动，应用参考帧预测图像及实时帧采样图像间光流场差值作为约束，使用最小二乘方法建立约束方程，通过powell方向族法辨识摄像机主点坐标及摄像机实际放大系数。(2) Calibrate the principal point of the camera and the zoom factor: the optical flow of the monitored image is used as the calibration primitive. Through the zooming movement of the camera, the optical flow field difference between the reference frame prediction image and the real-time frame sampling image is used as a constraint, the least square method is used to establish the constraint equation, and the camera principal point coordinates and the actual camera magnification factor are identified by the Powell direction family method .

(3)标定目标选择与参数线性求解：在监视场景中选择分道线角点作为标定参照物，利用相邻四个分道线角点的像平面投影确定消失线斜率及相应摄像机旋角；以分道线平行关系及分道线间基础路宽作为约束，利用此四个角点的像平面坐标，线性求解摄像机有效焦距及其空间位置参数。(3) Calibration target selection and parameter linear solution: In the surveillance scene, select the corner points of the lane-dividing line as the calibration reference object, and use the image plane projection of the four adjacent corner points of the lane-dividing line to determine the slope of the vanishing line and the corresponding camera rotation angle; Constraints on the parallel relationship between lane-dividing lines and the basic road width between lane-dividing lines, using the image plane coordinates of the four corner points, linearly solve the effective focal length of the camera and its spatial position parameters.

(4)摄像机畸变补偿：利用已定标出完备的理想的无透视畸变摄像机成像模型，计算图像上所有角点的理想象素坐标，通过其与真实角点像平面投影的坐标差值，求解非线性视觉模型中的一阶径向畸变。(4) Camera distortion compensation: use the calibrated and complete ideal camera imaging model without perspective distortion to calculate the ideal pixel coordinates of all corner points on the image, and solve the problem through the coordinate difference between them and the real corner point image plane projection. First-order radial distortion in nonlinear vision models.

(5)求精摄像机内外部参数：以监视图像中所有角点和对应的世界坐标点，以上述算法所求出的视觉模型参数作为优化模型初值，采用Levenberg-Marquardt优化算法求精摄像机模型参数，完成摄像机标定。(5) Refine the internal and external parameters of the camera: take all the corner points in the monitoring image and the corresponding world coordinate points, use the visual model parameters obtained by the above algorithm as the initial value of the optimization model, and use the Levenberg-Marquardt optimization algorithm to refine the camera model parameters to complete the camera calibration.

本发明特点是：利用本发明完成摄像机标定的方法实现简单，能满足摄像机高精度标定要求。适用于交通监视环境下的摄像机多级标定。The present invention is characterized in that: the method for completing camera calibration by using the present invention is simple to implement and can meet the high-precision calibration requirements of the camera. It is suitable for multi-level calibration of cameras in traffic surveillance environment.

附图说明Description of drawings

图1为本发明所建立的摄像机成像视觉模型Fig. 1 is the camera imaging visual model that the present invention establishes

图2为本发明所选择的角点标定目标鸟瞰示意图Fig. 2 is a bird's-eye view schematic diagram of the corner point calibration target selected by the present invention

图3为角点标定目标所决定灭点及地平线的示意图Figure 3 is a schematic diagram of the vanishing point and the horizon determined by the corner point calibration target

图4为本发明实施例所采用的交通场景原始图像Fig. 4 is the original image of the traffic scene used in the embodiment of the present invention

图5为本发明实施例中选择的角点标定目标示意图Fig. 5 is a schematic diagram of the corner point calibration target selected in the embodiment of the present invention

图6为本发明实施例中检测的摄像机可变参数分布示意图Fig. 6 is a schematic diagram of the variable parameter distribution of the camera detected in the embodiment of the present invention

其中图6a是主点座标分布，图6b是是横向座标分布Among them, Figure 6a is the principal point coordinate distribution, and Figure 6b is the horizontal coordinate distribution

具体实施方式Detailed ways

(1)视觉模型描述和相关坐标系建立：针对监控系统性能要求，沿用经典的Tsai透射投影模型，并针对路况成像特点，对其进行相应修正，提出新的视觉模型，如图1所示。(1) Visual model description and related coordinate system establishment: Aiming at the performance requirements of the monitoring system, the classic Tsai transmission projection model is used, and a new visual model is proposed according to the characteristics of road imaging, as shown in Figure 1.

图中定义了三种坐标系，其中地面坐标系X_w-Y_w-Z_w和摄像机坐标系X_c-Y_c-Z_c用来表征三维空间；图像平面坐标系X_f-Y_f用来表征成像平面。建立世界坐标系，其原点为摄像机光轴与地面交点。Y_w轴正向沿路面方向指向前方，X_w轴正向水平指向右方，Z_w轴正向垂直于地面，方向向上。建立摄像机坐标系，原点为摄像机光心位置，Z_c轴为摄像机光轴方向，X_c-Y_c平面平行于像平面。Three coordinate systems are defined in the figure, among which the ground coordinate system X _w -Y _w -Z _w and the camera coordinate system X _c -Y _c -Z _c are used to represent the three-dimensional space; the image plane coordinate system X _f -Y _f is used to Characterize the imaging plane. Establish a world coordinate system whose origin is the intersection of the camera optical axis and the ground. The positive direction of the Y _w axis is pointing forward along the road surface, the positive direction of the X _w axis is pointing to the right, and the positive direction of the Z _w axis is perpendicular to the ground, and the direction is upward. The camera coordinate system is established, the origin is the position of the optical center of the camera, the Z _c axis is the direction of the camera optical axis, and the X _c -Y _c plane is parallel to the image plane.

设摄像机光心与世界坐标系原点距离为1，摄像机的俯仰角(摄像机光轴与地平面夹角)为t，偏角(光轴与车道分割线的夹角)为p。旋角为s，忽略高速公路坡度影响，以地平面上平行线间的区域来对应摄像机视域内的高速公路路面。Let the distance between the optical center of the camera and the origin of the world coordinate system be 1, the pitch angle of the camera (the angle between the optical axis of the camera and the ground plane) be t, and the deflection angle (the angle between the optical axis and the lane dividing line) be p. The rotation angle is s, and the influence of the slope of the expressway is ignored, and the area between parallel lines on the ground plane corresponds to the expressway pavement in the field of view of the camera.

基于定义的摄像机空间方位参数，可建立地面坐标系与摄像机坐标系间的坐标变换关系，如上述式(1)所示。Based on the defined camera space orientation parameters, the coordinate transformation relationship between the ground coordinate system and the camera coordinate system can be established, as shown in the above formula (1).

根据透视变换原理，可建立二维图像坐标系与摄像机坐标系间的坐标映射关系，如上述式(2)所示。According to the principle of perspective transformation, the coordinate mapping relationship between the two-dimensional image coordinate system and the camera coordinate system can be established, as shown in the above formula (2).

式中f为广角有效焦距，M(z)为放缩系数，Cx(z)。Cy(z)为图像主点坐标。k为一阶径向畸变。Where f is the wide-angle effective focal length, M(z) is the scaling factor, and Cx(z). Cy(z) is the coordinates of the principal point of the image. k is the first-order radial distortion.

不失一般性，记X_f＝X_f-C_x(z)，Y_f＝Y_f-C_y(z)，f＝f*M(z)，联立式1、式2，可建立理想透视模型下，世界坐标系与像平面坐标系下映射关系，如上述式(3)所示。Without loss of generality, record X _f ＝X _f -C _x (z), Y _f ＝Y _f -C _y (z), f＝f*M(z), simultaneous formula 1 and formula 2, can establish the ideal Under the perspective model, the mapping relationship between the world coordinate system and the image plane coordinate system is shown in the above formula (3).

由式(3)出发，以Z_w为已知参数，同样可建立由像平面坐标到世界坐标系下的逆映射关系，如上述式(4)所示：Starting from formula (3), taking Z _w as a known parameter, the inverse mapping relationship from the image plane coordinates to the world coordinate system can also be established, as shown in the above formula (4):

(2)摄像机主点确定：高速公路监控场景作为复杂场景，难以提供监视图像间标定目标的精确匹配，需以光流作为标定基元。通过相机变焦或纯旋转运动，根据参考帧预测图像及实时帧采样图像光流场，使用最小二乘方法辨识摄像机内部参数(2) Determination of the main point of the camera: As a complex scene, the highway monitoring scene is difficult to provide accurate matching of calibration targets between surveillance images, and optical flow is required as the calibration primitive. Through camera zooming or pure rotation motion, predict the image according to the reference frame and sample the optical flow field of the real-time frame image, and use the least square method to identify the internal parameters of the camera

设I_r[x₁]参考帧灰度图像，I_f[x₂]为改变摄像机部分参数p所获得的采样灰度图像。因图像放缩与纯旋转并未引入新的场景，故可引入转换G实现象素坐标x₁，x₂间映射，如式(5)所示Let I _r [x ₁ ] refer to the grayscale image of the frame, and _If [x ₂ ] is the sampled grayscale image obtained by changing the parameter p of the camera. Since image scaling and pure rotation do not introduce new scenes, the transformation G can be introduced to realize the mapping between pixel coordinates x ₁ and x ₂ , as shown in formula (5)

$\{\begin{matrix} {x x}_{22} = = G G (({x x}_{11},, p p)) \\ {x x}_{11} = = H h (({x x}_{22},, p p)) \end{matrix} - - - - - - ((55))$

考虑在同一场景下，固定摄像机其余参数，以不同标称放大系数拍摄两幅场景图像，不失一般性，不妨设参考帧为广角图像I₀(标称放大系数＝0)，采样帧为I_n(标称放大系数n)。由式(2)可得参考帧与采样帧间坐标映射公式H如式(6)所示。Considering that in the same scene, the remaining parameters of the camera are fixed, and two scene images are taken with different nominal magnification factors, without loss of generality, it is advisable to set the reference frame as a wide-angle image I ₀ (nominal magnification factor=0), and the sampling frame as I _n (nominal amplification factor n). From formula (2), the coordinate mapping formula H between the reference frame and the sampling frame can be obtained as shown in formula (6).

X′_f＝M(n)[X_f-C_x(n)]+C_x(n)X' _f ＝M(n)[X _f -C _x (n)]+C _x (n)

(6)(6)

Y′_f＝M(n)[Y_f-C_y(n)]+C_y(n)Y′ _f ＝M(n)[Y _f -C _y (n)]+C _y (n)

据此坐标对应关系，即可写出基于参考帧灰度值的期望图像I_w[x]According to this coordinate correspondence, the expected image I _w [x] based on the gray value of the reference frame can be written

I_w[x]＝I[H(x，C_x(n)，C_y(n)，M(n))] _Iw [x]=I[H(x, _Cx (n), _Cy (n), M(n))]

故针对采样图像，可得出基于期望图像与采样图像间的标准方差函数，如式(7)所示Therefore, for the sampled image, the standard variance function based on the expected image and the sampled image can be obtained, as shown in formula (7)

$E E. ((p p)) = = \underset{x x &Element; &Element; V V}{Σ Σ} {(({I I}_{f f} [[x x]] - - {I I}_{w w} [[x x]]))}^{22} / / \underset{x x &Element; &Element; V V}{Σ Σ} 11$

$= = \underset{x x &Element; &Element; V V}{Σ Σ} {(({I I}_{f f} [[x x]] - - {I I}_{r r} [[H h ((x x,, {C C}_{x x} ((n no)),, {C C}_{y the y} ((n no)),, M m ((n no))))]]))}^{22} / / \underset{x x &Element; &Element; V V}{Σ Σ} 11 - - - - - - ((77))$

式中V为采样帧图像坐标子集，防止坐标转换H(x，p)中可能出现的坐标溢出。In the formula, V is a subset of the image coordinates of the sampling frame to prevent coordinate overflow that may occur in the coordinate transformation H(x, p).

使式(2)中E(p)最小是摄像机参数优化问题，优化模型的初值可设为M(n)＝M(n-1)，C_x(n)＝C_x(n-1)，C_y(n)＝C_y(n-1)，基值为M(0)＝1，(C_x(0)，C_y(0))为图像中心坐标。由于优化模型的采样点较多，为保证收敛，算法使用powell方向族法实现。To minimize E(p) in formula (2) is a camera parameter optimization problem, the initial value of the optimization model can be set as M(n)=M(n-1), C _x (n)=C _x (n-1) , C _y (n)=C _y (n-1), the base value is M(0)=1, (C _x (0), C _y (0)) is the image center coordinates. Since there are many sampling points in the optimization model, in order to ensure convergence, the algorithm is implemented using the Powell direction family method.

(3)标定目标选择与参数线性求解：摄像机在视角和焦距未知的条件下，若进行参数标定，必须拟定参照物，而高速公路分道线是严格划出的，可据此作为参照物，建立摄像机未标定参数与图像特征参数之间的对应关系。图2所示为监控路段上选取的基于分道线角点的平行四边形标定模块。(3) Calibration target selection and parameter linear solution: Under the condition of unknown viewing angle and focal length of the camera, if parameter calibration is performed, a reference object must be drawn up, and the expressway lane line is strictly drawn, which can be used as a reference object. Establish the corresponding relationship between uncalibrated camera parameters and image feature parameters. Figure 2 shows the parallelogram calibration module selected on the monitored road section based on the corner points of the lane divider.

地平线及灭点计算Calculation of horizon and vanishing point

根据透视投影原理，地面上互不重合的多条平行直线在像平面上的投影具有相同的灭点和不同的斜率。由四角点决定的两组平行直线决定的灭点及地平线如图(3)所示。According to the principle of perspective projection, the projections of multiple parallel straight lines on the ground that do not overlap with each other on the image plane have the same vanishing point and different slopes. The vanishing point and the horizon determined by two sets of parallel straight lines determined by the four corner points are shown in Figure (3).

由直线x_ax_d，x_bx_c决定的灭点记为x₀(u₀，v₀)，由直线x_ax_b，x_dx_c决定的灭点记为x₁(u₁，v₁)，其坐标如式(8)所示：The vanishing point determined by the straight line x _a x _d , x _b x _c is recorded as x ₀ (u ₀ , v ₀ ), and the vanishing point determined by the straight line x _a x _b , x _d x _c is recorded as x ₁ (u ₁ , v ₁ ), its coordinates are shown in formula (8):

${u u}_{00} = = \frac{(({u u}_{C C} {u u}_{D D.} - - {u u}_{B B} {u u}_{D D.})) {v v}_{A A} + + (({u u}_{A A} {u u}_{C C} - - {u u}_{C C} {u u}_{D D.})) {v v}_{B B} + + (({u u}_{B B} {u u}_{D D.} - - {u u}_{A A} {u u}_{B B})) {v v}_{C C} + + + + (({u u}_{A A} {u u}_{B B} - - {u u}_{A A} {u u}_{C C})) {v v}_{D D.}}{((- - {u u}_{B B} + + {u u}_{C C})) {v v}_{A A} + + (({u u}_{A A} - - {u u}_{D D.})) {v v}_{B B} + + ((- - {u u}_{A A} + + {u u}_{D D.})) {v v}_{C C} + + (({u u}_{B B} - - {u u}_{C C})) {v v}_{D D.}}$

${v v}_{00} = = \frac{(({v v}_{B B} {v v}_{D D.} - - {v v}_{C C} {v v}_{D D.})) {u u}_{A A} + + (({v v}_{C C} {v v}_{D D.} - - {v v}_{A A} {v v}_{C C})) {u u}_{B B} + + (({v v}_{A A} {v v}_{B B} - - {v v}_{B B} {v v}_{D D.})) {u u}_{C C} + + (({v v}_{A A} {v v}_{C C} - - {v v}_{A A} {v v}_{B B})) {u u}_{D D.}}{((- - {u u}_{B B} + + {u u}_{C C})) {v v}_{A A} + + (({u u}_{A A} - - {u u}_{D D.})) {v v}_{B B} + + ((- - {u u}_{A A} + + {u u}_{D D.})) {v v}_{C C} + + (({u u}_{B B} - - {u u}_{C C})) {v v}_{D D.}} - - - - - - ((88))$

${u u}_{11} = = \frac{(({u u}_{B B} {u u}_{C C} - - {u u}_{B B} {u u}_{D D.})) {v v}_{A A} + + (({u u}_{A A} {u u}_{D D.} - - {u u}_{A A} {u u}_{C C})) {v v}_{B B} + + (({u u}_{B B} {u u}_{D D.} - - {u u}_{A A} {u u}_{D D.})) {v v}_{C C} + + (({u u}_{A A} {u u}_{C C} - - {u u}_{B B} {u u}_{C C})) {v v}_{D D.}}{(({u u}_{C C} - - {u u}_{D D.})) {v v}_{A A} + + ((- - {u u}_{C C} + + {u u}_{D D.})) {v v}_{B B} + + ((- - {u u}_{A A} + + {u u}_{B B})) {v v}_{C C} + + (({u u}_{A A} - - {u u}_{B B})) {v v}_{D D.}}$

${v v}_{11} = = \frac{(({v v}_{B B} {v v}_{D D.} - - {v v}_{B B} {v v}_{C C})) {u u}_{A A} + + (({v v}_{A A} {v v}_{C C} - - {v v}_{A A} {v v}_{D D.})) {u u}_{B B} + + (({v v}_{A A} {v v}_{D D.} - - {v v}_{B B} {v v}_{D D.})) {u u}_{C C} + + (({v v}_{B B} {v v}_{C C} - - {v v}_{A A} {v v}_{C C})) {u u}_{D D.}}{(({u u}_{C C} - - {u u}_{D D.})) {v v}_{A A} + + (({- - u u}_{C C} + + {u u}_{D D.})) {v v}_{B B} + + ((- - {u u}_{A A} + + {u u}_{B B})) {v v}_{C C} + + (({u u}_{A A} - - {u u}_{B B})) {v v}_{D D.}}$

故地平线所在斜率即旋角的正切值为Therefore, the slope of the horizon, that is, the tangent of the rotation angle is

$tan the tan s the s = = - - \frac{{v v}_{00} - - {v v}_{11}}{{u u}_{00} - - {u u}_{11}} - - - - - - ((99))$

依所求出的旋转角，记According to the obtained rotation angle, record

$\overset{^^}{u u} = = u u cos cos s the s - - v v sin sin s the s$

$\overset{^^}{v v} = = u u sin sin s the s + + v v cos cos s the s$

可得像平面与地平面坐标映射方程(Z_w＝0)The coordinate mapping equation between image plane and ground plane can be obtained (Z _w ＝0)

${X x}_{w w} = = \frac{l l [[- - \overset{^^}{v v} sin sin ((p p)) + + \overset{^^}{u u} cos cos ((p p)) sin sin ((t t))]]}{(({\overset{^^}{v v}}_{00} - - \overset{^^}{v v})) cos cos ((t t))}$

${Y Y}_{w w} = = \frac{l l [[\overset{^^}{v v} cos cos ((p p)) + + \overset{^^}{u u} sin sin ((p p)) sin sin ((t t))]]}{(({\overset{^^}{v v}}_{00} - - \overset{^^}{v v})) cos cos ((t t))}$

根据角点间平行对应关系，有方程：According to the parallel correspondence between the corner points, there is an equation:

Y_A-Y_B＝Y_C-Y_D Y _A -Y _B = Y _C -Y _D

(10)(10)

X_A-X_C＝X_B-X_D X _A -X _C ＝X _B -X _D

由高速公路相对规范性，路宽一般为定值，故有方程Due to the relative standardization of expressways, the road width is generally a fixed value, so the equation

X_D-X_C＝w (11)X _D -X _C = w (11)

为方便记，已u、v表示

联立式(10)、(11)即可解出未知摄像机参数p，f，s，f，l，如式(12)所示For the convenience of remembering, it has been expressed by u and v

Simultaneous equations (10) and (11) can solve the unknown camera parameters p, f, s, f, l, as shown in equation (12)

$t t = = - - arcsin arcsin {{\frac{{v v}_{00}^{22} {(({v v}_{A A} - - {v v}_{B B} + + {v v}_{C C} - - {v v}_{D D.}))}^{22}}{\begin{matrix} [[(({v v}_{00} - - {v v}_{D D.})) {u u}_{A A} - - (({v v}_{00} - - {v v}_{C C})) {u u}_{B B} + + (({v v}_{00} - - {v v}_{B B})) {u u}_{C C} - - (({v v}_{00} - - {v v}_{A A})) {u u}_{D D.}]] \\ x x [[- - (({v v}_{00} - - {v v}_{B B})) {u u}_{A A} + + (({v v}_{00} - - {v v}_{A A})) {u u}_{B B} - - (({v v}_{00} - - {v v}_{D D.})) {u u}_{C C} + + (({v v}_{00} - - {v v}_{C C})) {u u}_{D D.}]] \end{matrix}}}}^{11 / / 22}$

f＝v₀/tan(t)f=v ₀ /tan(t)

$p p = = arctan arctan {{\frac{[[(({v v}_{00} - - {v v}_{D D.})) {u u}_{A A} - - (({v v}_{00} - - {v v}_{C C})) {u u}_{B B} + + (({v v}_{00} - - {v v}_{B B})) {u u}_{C C} - - (({v v}_{00} - - {v v}_{A A})) {u u}_{D D.}]] sin sin ((t t))}{{v v}_{00} (({v v}_{A A} - - {v v}_{B B} + + {v v}_{C C} - - {v v}_{D D.}))}}} - - - - - - ((1212))$

$l l = = \frac{w w ((f f sin sin t t + + {x x}_{A A} cos cos t t sin sin s the s + + {y the y}_{A A} cos cos t t cos cos s the s)) ((f f sin sin t t + + {x x}_{C C} cos cos t t sin sin s the s + + {y the y}_{C C} cos cos t t cos cos s the s))}{[\begin{matrix} - - ((f f sin sin t t + + {x x}_{A A} cos cos t t sin sin s the s + + {y the y}_{A A} cos cos t t cos cos s the s)) \\ (({x x}_{C C} cos cos p p sin sin s the s - - {x x}_{C C} sin sin p p sin sin t t cos cos s the s + + {y the y}_{C C} cos cos p p cos cos s the s + + {y the y}_{C C} sin sin p p sin sin t t sin sin s the s)) \\ + + ((f f sin sin t t + + {x x}_{C C} cos cos t t sin sin s the s + + {y the y}_{C C} cos cos t t cos cos s the s)) \\ (({x x}_{A A} cos cos p p sin sin s the s - - {x x}_{A A} sin sin p p sin sin t t cos cos s the s + + {y the y}_{A A} cos cos p p cos cos s the s + + {y the y}_{A A} sin sin p p sin sin t t sin sin s the s)) \end{matrix}]}$

(4)摄像机畸变补偿：考虑到非线性摄像机模型中存在一阶径向畸变，改写式(2)如下：(4) Camera distortion compensation: Considering the first-order radial distortion in the nonlinear camera model, formula (2) can be rewritten as follows:

X_d＝X_f+(X_f-C_x(z))*kr² X _d =X _f +(X _f -C _x (z))*kr ²

(13)(13)

Y_d＝Y_f+(Y_f-C_y(z))*kr² Y _d =Y _f +(Y _f -C _y (z))*kr ²

由于已定标出完备的理想的无透视畸变摄像机成像模型，故理想象素坐标(X_f，Y_f)可得，故可改写式(13)为方程形式，如式(12)所示：Since the complete ideal camera imaging model without perspective distortion has been calibrated, the ideal pixel coordinates (X _f , Y _f ) are available, so Equation (13) can be rewritten into an equation form, as shown in Equation (12):

$[\begin{matrix} (({X x}_{f f} - - {C C}_{x x} ((z z)))) {r r}^{22} \\ (({Y Y}_{f f} - - {C C}_{y the y} ((z z)))) {r r}^{22} \end{matrix}] k k = = [\begin{matrix} {X x}_{d d} - - {X x}_{f f} \\ {Y Y}_{d d} - - {Y Y}_{f f} \end{matrix}] - - - - - - ((1414))$

给一副图像中n个角点，故共可得2n个方程，表示成矩阵形式为Given n corner points in an image, a total of 2n equations can be obtained, expressed in matrix form as

Dk＝dDk=d

其最小二乘解如式(15)所示Its least squares solution is shown in formula (15)

k＝(D^TD)^-1D^Td (15)k＝(D ^T D) ^-1 D ^T d (15)

(5)求精摄像机内外部参数：为获取精确的摄像机模型参数，需考虑提取的全部图像角点和对应世界坐标点，建立如式(14)所示优化模型，求精所有内、外部参数。(5) Refine the internal and external parameters of the camera: in order to obtain accurate camera model parameters, it is necessary to consider all the extracted image corners and corresponding world coordinate points, establish an optimization model as shown in formula (14), and refine all internal and external parameters .

$min min = = {Σ Σ}_{i i = = 11}^{n no} {| | | | {\overset{\cap \cap}{M m}}_{i i} - - \overset{\cup \cup}{M m} (({C C}_{x x},, {C C}_{y the y},, m m,, f f,, l l,, t t,, p p,, s the s,, k k,, {W W}_{i i})) | | | |}^{22} - - - - - - ((1616))$

n：场景图像上角点个数n: the number of corner points on the scene image

W_i：场景图像上第i个角点的实际坐标W _i : the actual coordinates of the i-th corner point on the scene image

第i个角点的实际图像坐标

The actual image coordinates of the i-th corner point

由W_i代入实际成像模型求出映射图像坐标

Substituting W _i into the actual imaging model to obtain the mapped image coordinates

优化模型的初值为利用本节算法所求出的摄像机模型各参数值以及初始的畸变系数，模型参数可采用Levenberg-Marquardt优化算法求解。The initial value of the optimization model is the parameter values of the camera model and the initial distortion coefficient obtained by the algorithm in this section, and the model parameters can be solved by the Levenberg-Marquardt optimization algorithm.

(1)设监控摄像机的标称放大系数范围为0～k，针对同一场景，固定摄像机其余参数，逐渐增大标称放大系数，以步长m拍摄k/m幅场景图像，以图像中心为主点初值，利用式(6)、式(7)，使用powell方向族法，迭代求出不同标称放大系数下，摄像机主点及图像实际放大系数；(1) Assuming that the nominal magnification factor of the surveillance camera ranges from 0 to k, for the same scene, fix the other parameters of the camera, gradually increase the nominal magnification factor, and shoot k/m scene images with a step size of m, taking the center of the image as The initial value of the main point, using formula (6) and formula (7), using the Powell direction family method, iteratively find the main point of the camera and the actual magnification factor of the image under different nominal magnification factors;

(2)检测场景图像角点，利用图像主点附近四相邻角点，利用式(9)、(12)，计算摄像机的内部固定参数及外部空间方位参数；利用图像上全部角点，利用式(15)计算摄像机一阶径向畸变。(2) Detect the corner points of the scene image, use the four adjacent corner points near the principal point of the image, and use formulas (9) and (12) to calculate the internal fixed parameters of the camera and the external space orientation parameters; use all the corner points on the image, use Equation (15) calculates the first-order radial distortion of the camera.

(3)将检测的图像角点及其对应的世界坐标点代入式(16)，求解非线性最小化模型，求精摄像机全部参数。(3) Substitute the detected image corners and their corresponding world coordinates into Equation (16), solve the nonlinear minimization model, and refine all the parameters of the camera.

表1 物像换算精度Table 1 Object image conversion accuracy

最小值minimum value 最大值Maximum 方差Variance 像平面image plane uu -0.813-0.813 0.7040.704 0.43310.4331 vv -0.544-0.544 0.4270.427 0.40090.4009 世界坐world sit xx -0.585-0.585 0.3220.322 0.23300.2330 yy -0.233-0.233 0.2120.212 0.13310.1331

为了验证本发明所提出方法的有效性，本发明的一个实施例采用了图4所示的高速公路交通场景图像，并在此交通场景图像中选择分道线角点作为标定目标，并以图像主点附近四角点作为主要标定特征，记为A、B、C、D，如图5所示。分道线间距离事先已知。In order to verify the effectiveness of the method proposed by the present invention, an embodiment of the present invention adopts the highway traffic scene image shown in Figure 4, and selects the corner point of the lane line in this traffic scene image as the calibration target, and uses the image The four corner points near the main point are used as the main calibration features, which are marked as A, B, C, and D, as shown in Figure 5. The distance between lane lines is known in advance.

图6，包括6a、6b所示为不同标称焦距下摄像机主点及放大系数的实际分布，显然随标称放大系数的增加，图像主点坐标拟直线递增，其与图像中心的位移范围为±30个象素。而实际放大系数则为拟抛物线递增过程。以分道线角点作为测试样本点，利用摄像机标定结果，采用样本点的物像空间正反换算与样本点的标准值进行比较作为评价尺度。表1所示为样本点物像换算与标准值间的差值精度。实验结果表明，本发明提出的分级标定方法物像换算精度高，完全能够满足交通监控的精度要求。Figure 6, including 6a and 6b, shows the actual distribution of the principal point and the magnification factor of the camera at different nominal focal lengths. Obviously, with the increase of the nominal magnification factor, the coordinates of the principal point of the image increase in a quasi-linear manner, and the displacement range between the principal point and the image center is ±30 pixels. The actual amplification factor is a quasi-parabolic increasing process. The corner points of the lane line are used as the test sample points, and the camera calibration results are used to compare the positive and negative conversion of the object image space of the sample point with the standard value of the sample point as the evaluation scale. Table 1 shows the difference accuracy between the sample point object image conversion and the standard value. Experimental results show that the hierarchical calibration method proposed by the present invention has high accuracy in object-image conversion, and can fully meet the accuracy requirements of traffic monitoring.

Claims

1. The road condition camera calibration method under the traffic monitoring environment is characterized in that it comprises the following calibration steps:

(1) Visual model description and related coordinate system establishment: In view of the performance requirements of the monitoring system, the classic Tsai transmission projection model is used, and according to the characteristics of road condition imaging, it is corrected accordingly, a new visual model is proposed, and three coordinate systems are established:

The ground coordinate system X _w -Y _w -Z _w and the camera coordinate system X _c -Y _c -Z _c are used to represent the three-dimensional space; the image plane coordinate system X _f -Y _f is used to represent the imaging plane. Establish a world coordinate system whose origin is the intersection of the camera optical axis and the ground. The positive direction of the Y _w axis is pointing forward along the road surface, the positive direction of the X _w axis is pointing to the right, and the positive direction of the Z _w axis is perpendicular to the ground, and the direction is upward. Establish a camera coordinate system, the origin is the position of the optical center of the camera, the Z _c axis is the direction of the optical axis of the camera, and the X _c -Y _c plane is parallel to the image plane;

X _c = (cos(p)cos(s)+sin(t)sin(p)sin(s)) _Xw

+(sin(p)cos(s)-sin(t)cos(p)sin(s))Y _w

Y _c ＝(-cos(p)sin(s)+sin(t)sin(p)cos(s)) _Xw (1)

+(-sin(p)sin(s)+sin(t)cos(p)cos(s))Y _w

Z _c ＝-cos(t)sin(p) _Xw +cos(t)cos(p) _Yw +l

\{\begin{matrix} \frac{11}{M m ((z z))} [[{X x}_{f f} - - {C C}_{x x} ((Z Z))]] ((11 + + {kr kr}^{22})) = = f f \frac{{X x}_{C C}}{{Z Z}_{C C}} \\ \frac{11}{M m ((z z))} [[{Y Y}_{f f} - - {C C}_{y the y} ((Z Z))]] ((11 + + {kr kr}^{22})) = = - - f f \frac{{Y Y}_{C C}}{{Z Z}_{C C}} \end{matrix} - - - - - - ((22))

Establish the mapping relationship between the world coordinate system and the image plane coordinate system under this visual model, as shown in formula (3)

{X x}_{f f} = = \frac{f f [\begin{matrix} ((cos cos ((p p)) cos cos ((s the s)) + + sin sin ((t t)) sin sin ((p p)) sin sin ((s the s)))) {X x}_{w w} \\ + + ((sin sin ((p p)) cos cos ((s the s)) - - sin sin ((t t)) cos cos ((p p)) sin sin ((s the s)))) {Y Y}_{w w} \end{matrix}]}{- - cos cos ((t t)) sin sin ((p p)) {X x}_{w w} + + cos cos ((t t)) cos cos ((p p)) {Y Y}_{w w} + + l l} - - - - - - ((33))

{Y Y}_{f f} = = \frac{f f [\begin{matrix} ((- - cos cos ((p p)) sin sin ((s the s)) + + sin sin ((t t)) sin sin ((p p)) cos cos ((s the s)))) {X x}_{w w} \\ + + ((- - sin sin ((p p)) sin sin ((s the s)) + + sin sin ((t t)) cos cos ((p p)) cos cos ((s the s)))) {Y Y}_{w w} \end{matrix}]}{- - cos cos ((t t)) sin sin ((p p)) {X x}_{w w} + + cos cos ((t t)) cos cos ((p p)) {Y Y}_{w w} + + l l}

Starting from formula (3), taking Z _w as a known parameter, the inverse mapping relationship from the image plane coordinates to the world coordinate system can also be established, as shown in formula (4):

{X x}_{w w} = = \frac{[\begin{matrix} sin sin p p ((l l + + {Z Z}_{w w} sin sin t t)) (({x x}_{f f} sin sin s the s + + {y the y}_{f f} cos cos s the s)) \\ + + cos cos p p ((l l sin sin t t + + {Z Z}_{w w})) (({x x}_{f f} cos cos s the s - - {y the y}_{f f} sin sin s the s)) - - {Z Z}_{w w} f f cos cos t t sin sin p p \end{matrix}]}{{x x}_{f f} cos cos t t sin sin s the s + + {y the y}_{f f} cos cos t t cos cos s the s + + f f sin sin t t} - - - - - - ((44))

{Y Y}_{w w} = = \frac{[\begin{matrix} - - cos cos p p ((l l + + {Z Z}_{w w} sin sin t t)) (({x x}_{f f} sin sin s the s + + {y the y}_{f f} cos cos s the s)) \\ + + sin sin p p ((l l sin sin t t + + {Z Z}_{w w})) (({x x}_{f f} cos cos s the s - - {y the y}_{f f} sin sin s the s)) - - {Z Z}_{w w} f f cos cos t t sin sin p p \end{matrix}]}{{x x}_{f f} cos cos t t sin sin s the s + + {y the y}_{f f} cos cos t t cos cos s the s + + f f sin sin t t}

In the formula, H is the vertical installation height of the camera, t, p, s are the pitch angle, deflection angle and rotation angle of the camera respectively, f is the wide-angle effective focal length, M(z) is the scaling factor, Cx(z), Cy(z ) is the principal point coordinates of the image, and k is the first-order radial distortion.

(2) Calibrate the principal point of the camera and the scaling factor: take the optical flow of the monitoring image as the calibration primitive; use the optical flow field difference between the predicted image of the reference frame and the sampled image of the real-time frame as a constraint through the zooming movement of the camera, and use the minimum The constraint equation is established by the square method, and the coordinates of the principal point of the camera and the actual magnification factor of the camera are identified by the Powell direction family method.

(3) Calibration target selection and parameter linear solution: In the surveillance scene, select the corner points of the lane-dividing line as the calibration reference object, and use the image plane projection of the four adjacent corner points of the lane-dividing line to determine the slope of the vanishing line and the corresponding camera rotation angle; With the parallel relation of the lane-dividing lines and the basic road width between the lane-dividing lines as constraints, the effective focal length of the camera and its spatial position parameters are linearly solved by using the image plane coordinates of the four corner points;

(4) Refine the internal and external parameters of the camera: take all the corner points in the monitoring image and the corresponding world coordinate points, use the visual model parameters obtained by the above algorithm as the initial value of the optimization model, and use the Levenberg-Marquardt optimization algorithm to refine the camera model parameters to complete the camera calibration.

2. the road condition camera calibration method under the traffic surveillance environment according to claim 1, it is characterized in that adopting camera distortion compensation: utilize calibration to go out complete ideal camera imaging model without perspective distortion, calculate the angle of all corners on the image The ideal pixel coordinates are used to solve the first-order radial distortion in the nonlinear visual model through the coordinate difference between the ideal pixel coordinates and the projection of the real corner image plane.