CN112070842B - Multi-camera global calibration method based on orthogonal coding stripes - Google Patents

Abstract

The invention discloses a multi-camera global calibration method based on orthogonal coding stripes, which specifically includes the following steps: step S1: designing a coding sequence; step S2: establishing a global coordinate system; step S3: performing local calibration of two cameras respectively, Two cameras are used to collect a set of orthogonally encoded fringe images, and at least three groups of images are taken; Step S4: For an orthogonally encoded fringe image, calculate its absolute unwrapped phase in the horizontal/vertical direction; Step S5: According to the absolute unwrapped phase , set up the one-to-one mapping relationship between the image coordinates of each pixel in the orthogonal coded stripe image and the world coordinates; set up the camera projection equation, solve the internal parameters and external parameters of the two cameras; step S6: repeat steps S3-S5 until Each camera in the multi-camera system has been calibrated locally, and all cameras are unified into the global coordinate system to realize the global calibration of the multi-camera system.

Description

A Multi-camera Global Calibration Method Based on Orthogonal Coded Stripes

technical field

The invention belongs to the technical field of multi-camera imaging, in particular, the invention relates to a multi-camera global calibration method based on orthogonal coding stripes.

Background technique

Multi-camera systems are widely used in security monitoring, target tracking, industrial measurement, autonomous navigation and other fields. One of the key issues is multi-camera calibration to determine the internal parameters (such as focal length, principal point, distortion, etc.) and external parameters ( Such as rotation matrix, translation vector, etc.), its calibration accuracy will directly affect the performance of the multi-camera system.

According to the dimensions, traditional targets can be divided into: one-dimensional targets, two-dimensional targets and three-dimensional targets. In contrast, the processing and maintenance of two-dimensional targets are relatively simple and are the most widely used. Commonly used two-dimensional targets include checkerboards and dot arrays. The feature points of the checkerboard are the corner points, and the feature points of the dot array are the center of the circle. Their extraction accuracy is greatly affected by the gray scale, which in turn affects the camera calibration. precision.

Many scholars have designed different types of phase targets, including phase-shifted fringes, phase-shifted circular fringes, and orthogonal fringes. The common point is to extract feature points from the phase distribution of the target.

Among them, phase-shift fringes and phase-shift circular fringes use the phase-shift method to calculate their phases, but the phase-shift method generally requires multiple fringes. Therefore, during the calibration process, the camera needs to be kept fixed, and multiple fringe images need to be taken for each camera orientation. Relatively time-consuming and labor-intensive.

Among them, the phase of the orthogonal fringe is calculated by Fourier analysis method, and the calibration flexibility is improved. However, for multi-camera calibration, the arrangement of multi-cameras is different in different applications, and there may be situations where adjacent cameras have no common field of view.

Regardless of the traditional two-dimensional target or phase target, the traditional multi-camera calibration method needs to place the target in the public field of view for shooting, which limits its application range. To sum up, how to improve the accuracy and flexibility of multi-camera calibration is still of great significance.

Contents of the invention

The present invention provides a multi-camera global calibration method based on orthogonal coding stripes to solve the problems in the background technology above.

In order to achieve the above object, the technical solution adopted by the present invention is: a multi-camera global calibration method based on orthogonal coding stripes, which specifically includes the following steps,

Step S1: Design the coding sequence, and embed several coding points into the orthogonal stripes according to the coding sequence to generate the orthogonal coding stripes I(x,y); then display the orthogonal coding stripes on the LCD screen or print them on a white On a flat plate, as a phase target;

Step S2: Select the camera in the multi-camera system that overlaps more with the field of view of other cameras as the reference camera, and establish a global coordinate system based on the reference camera coordinate system. Two-two cameras with more degrees are divided into one group;

Step S3: Carry out local calibration of two cameras respectively, place the orthogonal coding stripes within the field of view of the two cameras, and the two cameras include a reference camera or a camera that has completed local calibration; change the placement of the orthogonal coding stripes Orientation, two cameras are used to collect a set of orthogonally coded fringe images I _a (u, v), I _b (u, v) in each orientation, and at least three sets of images are taken;

Step S4: For an orthogonally coded fringe image I(u,v), calculate its absolute unwrapped phase Φ _u (u,v),Φ _v (u,v) in the horizontal/vertical direction;

Step S5: According to the absolute unwrapped phase Φ _u (u, v), Φ _v (u, v), establish the image coordinates (u, v) and the world The one-to-one mapping relationship of coordinates (X, Y, Z); establish the camera projection equation, and solve the internal parameters and external parameters of the two cameras;

Step S6: Repeat steps S3-S5 until each camera in the multi-camera system has completed local calibration; finally use the external parameters between two cameras to unify all cameras into the global coordinate system to realize the global calibration of the multi-camera system .

Preferably, in the step S1, the target intensity of the orthogonal coding stripes before embedding the coding points can be expressed as:

I(x,y)＝0.5+0.25cos(2πx/p _x )+0.25cos(2πy/p _y )

Where (x, y) represents the coordinates of the target; p _x and p _y represent the fringe period in the horizontal/vertical direction, respectively; the intensity of the orthogonal coded fringe image before embedding the code point can be expressed as:

I(u,v)＝A(u,v)+B _u (u,v)cos[φ _u (u,v)]+B _v (u,v)cos[φ _v (u,v)]

where (u,v) denotes the image coordinates; A(u,v) denotes the background intensity; B _u (u,v) and B _v (u,v) denote the modulation intensity in the horizontal/vertical directions, respectively.

Preferably, in the step S3, in the local calibration of the two cameras, the field of view of the two cameras partially overlaps or does not overlap, but both cameras can collect some or all images of the orthogonal coding stripes.

Preferably, the step S4 specifically includes the following steps,

Step S41: Carry out Fourier analysis on the orthogonal coded fringe image I(u,v), and calculate the truncated phases φ _u (u,v) and φ _v (u,v) in the horizontal/vertical direction respectively;

Step S42: Perform local phase unwrapping on the truncated phases φ _u (u, v), φ _v (u, v) respectively, and calculate the local unwrapped phases Φ′ _u (u, v), Φ′ _v in the horizontal/vertical direction (u,v);

_{Step S43 :} Solve the _horizontal / local fringe order k′ _u (u,v), k′ _v (u,v) in the vertical direction;

Step S44: Perform median filtering on the orthogonally encoded stripe image I(u,v) to obtain the filtered image I'(u,v), and filter out the coded points in the orthogonally encoded stripe image I(u,v) ;

Step S45: Subtract the filtered image I′(u,v) from the orthogonal coded stripe image I(u,v), and then binarize the subtracted image to extract the coded point image H(u,v) ;

Step S46: Correct the local fringe order k′ _u (u, v), k′ _v (u, v) according to the code point image H(u, v), and obtain the absolute fringe order k in the horizontal/vertical direction _u (u, v), k _v (u, v);

Step S47: Combining the truncated phase φ _u (u,v), φ _v (u,v) and the absolute fringe order k _u (u,v), k _v (u,v), calculate the absolute unwrapped phase Φ _u ( u, v), Φ _v (u, v).

Preferably, in the step S43, the truncated phase φ _u (u, v), φ _v (u, v) and the local expansion phase Φ′ _u (u, v), Φ′ _v (u, v) The mathematical relationship is:

k' _r (u, v) = floor[Φ' _r (u, v)/(2π)], r = u, v.

Preferably, in the step S46, the calculation method of the absolute fringe order k _u (u, v), k _v (u, v) is: according to the local fringe order k' _u (u, v), k ' _v (u, v) distinguishes each stripe cycle in the horizontal/vertical direction; extracts the code value of each stripe cycle from the code point image, and forms the code word codeword with the code values of continuous m=6 stripe cycles; Then look up the codeword codeword in the coding sequence, and determine the absolute stripe order k _u (u, v), k _v (u, v) according to its position.

Preferably, in the step S47, the calculation formula of the absolute unwrapped phase Φ _u (u, v), Φ _v (u, v) is:

Φ _r (u, v) = φ _r (u, v) + 2πk _r (u, v), r = u, v.

Preferably, in the step S5, according to the absolute unfolding phase Φ _u (u, v), Φ _v (u, v), one of the established image coordinates (u, v) and world coordinates (X, Y, Z) A mapping relationship is:

Wherein, d _x , d _y represent the pixel widths of the orthogonal coding stripes in the horizontal/vertical direction, respectively.

Preferably, in the step S5, the method for solving internal parameters and external parameters of the two cameras is preferably Zhang Zhengyou's calibration algorithm (IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-4), and the camera model Select the pinhole imaging model.

The beneficial effect of adopting above technical scheme is:

1. The technical solution of the present invention only uses one orthogonal coding fringe as a phase target, uses Fourier analysis method to extract its truncated phase, and uses coded points to recover its absolute unfolded phase, which has high phase extraction and unfolding precision.

2. For the orthogonal coding stripes adopted in the technical solution of the present invention, all the pixels with effective phases can be used as phase feature points, which effectively guarantees the calibration accuracy of the camera.

3. The technical solution of the present invention can simultaneously use the phase feature points in the overlapping field of view and the non-overlapping field of view to perform pairwise camera calibration, with strong flexibility and wide applicability.

Description of drawings

Figure 1 is a schematic diagram of orthogonal coding stripes;

Figure 2 is a schematic diagram of the calibration of two cameras;

Fig. 3 is the absolute unwrapped phase calculation flowchart of the orthogonally coded fringe image;

Detailed ways

The specific embodiment of the present invention will be described in further detail by describing the embodiments below with reference to the accompanying drawings, the purpose is to help those skilled in the art to have a more complete, accurate and in-depth understanding of the concept and technical solutions of the present invention, and contribute to its implementation.

As shown in Figures 1 to 3, the present invention is a multi-camera global calibration method based on orthogonal coding stripes, which specifically includes the following steps,

Step S1: Design the coding sequence, and embed several coding points into the orthogonal stripes according to the coding sequence to generate the orthogonal coding stripes I(x,y); then display the orthogonal coding stripes on the LCD screen or print them on a white on the flat plate as a phase target, as shown in Figure 1;

Step S2: Select the camera in the multi-camera system that overlaps more with the field of view of other cameras as the reference camera, and establish a global coordinate system based on the reference camera coordinate system. Two-two cameras with more degrees are divided into one group;

Step S3: Carry out local calibration of the two cameras respectively, place the orthogonal coding stripes within the field of view of the two cameras, and the two cameras include a reference camera or a camera that has been locally calibrated; as shown in Figure 2, change the normal The placement orientation of the cross-coding stripes, use two cameras to collect a set of orthogonal coding stripe images I _a (u, v), I _b (u, v) under each placement orientation, and take at least three sets of images;

Step S4: For an orthogonally coded fringe image I(u,v), calculate its absolute unwrapped phase Φ _u (u,v),Φ _v (u,v) in the horizontal/vertical direction, as shown in Figure 3 ;

Step S5: According to the absolute unwrapped phase Φ _u (u, v), Φ _v (u, v), establish the image coordinates (u, v) and the world The one-to-one mapping relationship of coordinates (X, Y, Z); establish the camera projection equation, and solve the internal parameters and external parameters of the two cameras;

Step S6: Repeat steps S3-S5 until each camera in the multi-camera system has completed local calibration; finally use the external parameters between two cameras to unify all cameras into the global coordinate system to realize the global calibration of the multi-camera system .

In the step S1, the target intensity of the orthogonal coding stripes before embedding the coding points can be expressed as:

I(x,y)＝0.5+0.25cos(2πx/p _x )+0.25cos(2πy/p _y )

Where (x, y) represents the coordinates of the target; p _x and p _y represent the fringe period in the horizontal/vertical direction, respectively; the intensity of the orthogonal coded fringe image before embedding the code point can be expressed as:

I(u,v)＝A(u,v)+B _u (u,v)cos[φ _u (u,v)]+B _v (u,v)cos[φ _v (u,v)]

where (u,v) denotes the image coordinates; A(u,v) denotes the background intensity; B _u (u,v) and B _v (u,v) denote the modulation intensity in the horizontal/vertical directions, respectively.

In the step S1, the coded sequence is composed of two code values of '0' and '1', as shown in Figure 1, the selected coded sequence is:

'01101110101011110010110011010010011100010100011000010 00000111111', note that only part of the code sequence (the first 15 code values) is shown in Figure 1; a codeword codeword composed of consecutive m=6 code values, each codeword codeword can only appear in the entire code sequence Once; take the stripe level k _u (x, y), k _v (x, y) as the index value, if the code value of its index in the coding sequence is '1', then embed the code point in the corresponding stripe period , if the code value of its index in the code sequence is '0', no code point is embedded in the corresponding stripe period; for example, when the stripe order k _u (x,y)=8,10,12, it is in The code values of the indexes in the coding sequence are all '1', and the coding points are embedded in the corresponding stripe period; when the stripe level k _u (x, y)=7,9,11, the indexed If the code values are all '0', the code point is not embedded in the corresponding stripe period; the target coordinates (x _c , y _c ) embedded in the code point can be expressed as:

Among them, c _i represents the i-th code value, and c _j represents the j-th code value.

In the step S3, the local calibration of the two cameras, the fields of view of the two cameras are partially overlapped or not overlapped, but the two cameras can collect some or all images of the orthogonal coding stripes; as shown in Figure 2, the two cameras are Partial images of orthogonally encoded stripes can be acquired.

The step S4 specifically includes the following steps,

Step S41: Carry out Fourier analysis on the orthogonal coded fringe image I(u,v), and calculate the truncated phases φ _u (u,v) and φ _v (u,v) in the horizontal/vertical directions respectively; this process As an existing technology, the specific implementation can refer to the literature (Optik, 2012, 123(2): 171-5);

Step S42: Perform local phase unwrapping on the truncated phases φ _u (u, v), φ _v (u, v) respectively, and calculate the local unwrapped phases Φ′ _u (u, v), Φ′ _v in the horizontal/vertical direction (u,v); Space algorithms such as Quality guided method and Branch cut method can be used;

_{Step S43 :} Solve the _horizontal /local fringe order k _u ′(u,v),k _v ′(u,v) in the vertical direction;

Step S44: Perform median filtering on the orthogonally encoded stripe image I(u,v) to obtain the filtered image I'(u,v), and filter out the coded points in the orthogonally encoded stripe image I(u,v) ;

Step S45: Subtract the filtered image I′(u,v) from the orthogonal coded stripe image I(u,v), and then binarize the subtracted image to extract the coded point image H(u,v) ;

Step S46: Correct the local fringe order k _u '(u,v), k _v '(u,v) according to the code point image H(u,v), and obtain the absolute fringe order k in the horizontal/vertical direction _u (u, v), k _v (u, v);

Step S47: Combining the truncated phase φ _u (u,v), φ _v (u,v) and the absolute fringe order k _u (u,v), k _v (u,v), calculate the absolute unwrapped phase Φ _u ( u, v), Φ _v (u, v).

In the step S43, the mathematical relationship between the truncated phase φ _u (u, v), φ _v (u, v) and the locally expanded phase Φ′ _u (u, v), Φ′ _v (u, v) is :

k' _r (u, v) = floor[Φ' _r (u, v)/(2π)], r = u, v.

In the step S46, the calculation method of the absolute fringe order k _u (u, v), k _v (u, v) is: according to the local fringe order k' _u (u, v), k' _v ( u, v) Distinguish each stripe cycle in the horizontal/vertical direction; extract the code value of each stripe cycle from the coded point image, and form the code word codeword with the code values of continuous m=6 stripe cycles; then encode The codeword codeword is searched in the sequence, and the absolute stripe order k _u (u, v), k _v (u, v) can be determined according to its position.

In the step S47, the calculation formula of the absolute unwrapped phase Φ _u (u, v), Φ _v (u, v) is:

Φ _r (u, v) = φ _r (u, v) + 2πk _r (u, v), r = u, v.

In the step S42 and step S47, for the same group of orthogonal coded fringe images I _a (u, v) and I _b (u, v), the zero phase points of their local unwrapped phases may be different, and the zero phase points of their absolute unwrapped phases The phase points must be the same.

In the step S5, according to the absolute unwrapped phase Φ _u (u, v), Φ _v (u, v), the one-to-one mapping relationship between the image coordinates (u, v) and the world coordinates (X, Y, Z) established for:

Wherein, d _x , d _y represent the pixel widths of the orthogonal coding stripes in the horizontal/vertical direction, respectively.

In the step S5, the method for solving internal parameters and external parameters of the two cameras is preferably Zhang Zhengyou's calibration algorithm (IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-4), and the camera model selects pinhole imaging model.

The present invention has been exemplarily described above in conjunction with the accompanying drawings. Obviously, the specific implementation of the present invention is not limited by the above methods, as long as various insubstantial improvements are made by adopting the method concept and technical solutions of the present invention; or not After improvement, the above-mentioned ideas and technical solutions of the present invention are directly applied to other occasions, all within the protection scope of the present invention.

Claims (6)

1. A multi-camera global calibration method based on orthogonal coding stripes, characterized in that: specifically comprise the following steps, Step S1: Design the coding sequence, and embed several coding points into the orthogonal stripes according to the coding sequence to generate the orthogonal coding stripes I(x,y); then display the orthogonal coding stripes on the LCD screen or print them on a white On a flat plate, as a phase target; Step S2: Select the camera in the multi-camera system that overlaps more with the field of view of other cameras as the reference camera, and establish a global coordinate system based on the reference camera coordinate system. Two-two cameras with more degrees are divided into one group; Step S3: Carry out local calibration of two cameras respectively, place the orthogonal coding stripes within the field of view of the two cameras, and the two cameras include a reference camera or a camera that has completed local calibration; change the placement of the orthogonal coding stripes Orientation, two cameras are used to collect a set of orthogonally coded fringe images I _a (u, v), I _b (u, v) in each orientation, and at least three sets of images are taken; Step S4: For an orthogonally coded fringe image I(u,v), calculate its absolute unwrapped phase Φ _u (u,v),Φ _v (u,v) in the horizontal/vertical direction; Step S5: According to the absolute unwrapped phase Φ _u (u, v), Φ _v (u, v), establish the image coordinates (u, v) and world The one-to-one mapping relationship of coordinates (X, Y, Z); establish the camera projection equation, and solve the internal parameters and external parameters of the two cameras; Step S6: Repeat steps S3-S5 until each camera in the multi-camera system has completed local calibration; finally use the external parameters between two cameras to unify all cameras into the global coordinate system to realize the global calibration of the multi-camera system ; In the step S1, the target intensity of the orthogonal coding stripes before embedding the coding points can be expressed as: I(x,y)＝0.5+0.25cos(2πx/p _x )+0.25cos(2πy/p _y ) Where (x, y) represents the coordinates of the target; p _x and p _y represent the fringe period in the horizontal/vertical direction, respectively; the intensity of the orthogonal coded fringe image before embedding the code point can be expressed as: I(u,v)＝A(u,v)+B _u (u,v)cos[φ _u (u,v)]+B _v (u,v)cos[φ _v (u,v)] where (u, v) represents the image coordinates; A(u, v) represents the background intensity; B _u (u, v) and B _v (u, v) represent the modulation intensity in the horizontal/vertical direction, respectively; In the step S3, the local calibration of the two cameras, the fields of view of the two cameras are partially overlapped or not overlapped, but both cameras can collect some or all images of the orthogonal coding stripes; The step S4 specifically includes the following steps, Step S41: Carry out Fourier analysis on the orthogonally coded fringe image I(u, v), and calculate the truncated phases φ _u (u, v) and φ _v (u, v) in the horizontal/vertical direction respectively; Step S42: Perform local phase unwrapping on the truncated phases φ _u (u, v), φ _v (u, v) respectively, and calculate the local unwrapped phases Φ′ _u (u, v), Φ′ _v in the horizontal/vertical direction (u,v); _{Step S43 :} Solve the _horizontal /local fringe order k _u ′(u,v),k _v ′(u,v) in the vertical direction; Step S44: Perform median filtering on the orthogonally encoded stripe image I(u,v) to obtain the filtered image I'(u,v), and filter out the coded points in the orthogonally encoded stripe image I(u,v) ; Step S45: Subtract the filtered image I′(u,v) from the orthogonal coded stripe image I(u,v), and then binarize the subtracted image to extract the coded point image H(u,v) ; Step S46: Correct the local fringe order k _u '(u,v), k _v '(u,v) according to the code point image H(u,v), and obtain the absolute fringe order k in the horizontal/vertical direction _u (u, v), k _v (u, v); Step S47: Combining the truncated phase φ _u (u,v), φ _v (u,v) and the absolute fringe order k _u (u,v), k _v (u,v), calculate the absolute unwrapped phase Φ _u ( u, v), Φ _v (u, v). 2. A kind of multi-camera global calibration method based on orthogonal coding stripes according to claim 1, characterized in that: in the step S43, the truncated phase φ _u (u, v), φ _v (u, The mathematical relationship between v) and the locally expanded phase Φ′ _u (u, v), Φ′ _v (u, v) is: k _r '(u, v)=floor[Φ' _r (u, v)/(2π)], r=u or v. 3. A multi-camera global calibration method based on orthogonal coding fringes according to claim 1, characterized in that: in the step S46, the absolute fringe order k _u (u, v), k _v ( The calculation method of u,v) is: according to the local fringe order k _u ′(u,v),k _v ′(u,v) to distinguish each fringe cycle in the horizontal/vertical direction; extracting from the code point image For the code value of each stripe period, the code values of continuous m=6 stripe periods form the codeword codeword; then the codeword codeword is searched in the coding sequence, and the absolute stripe order k _u (u,v can be determined according to its position ), k _v (u, v). 4. A kind of multi-camera global calibration method based on orthogonal coding stripes according to claim 1, characterized in that: in the step S47, the absolute unwrapped phase Φ _u (u, v), Φ _v (u ,v) is calculated as: Φ _r (u, v) = φ _r (u, v) + 2πk _r (u, v), r = u or v. 5. A kind of multi-camera global calibration method based on orthogonal coding stripes according to claim 1, characterized in that: in the step S5, according to the absolute unwrapped phase Φ _u (u, v), Φ _v (u, v), the one-to-one mapping relationship between the established image coordinates (u, v) and world coordinates (X, Y, Z) is:

Wherein, d _x , d _y represent the pixel widths of the orthogonal coding stripes in the horizontal/vertical direction, respectively. 6. A multi-camera global calibration method based on orthogonal coding stripes according to claim 1, characterized in that: in the step S5, the method for solving the internal parameters and external parameters of the two cameras adopts Zhang Zhengyou's calibration algorithm , the camera model selects the pinhole imaging model.

Images