CN102682445B - Coordinate extracting algorithm of lacertilian-imitating suborder chamaeleonidae biological vision - Google Patents

Abstract

The invention belongs to the category of computer vision, in particular to a computer vision algorithm capable of simultaneously performing target tracking, positioning and camera calibration. The invention proposes a set of camera calibration method and target tracking and positioning method applied to the active and stereoscopic vision system modeled on the structure of the lizard suborder evasionidae biological vision system. Its main feature is camera calibration. The tracking and positioning of the target of interest are carried out simultaneously. The calibration process of the camera does not require the target of interest to remain still, nor does it need to be performed before the target tracking and positioning. The calibration method adopted in the present invention transforms the continuous calibration process into a discrete calibration process, uses the discrete static state of the target during the operation of the visual algorithm to calibrate the camera, reduces the requirements for camera calibration and realizes the visual algorithm and calibration process is performed synchronously. Its self-calibration feature reduces the manual calibration work and greatly simplifies the preparatory work of the algorithm.

Description

An Algorithm for Extracting Visual Coordinates of Imitation Lizards

technical field

The invention belongs to the category of computer vision, in particular to a computer vision algorithm capable of simultaneously performing target tracking, positioning and camera calibration. The biological system that it imitates is the visual system of the lizard suborder evasion family organisms.

Background technique

Computer vision is an important research direction in the field of artificial intelligence. Compared with other information detection methods, the detection device based on the concept of computer vision has the characteristics of large amount of information, low cost, no interference to the environment, and easy control. There are many ways to structure the vision system, which can be divided into: monocular vision, binocular vision, and multi-eye vision based on the number of cameras; it can be divided into: active vision and passive vision based on mechanical structure constraints; The degree of association can be divided into: ordinary vision, stereo vision, etc.

For the simplest monocular vision, it has the advantages of simple structure, simple vision and control algorithm, and low cost. However, since the visual process based on ordinary monocular vision is a process of mapping from the high-dimensional real world to the low-dimensional image space, a lot of information will inevitably be lost in this process, the most important of which is the conventional monocular vision. Visual vision cannot obtain the depth information of the target.

Stereo vision based on binocular vision can make up for the defect that ordinary monocular vision cannot obtain depth information. By ensuring that the two fixed cameras in the vision system have a suitable common field of view, supplemented by precise camera calibration and stereo matching algorithms, the stereo vision system based on binocular vision can obtain the depth information of the target. However, this kind of vision system architecture is still an open-loop system in essence, and it cannot adapt to changes in the environment by changing its own parameters when running the vision algorithm.

The way to solve this defect and make the vision system a stable closed-loop system is to build the vision system with the architecture of active vision, that is, to add a certain movement ability to the camera, so that the vision system can use the image information it obtains As the feedback amount, the camera is driven to rotate, and the entire vision system is controlled in a closed loop.

Since the vision system based on the active vision concept architecture is different from the traditional vision system based on the passive vision concept architecture in terms of mechanical structure, its applicable depth information acquisition method, interest target positioning method, and camera calibration method are also different from passive vision. The approaches taken vary. For a vision system based on the concept of active vision, if subsequent vision algorithm processing is required, it first needs to calibrate the camera used. However, the conventional calibration method requires that the vision algorithm be executed before the operation of the vision algorithm, and the target of interest is required to be stationary during the calibration process. These two requirements make the use of active vision systems cumbersome, especially when the cameras in the vision system are changed frequently, or the system uses zoom cameras, which can make the vision processing process extremely cumbersome.

Contents of the invention

The invention proposes a set of camera calibration method and target tracking and positioning method applied to the active and stereoscopic vision system modeled on the structure of the lizard suborder evasionidae biological vision system. Its main feature is camera calibration. The tracking and positioning of the target of interest are carried out simultaneously. The calibration process of the camera does not require the target of interest to remain still, nor does it need to be performed before the target tracking and positioning. The applied vision system uses a wide-angle fixed-focus camera structure, and two wide-angle fixed-focus cameras can move independently of each other.

Its overall structure of the imitation lizard suborder evasion family biological visual system applied in the present invention can be summarized as follows:

The entire visual system is modeled on the visual system architecture of lizard suborder escaping creatures: its main components are 2 wide-angle fixed-focus cameras, and each wide-angle fixed-focus camera is equipped with 2 stepping motors to provide independent horizontal and vertical directions. Directional 2-DOF motion capability. Compared with conventional vision systems, the horizontal motion between the two wide-angle fixed-focus cameras is independent, and the pitch motion is also independent. The biological system that it imitates is the visual system of the lizard suborder Refugeidae (commonly known as chameleons). The characteristic of its visual system is that the two eyeballs can move independently of each other, rather than related movements like primates. It should be noted that the respective geometric center points of each wide-angle fixed-focus camera and the two matching stepping motors are on the same straight line perpendicular to the horizontal plane. This feature guarantees the depth The information and world coordinate extraction algorithm is carried out smoothly.

The main content of the present invention is to control two wide-angle fixed-focus cameras to work together, call the angle information calibration learning algorithm, depth information and world coordinate extraction algorithm detailed below, and use the Cam shift tracking algorithm as an auxiliary algorithm to simulate lizards. The biological vision system of the avoidance department performs motion control and parameter calculation, and finally obtains the world coordinates of the midpoint of the line between the geometric center points of the tracking target of the vision system relative to the two wide-angle fixed-focus cameras of the vision system in real time.

Technical scheme of the present invention is:

1. Let the two wide-angle fixed-focus cameras search for the target separately. When the A camera finds the target, start the monocular vision tracking algorithm based on the Cam shift algorithm to keep tracking the target, and return the normal vector of the image plane of the A camera at this time relative to the A in real time. The horizontal angle and the vertical angle of the camera head plane normal vector during the initial position of the camera; the A camera is the camera that first searches for the target, and the B camera is another camera; the initial position is: make the wide-angle fixed The image plane normal vector of the focus camera is parallel to the horizontal plane, and the image plane normal vector of the two wide-angle fixed-focus cameras is perpendicular to the position of the line segment formed by the geometric center point connection of the two wide-angle fixed-focus cameras;

2. The camera B follows the camera A to search for the target. After the camera B searches the target, it also enables the monocular vision tracking algorithm based on the Cam shift algorithm to keep track of the target;

3. Call the angle information calibration learning algorithm to calculate the horizontal and vertical angles of the respective image plane normal vectors relative to the respective initial position camera head plane normal vectors after the cameras A and B track the target in real time;

4. According to the result obtained in step 3), use the depth information and the world coordinate extraction algorithm to calculate and output the target depth information and the world coordinates of the target in the world coordinate system in real time; the world coordinate system is: focus with two wide angles The midpoint of the line connecting the geometric center points of the cameras is the origin, the direction parallel to the horizontal plane, perpendicular to the line connecting the geometric center points of the two wide-angle fixed-focus cameras, and the direction along the visual tracking is the positive direction of the x-axis, and the direction perpendicular to the horizontal plane is upward The right-handed coordinate system in the positive direction of the z-axis;

5. If the target is not lost, return to step 3), if the target is lost, return to step 1).

The angle information calibration learning algorithm includes the following steps:

2.1) Initialize learning entrance angle (Δθ ₁ , Δη ₁ )

The learning entrance angle refers to the horizontal angle and vertical angle of the corresponding camera rotation when the target moves from the origin to the position in the camera image when the target moves during the calibration process;

2.2) Calculate respectively in the camera image, the percentage f(P) and g(Q) of the horizontal distance P and the vertical distance Q of the camera image center point and the target with respect to the main diagonal length of the camera image;

2.3) Determine whether the target is at the center of the camera image,

If (|f(P)|<ε)∧(|g(Q)|<ε), threshold ε>0, then the target is at the center point, skip to step 2.7);

If (|f(P)|＞ε)∨(|g(Q)|＞ε), the threshold ε＞0, then the target is not at the center point, skip to step 2.4);

2.4) Check the calibration information table,

If there is angle information corresponding to |f(P)| and |g(Q)| in the calibration information table, the algorithm jumps to step 2.8);

If there is no angle information corresponding to |f(P)| and |g(Q)| in the calibration information table, then jump to step 2.5);

2.5) Turn the camera step by step so that the target coincides with the center point of the image in the camera image;

2.6) When the target coincides with the center point of the camera image, jump to step 2.7); otherwise, return to step 2.5);

2.7) When the target coincides with the center point of the image, read out the horizontal angle θ and the vertical angle η of the normal vector of the image plane of the camera at this time relative to the normal vector of the camera’s initial position of the camera, and output it, and go to step 2.10 );

2.8) If there is angle information corresponding to |f(P)| and |g(Q)| in the calibration information table, the camera rotates the angle so that the geometric center point of the target coincides with the image center point in the camera image, and the rotation direction is according to The calibration information conversion table is determined;

2.9) Under the output world coordinate system, the horizontal angle and the vertical angle of the target relative to the normal vector of the initial position camera; delay T time, wherein T is the preset delay amount, return to step 2.2);

2.10) Judging whether the learning is completed through the preset learning flag bit,

If the study is completed, then go to step 2.2);

If the learning is not completed, the algorithm continues and calibration begins;

2.11) Judging whether it is the first calibration,

If the learning entrance angle is equal to the initial value, it is the first calibration, and the algorithm continues at this time;

If the learning entrance angle is not equal to the initial value, it is not the first calibration, and then skip to step 2.13);

2.12) Let the camera rotate in the horizontal direction and the vertical direction (θ _ε , η _ε ), θ _ε , η _ε are threshold angles, skip to step 2.14)

2.13) The camera turns over the learning entrance angle recorded when the previous calibration was interrupted, and jumps to step 2.14);

2.14) If it is the initial calibration, the threshold angle (θ _ε , η _ε ) of the camera rotating in the horizontal direction and the vertical direction respectively and the corresponding |f(P)| and |g( Q)|Write into the calibration information table; if it is not the initial calibration, first judge whether the target has moved before entering this step; if the target has moved, update the learning entrance angle, and jump to step 2.2); if the target does not Movement, update the calibration information table;

2.15) Keeping the vertical angle of the camera unchanged, carry out line calibration, whenever the camera rotates a unit horizontal angle in the horizontal direction, judge whether the geometric center point of the target leaves the camera image at this time; if it has left, go to step 2.17); If not, continue the algorithm;

2.16) Determine whether the target moves after the camera rotates a unit horizontal angle in the horizontal direction. If there is no movement, update the calibration information table and jump to step 2.15); if it moves, update the learning entrance angle, and Skip to step 2.2):

2.17) Change the vertical angle of the camera and perform another line of horizontal calibration. If the geometric center of the target leaves the camera image, the calibration information table is established, and then go to step 2.18); if not, then go to step 2.14);

2.18) After the calibration information table is established, clear the learning flag and go to step 2.2);

The calibration information table is a two-dimensional vector table that records the camera imaging calibration information; the target position information is recorded in the table, and in the camera image, the target moves from the origin to the target position, and the camera head plane normal vector corresponds to the turned level Included angle and vertical included angle; Described target location information is by the percentage f(P) and g(Q) of the horizontal distance P of the camera image center point and the target and the vertical distance Q relative to the main diagonal length of the camera image express.

The calibration information conversion table can be described as:

First, the camera image is divided into 4 regions according to the quadrant:

Wherein imax is the number of rows of the camera image; jmax is the column number of the camera image; i refers to the i-th row of the camera image; _j refers to the j column of the camera image;

The calibration information table only records the calibration information of the I-quadrant camera image, and the calibration information of the other quadrant camera images can be calculated by following the rules of the calibration information conversion table; the calibration information conversion table is as follows:

I quadrant IF: (f(P)=|f(P)|)∧(g(Q)=|g(Q)|) |Δθ| |Δη| II quadrant IF: (-f(P)＝|f(P)|)∧(g(Q)＝|g(Q)|) -|Δθ| |Δη| III quadrant IF: (-f(P)＝|f(P)|)∧(-g(Q)＝|g(Q)|) -|Δθ| -|Δη| IV quadrant IF: (f(P)＝|f(P)|)∧(-g(Q)＝|g(Q)|) |Δθ| |Δη|

The depth information and world coordinate extraction algorithm includes the following steps:

According to the distance between the geometric centers of the two wide-angle fixed-focus cameras and the angle information calibration learning algorithm, the sum of the horizontal angles of the respective image plane normal vectors of the two wide-angle fixed-focus cameras relative to the respective initial position camera normal vectors The vertical angle is calculated using trigonometric functions to obtain the world coordinates of the target.

Scope of application of the present invention:

The camera calibration part in this set of technical solutions can be used as a calibration algorithm for monocular vision, and can also be used to calibrate multi-eye vision as independent monocular vision to reduce camera calibration requirements. The matching target tracking and localization method can be applied to the active vision system.

The present invention has the following advantages:

1) The conventional active vision camera calibration process requires the target to be stationary during the calibration process and the calibration process must precede the image processing process. The calibration method adopted in the present invention transforms the continuous calibration process into a discrete calibration process, uses the discrete static state of the target during the operation of the visual algorithm to calibrate the camera, reduces the requirements for camera calibration and realizes the visual algorithm and calibration process is performed synchronously. Its self-calibration feature reduces the manual calibration work and greatly simplifies the preparatory work of the algorithm. Its self-calibration function has an automatic interrupt recovery function. Even if the self-calibration process is disturbed by the outside world, it can also save the calibration breakpoint, and continue to complete the self-calibration after meeting the calibration conditions again.

2) Since the vision system architecture adopted in the present invention is a stereo vision architecture based on active vision, the vision system can accurately obtain the depth information of the target and calculate the corresponding world coordinates. Compared with the conventional stereo vision system, its supporting depth information and world coordinate extraction algorithm have a strong tolerance to the image distortion of the camera lens, so the system can use a wide-angle fixed-focus camera to expand its visual range without having to I am worried that the distortion caused by the wide-angle lens will affect the calculation of depth information and world coordinates.

3) The vision system applied in the present invention has at least 4 degrees of freedom, exceeding the number of degrees of freedom of conventional vision systems. The characteristics of its active vision make the system constitute a closed-loop control system based on image feedback, so that the matching vision algorithm has stronger environmental adaptability.

Description of drawings

Figure 1 Abstract schematic diagram of the basic structure of the biological visual system of the lizard-mimicking suborder Rescueidae

Figure 2 The general structure diagram of the extended structure of the biological visual system of the lizard-mimicking suborder Rescueidae

Figure 3 Schematic diagram of the image plane during the operation process of the angle information calibration learning algorithm

Figure 4 Schematic diagram of X, Y coordinate extraction

Figure 5 Schematic diagram of Z coordinate extraction

Figure 6 Flowchart of the overall steps

Figure 7 Flow chart of angle information calibration learning algorithm

Figure 8 Schematic diagram of the learning process of the angle information calibration learning algorithm

Figure 9 Flowchart of learning process of angle information calibration learning algorithm

Figure 10 Schematic diagram of the calibration information table

Figure 11 Depth information and world coordinates extraction algorithm flow chart

In the figure: 1-wide-angle fixed-focus camera, 2-vertical stepping motor, 3-horizontal stepping motor, 4-image acquisition processor, 5-motor controller.

Detailed ways

This example will be described in detail below with reference to FIG. 1 to FIG. 11 .

1. The hardware platform and basic working principle to realize the visual coordinate extraction algorithm of the lizard-like suborder evaderidae.

In this embodiment, the hardware platform for realizing the algorithm for extracting the biological visual coordinates of the imitation lizard suborder avoidant family is as follows:

As shown in Figure 1, this hardware platform is provided with two wide-angle fixed-focus cameras (1), a stepper motor (2) moving vertically, a stepping motor (3) moving horizontally, an image acquisition processor (4 ) and motor controller (5). Each wide-angle fixed-focus camera (1) is equipped with a stepping motor (2) moving vertically and a stepping motor (3) moving horizontally, the purpose of which is to imitate the vision of lizards The system, especially the mechanism of simulating the eye movement of living beings, has degrees of freedom in vertical and horizontal directions. Compared with the conventional vision system, this structure has distinct active vision characteristics, which helps to reduce the complexity of the algorithm. The reason for choosing a stepper motor is that it returns more accurate angle information than a servo.

The wide-angle fixed-focus camera (1) obtains external image information and sends it to the image acquisition processor (4) connected to it; the image acquisition processor (4) sends the control amount to the motor controller after processing the corresponding preset algorithm (5), the motor controller (5) drives the stepping motor (2) moving in the vertical direction and the stepping motor (3) moving in the horizontal direction to rotate, and then drives the wide-angle fixed-focus camera (1) to rotate.

Special attention is that the geometric center point of the wide-angle fixed-focus camera (1) is on the same line perpendicular to the horizontal plane as the geometric center point of the stepping motor (2) moving in the vertical direction and the stepping motor (3) moving in the horizontal direction. This feature ensures the smooth progress of the depth information and world coordinate extraction algorithm that will be introduced in detail later.

As shown in Figure 2, on this basic vision system, a telephoto zoom camera with horizontal and vertical 2-degree-of-freedom movement capabilities can be added above the midpoint of the connection between the two wide-angle fixed-focus cameras, and the telephoto The zoom camera is higher than the wide-angle fixed-focus camera, which improves the recognition ability of the extended vision system. The added telephoto zoom camera needs to have the movement capability of 180 degrees in the horizontal and vertical directions respectively to cooperate with the subsequent recognition steps. The telephoto zoom camera's long focal length and zoom function enable it to ensure the clarity of the target image to be recognized and the percentage of the telephoto zoom camera image, making full use of the camera's image acquisition capabilities. It mainly imitates the physiological structure of the retinal fovea in the human visual structure.

In order to effectively solve the problem that the target is easy to leave the common field of view of the vision system, a stepping motor moving vertically and a stepping motor moving horizontally can be added to the bottom of the aforementioned hardware platform. The power of the newly added motor is greater than that of the aforementioned stepping motor. When the target is about to leave the edge of the vision system image, the rotation angle of the newly added structure is adjusted to prevent the target from leaving the common field of vision of the vision system. This structure is mainly imitated from the neck structure of the lizards, which can assist the work of the entire visual system. The newly added telephoto zoom camera and two motors make the whole system have more obvious active visual features, which is more conducive to preventing the loss of targets during the operation of the algorithm; with clearer target image details, it is more conducive to the realization of recognition and Navigation algorithms.

2. Realization of the visual coordinate extraction algorithm of the imitation lizard suborder avoidance family, as shown in Figure 6:

1) Initialize the position of the wide-angle fixed-focus camera

First, let the two wide-angle fixed-focus cameras rotate to the preset initialization positions: even if the normal vectors of the image planes of the wide-angle fixed-focus cameras are parallel to the horizontal plane, and make the normal vectors of the image planes of the two wide-angle fixed-focus cameras perpendicular to the two wide-angle fixed-focus cameras The position of the line connecting the geometric center points of the focal camera;

2) Imitate the visual system of lizard suborder evasion family creatures, let two wide-angle fixed-focus cameras search for targets in different preset directions, and one of them will start the monocular visual tracking based on the Cam shift algorithm after searching for the target The algorithm keeps track of the target, and returns in real time the horizontal and vertical angles of the camera’s image plane normal vector relative to the camera’s initial position;

The Cam shift algorithm can provide the position of the center point of the target in the current image and the size of the target in the current image.

This bionic search method combined with a wide-angle fixed-focus camera can obtain a larger visual search range. At this time, the images collected by the two wide-angle fixed-focus cameras may not have a common field of view, and can be regarded as two sets of independent monocular vision video sequences during processing.

To illustrate the algorithm, it is assumed that the camera that first searches for the target is the wide-angle fixed-focus camera A.

Let the wide-angle fixed-focus camera B follow the wide-angle fixed-focus camera A to search for the target. After the camera B searches the target, it also enables the monocular vision tracking algorithm based on the Cam shift algorithm to keep track of the target;

In this embodiment, the process of the wide-angle fixed-focus camera B following the wide-angle fixed-focus camera A for target search is decomposed into vertical and horizontal searches. First, keep the vertical angle of the wide-angle fixed-focus camera B the same as that of the wide-angle fixed-focus camera A; then, the wide-angle fixed-focus camera B starts to search for the target in the horizontal direction. The horizontal search process is as follows:

In this embodiment, the stepper motor (3) moving in the horizontal direction provides the camera with a horizontal rotation angle of 180 degrees. In the world coordinate system, define the angle between the normal vector of the image plane of the wide-angle fixed-focus camera A and the positive direction of the Y-axis to be α, and the angle between the normal vector of the image plane of the wide-angle fixed-focus camera B and the positive direction of the Y-axis to be β. When 0°<α≤90°, the wide-angle fixed-focus camera B scans along the direction of increasing angle from β=0°; when 90°<α<180°, the wide-angle fixed-focus camera B starts from β=180° Start scanning along the direction of decreasing angle;

The described world coordinate system is: take the midpoint of the line connecting the geometric center points of the two wide-angle fixed-focus cameras as the origin, and take the line parallel to the horizontal plane, perpendicular to the line connecting the geometric center points of the two wide-angle fixed-focus cameras, and along the direction of visual tracking as The positive direction of the X-axis, the right-handed coordinate system with the direction perpendicular to the horizontal plane as the positive direction of the Z-axis;

When the wide-angle fixed-focus camera B searches for the target, it also enables the monocular vision tracking algorithm based on the Cam shift algorithm to keep track of the target.

When the tracking algorithm is enabled on the two wide-angle fixed-focus cameras, if the target continues to move, there may be a certain amount of tracking error and tracking lag. In order to eliminate the tracking residual error and tracking lag of the algorithm, the extended algorithm can be considered, and the tracking residual error and tracking lag can be eliminated by applying the PID algorithm. The specific method is: in the current image of the camera, the horizontal distance and vertical distance of the target relative to the center point of the current image of the camera are used as the feedback amount, and the PID control algorithm is applied to make the wide-angle fixed-focus camera track the target more accurately.

3) Invoke the angle information calibration learning algorithm to calculate in real time respectively the horizontal and vertical angles of the respective image plane normal vectors relative to the respective initial position camera image plane normal vectors after each camera A and B track the target;

4) Use the angle information to calibrate the results obtained by the learning algorithm, call the depth information and the world coordinate extraction algorithm, and calculate the target depth information and the world coordinates of the target in the world coordinate system in real time; the world coordinate system is: focus with two wide angles The midpoint of the line connecting the geometric center points of the cameras is the origin, the line parallel to the horizontal plane, perpendicular to the line connecting the geometric center points of the two wide-angle fixed-focus cameras, and the direction along the visual tracking is the positive direction of the X-axis, and the direction perpendicular to the horizontal plane is the upward direction. The right-handed coordinate system in the positive direction of the Z axis;

5) Output the target world coordinates. If the target is not lost, return to step 3); if the target is lost, return to step 2).

If the stepper motor described above is added at the bottom of the visual system used in the algorithm for extracting the visual coordinates of the lizard-like evasion family, then the monocular visual tracking based on the Cam shift algorithm will be enabled on the two wide-angle fixed-focus cameras After this step of the algorithm, if the target is about to move out of the common field of view of the two wide-angle fixed-focus cameras, the rotation angle of the newly added motor in the horizontal and vertical directions can be adjusted to keep the target within the range of the two wide-angle fixed-focus cameras. within the common field of view of the focal camera.

If the vision system applied in the algorithm for extracting the visual coordinates of the lizard-like suborder evaderidae biological visual coordinates extends the telephoto zoom camera described above, then on the basis of the target world coordinates obtained in step 4), coordinate transformation is carried out, and the world The origin of the coordinate system is moved to the geometric center point of the telephoto zoom camera, and the coordinates of the target in the new coordinate system are calculated. Rotating the camera according to this coordinate can make the center point of the target approximately coincide with the image center point of the telephoto zoom camera, enable the monocular vision tracking algorithm based on the Cam shift algorithm to keep the telephoto zoom camera tracking the target, and change the focal length of the telephoto zoom camera to use The percentage of the target in the entire telephoto zoom camera image is a preset percentage, and the recognition can begin.

2. Angle information calibration learning algorithm

The angle information calibration learning algorithm described in the visual coordinate extraction algorithm of the lizard-like suborder avoidance family can be described as follows:

Figure 3 shows the initial image collected by any wide-angle fixed-focus camera at the beginning of the angle information calibration learning algorithm. The intersection point of two dotted lines in the figure is the center point of camera imaging, with the center point as the origin, the horizontal dotted line is the LX axis, and the vertical dotted line is the LY axis to establish the camera image coordinate system. The upper sphere is the target of interest, its coordinates are (P, Q), the horizontal length of the entire image is marked as X, and the vertical length is marked as Y.

Prescribed flag bit: Let Marker_learning be the flag bit of the learning mode. When the value of Marker_learning is non-zero, the algorithm runs in learning mode. When the value of Marker_learning is 0, the algorithm does not run the learning mode.

The purpose of the algorithm: to output in real time the horizontal and vertical angles between the normal vector of the camera's head plane and the normal vector of the camera's face plane at its initial position during the process of the camera tracking the target.

The flowchart of the algorithm is shown in Figure 7.

The angle information calibration learning algorithm is an algorithm for the self-calibration behavior of a single camera. When describing the steps of the algorithm, it is assumed that the wide-angle fixed-focus camera A and its corresponding two stepping motors are discussed. But the algorithm is also applicable to the wide-angle fixed-focus camera B and its corresponding two stepping motors.

The algorithm steps are as follows:

1) Initialize the learning entrance angle (Δθ ₁ , Δη ₁ ), set Δθ ₁ = θ _ε , Δη ₁ = η _ε , where (θ _ε , η _ε ) is the threshold angle, and its specific size needs to be obtained through actual debugging according to the degree of camera distortion , such as (3°, 3°). The learning entry angle refers to the horizontal and vertical angles of the corresponding camera rotation when the target moves from the origin to the moving position in the camera image when the target moves during the calibration process. The learning entry angle is used to record the progress of the algorithm self-calibration in the angle information calibration learning algorithm. When (Δθ ₁ ≠θ _ε )ˇ(Δη ₁ ≠η _ε ), it means that the algorithm has stored a set of self-calibration breakpoints. When Δθ ₁ =θ _ε , Δη ₁ =η _ε , it means that the algorithm does not store self-calibration breakpoints.

2) Calculate the horizontal distance P and the vertical distance Q between the image center point and the target in the wide-angle fixed-focus camera image (Figure 3) relative to the main diagonal length of the image percentage of Among them, X and Y represent the length and width of the camera image respectively; when P is positive, it means that the target is located on the right of the center point of the image, and when P is negative, it means that the target is located on the left of the target center point. When Q is positive, it means that the target is located above the center point of the image, and when Q is negative, it means that the target is located below the center point of the target.

3) Judging whether the target is at the center of the camera image, that is, judging whether |f(P)| and |g(Q)| are both smaller than a preset threshold ε greater than 0.

If (|f(P)|<ε)∧(|g(Q)|<ε), the target is approximately coincident with the center point of the image, go to step 7),

If (|f(P)|＞ε)∨(|g(Q)|＞ε), the target does not coincide with the center point of the image, and the algorithm continues.

Among them, the threshold ε is used to determine whether the target has approximately coincided with the center point of the image. A larger threshold ε will make the algorithm run faster, but it will also reduce the accuracy of the algorithm. A smaller threshold ε will make the algorithm have higher accuracy, but it will also reduce the running speed of the algorithm. The specific size needs to be adjusted according to the actual angular accuracy and the real-time requirements of the vision system. Reference value: 1/50.

4) If the target does not coincide with the center point of the image, check the calibration information table and judge whether the corresponding |Δθ| value and |Δη| have been recorded in the calibration information table relative to |f(P)| and |g(Q)| value, if there is a corresponding |Δθ| value and |Δη| value, the algorithm jumps to step 8); if there is no corresponding |Δθ| value and |Δη| value, the algorithm continues.

The Δθ value refers to the vector starting from the geometric center of the camera and ending at the geometric center of the target in the world coordinate system The horizontal angle value between the projection on the horizontal plane and the projection of the normal vector of the camera image plane on the horizontal plane at this time. The Δη value refers to the vector starting from the center of the camera and ending at the center of the target of interest in the world coordinate system The vertical angle value between the projection on the plane perpendicular to the horizontal plane and the line connecting the center points of the two cameras and the projection of the normal vector of the camera on the plane perpendicular to the horizontal plane and the line connecting the center points of the two cameras.

It should be noted that the reason for judging in this step is whether there are corresponding |Δθ| The principle is similar to computer interrupt handling. If the target moves, save the learning entrance angle of the calibration breakpoint; jump out of the calibration learning process and return to the main algorithm; when the center point of the target coincides with the center point of the image again, and the target is still, call the learning entrance angle to return to the learning process. This means that the calibration information in the calibration information table may be incomplete when judging in this step, rather than completely existing or not existing at all. Therefore, when determining the calibration information, it is necessary to specifically determine whether the calibration information corresponding to each |f(P)| and |g(Q)| exists.

The benefits of adding a breakpoint return mechanism to the self-calibration process are obvious: 1. It enhances the adaptability of the algorithm, making the calibration process change from a continuous process to a discrete calibration process, that is, it is no longer required that the target be maintained first after the algorithm starts. Calibration is done at rest, but the discrete static state of the target is used for calibration during the running of the algorithm. 2. It enables the algorithm to run normally without waiting for the completion of the self-calibration process, which means that the output frequency of the angle information during the operation of the algorithm will become more and more real-time with the progress of the self-calibration process. 3. It enhances the intelligence of the algorithm.

5) Turn the camera step by step so that the target in the camera image coincides with the center point of the image

Concrete implementation: Gradually change the PWM wave duty cycle sent to the stepping motor (2) (3), so that the wide-angle fixed-focus camera (1) can reduce the distance |P| and |Q| between the center point of the image and the center point of the target direction of rotation. Among them, the change step size of the PWM wave duty cycle of the camera rotation control amount is divided into a change step size μ ₁ in the horizontal direction and a change step size μ ₂ in the vertical direction. The change step size is not a fixed value, but a value according to f (P) or g(Q), the calculation formulas are respectively μ ₁ =k*μ ₀ *f(P) and μ ₂ =k*μ ₀ *g(Q). Among them, k is a pre-set rate constant, μ ₀ is the unit change step size, f(P) or g(Q) can be positive or negative, so the change step size μ can also be positive or negative.

When increasing or decreasing the duty cycle of the PWM wave, if the stepper motor that controls the overall horizontal and vertical motion described above is added to the hardware system, then the normal vector of the image plane of the camera at this time is calculated relative to the normal vector of the camera at the initial position The horizontal angle θ and the vertical angle η. Calculate whether |θ| is close to the critical range of motion of the stepper motor (3), and calculate whether |η| is close to the critical range of motion of the stepper motor (2). If |θ|＞|θ _max |-|θ _ε2 |, where |θ _max | refers to the maximum motion angle of the stepper motor (3), and θ _ε2 is the threshold angle, which means a smaller stepper motor (3) For example, the motion angle of the stepper motor corresponding to the step size μ0 of 3 to 5 times the PWM wave unit change, then adjust the output angle of the newly added horizontal motion stepper motor before the algorithm continues, so that the entire visual system Move in the horizontal direction to reduce the size of |θ|, and expand the horizontal range of motion of the visual system applied by the visual coordinate extraction algorithm of the lizard-like suborder evacuidae. If |η|>|η _max |-|η _ε2 |, where |η _max | refers to the maximum motion angle of the stepper motor (2), and η _ε2 is the threshold angle, which means a smaller stepper motor (2) For example, the movement angle of the stepper motor corresponding to the step size μ ₀ of 3 to 5 times the PWM wave unit change. Then, before the algorithm continues further, adjust the output angle of the newly added vertical motion stepping motor, so that the entire visual system moves in the vertical direction to reduce the size of |η|, and expand the visual coordinates of the imitation lizards. Extracts the vertical range of motion of the vision system to which the algorithm is applied. When |θ|<|θ _max |-|θ _ε2 | and |η|<|η _max |-|η _ε2 |, the algorithm continues.

6) When the absolute values of f(P) and g(Q) are less than a preset threshold ε, that is (|f(P)|<ε)∧(|g(Q)|<ε), If it is determined that the center point of the target and the camera image approximately coincides, skip to step 7); otherwise, return to step 5).

7) When the target approximately coincides with the center point of the image, read out the horizontal angle θ and the vertical angle η of the normal vector of the image plane of the camera relative to the normal vector of the camera at the initial position of the camera and output it. Go to step 10).

8) If there is corresponding calibration information in the calibration information table, check the calibration information table according to |f(P)| value and |g(Q)| value to get the corresponding |Δθ| value and |Δη| value. According to the calibration information conversion diagram, determine the symbols attached before |Δθ| and |Δη| according to the image quadrant of the camera where the target is located, and determine the horizontal angle Δθ and vertical angle Δη of the camera rotation.

The calibration information table described here will be described in detail after the calibration principle section below, and the calibration information conversion table will be described in detail after the calibration information table section below.

If the stepper motor described above that controls the horizontal and vertical movement of the entire system is added to the hardware system, then calculate the horizontal angle θ and vertical angle between the normal vector of the image plane of the camera at this time and the normal vector of the camera at the initial position. Angle η. According to the quadrant of the wide-angle fixed-focus camera image where the target is currently located, the signs of Δθ and Δη are determined according to the calibration information conversion diagram, and whether |θ±Δθ| is close to the critical motion range of the stepping motor (3) is calculated, and |η± Whether Δη| is close to the critical range of motion of the stepper motor (2).

If |θ±Δθ|＞|θ _max |-|θ _ε2 |, where |θ _max | refers to the maximum motion angle of the stepping motor (3), and θ _ε2 is the threshold angle, which means a smaller stepping motor ( 3) the angle of motion, such as the angle of motion of the stepping motor corresponding to 3 to 5 times of PWM wave unit change step size μ ₀ , then adjust the output angle of the newly added horizontal motion stepping motor before the algorithm continues further, so that The entire visual system moves in the horizontal direction to reduce |θ±Δθ|, thereby expanding the horizontal range of motion of the visual system applied by the visual coordinates extraction algorithm of the lizard-like suborder escapidae. Because this method of expanding the range of motion changes the position of the target on the image, the calibration information in the calibration information table cannot be used directly, and it is necessary to go to step 4 to find the calibration information.

If |η ± Δη | > |η _max |-|η _ε2 |, where |η _max | refers to the maximum motion angle of the stepping motor (2), and η _ε2 is the threshold angle, which refers to a smaller stepping motor ( 2) the angle of motion, such as the angle of motion of the stepping motor corresponding to 3 to 5 times of PWM wave unit change step size μ ₀ , then adjust the output angle of the newly added vertical motion stepping motor before the algorithm further continues, Make the entire visual system move in the vertical direction to reduce the size of |η±Δη|, and expand the vertical motion range of the visual system applied by the visual coordinate extraction algorithm of the imitation lizards. Because this method of expanding the range of motion changes the position of the target on the image, the calibration information in the calibration information table cannot be used directly, and it is necessary to go to step 4 to find the calibration information.

If |θ±Δθ|<| _θmax |-| _θε2 | and |η±Δη|<| _ηmax |-| _ηε2 |, the algorithm continues.

After checking the calibration information table and the calibration information conversion table, send a PWM wave corresponding to the duty cycle of the camera rotation horizontal angle Δθ and vertical angle Δη to the stepping motor to control the wide-angle fixed-focus camera to make |f(P)| and |g(Q)| decreases.

9) In the world coordinate system, the target low-precision actual horizontal angle θ+Δθ and the low-precision actual vertical angle η+Δη at this time are output. Among them, θ and η are the horizontal and vertical angles between the normal vector of the image plane of the camera at this time and the normal vector of the camera at its initial position, respectively. Delay for T time, where T is the preset delay amount, return to step 2).

Since the target may move in real time during the operation of the algorithm, the control quantity of the stepping motor of the system also needs to be updated in real time. Therefore, after the PWM wave calculated here is sent, after a certain time T is delayed, the next step can be continued to update the new motor control value. The delay time T here needs to be debugged and set according to actual needs: a longer delay time T makes the calculation amount of the algorithm small but the system has a hysteresis, and a shorter delay time T makes the system more sensitive but the calculation amount of the algorithm is large.

10) Determine whether the Marker_learning flag is 0, and the value of this flag is preset. If the value of the Marker_learning flag is 0, it means that the calibration information table has been established, and go to step 2). If the value of the Marker_learning flag is not 0, continue the algorithm and start or continue to build the calibration information table.

Calibration principle

Figure 8 is a schematic diagram of the operation of the self-calibration process of the angle information calibration learning algorithm. The calibration process is as follows: Calibrate in the first quadrant of the camera image coordinate system, fix the vertical angle of the camera, and then perform horizontal calibration, then change the vertical angle of the camera, and perform horizontal calibration corresponding to the new vertical angle Calibrate and loop continuously until the target leaves the camera image in the vertical direction. When calibrating the pixels of the same row in the first quadrant, the camera moves horizontally, which means that the fluctuation of the g(Q) value of the target should be less than a threshold range, which needs to be determined during actual debugging.

Calibration needs to meet two conditions, first, the target is located at the center of the camera image, and second, the target is stationary. The first condition has been guaranteed in the previous steps, so in the next steps it is only necessary to judge whether the target is stationary or not. The judgment method is divided into two steps:

The first step is to determine whether the target is stationary during the horizontal calibration of the camera at a fixed vertical angle. The method of judging is: before starting to calibrate in the horizontal direction, first calculate g(Q) and assign it to g(Q ₀ ), that is, g(Q ₀ )=g(Q), and then start to calibrate in the horizontal direction, Calculate g(Q) in real time and judge whether |g(Q)-g(Q ₀ )|<ε ₂ is true. If true, it means that the target has not moved. If not, it means that the target has moved. ε ₂ is the reference threshold, which is a small positive value, and can be taken as 1/50 of the main diagonal length of the camera image. The setting of the threshold is related to the performance of the camera and the requirements of the experiment.

In the second step, after the camera completes the horizontal calibration of the previous row and continuously rotates to the initial calibration position in the horizontal direction of the next row, it is judged whether the target moves. The method of judgment is: assign g(Q ₀ ) corresponding to the previous row to g(Q ₁ ), then calculate g(Q) before the horizontal calibration of the next row starts, and assign it to g(Q ₀ ), that is, g(Q ₀ )=g(Q), judge |g(Q ₁ )-g(Q ₀ )|<ε ₃ . If the inequality is true, it means that there is no movement in the target, and if the inequality is not true, it means that the target has moved. _ε3 is the reference threshold, which is a positive value slightly larger than ε2, and can be taken as 1/30 of the length of the main diagonal, depending on the actual required accuracy.

Once the target moves during the calibration process, the calibration process stops immediately, and the corresponding learning entrance angle when the target moves is recorded. When the condition for triggering re-calibration is met, the unfinished calibration work is continued according to the learning entry angle.

The detailed flow chart of the program from step 11 to step 18 is shown in FIG. 9 .

11) Determine whether it is the first calibration

Define variables and assign values:

Standard horizontal angle θ ₀ : the horizontal angle between the normal vector of the image plane and the normal vector of the camera at its initial position when the camera starts to calibrate;

Standard vertical angle η ₀ : the vertical angle between the normal vector of the image plane and the normal vector of the camera at the initial position when the camera starts to calibrate;

Δθ, Δη, θ, η are defined as before;

The relationship between θ, Δθ, and θ ₀ can be expressed as: θ=θ ₀ +Δθ

η, Δη, and the relationship between η ₀ can be expressed as: η=η ₀ +Δη

Judgment method:

Judging whether the learning entrance angle (Δθ ₁ , Δη ₁ ) is (θ _ε , η _ε ), that is, whether (Δθ ₁ =θ _ε )^(Δη ₁ =η _ε ) holds true.

critical result:

If true, the algorithm proceeds to the next step. If the judgment is established, it means that the initialization value of the learning entrance angle has not been reassigned, that is, there was no interruption of the self-calibration process before, and the algorithm did not store the learning entrance angle when the calibration was interrupted. Next, it is necessary to start building the calibration information table;

If not, go to step 13). If the judgment is not established, it means that the calibration process is interrupted, and the learning entrance angle corresponding to the last calibration interruption is stored.

12) In the camera coordinate system, let the camera rotate (θ _ε , η _ε ) in the horizontal direction and vertical direction respectively, θ _ε , η _ε are the threshold angles, and calculate g(Q) after rotation; define the target static reference value g(Q ₀ ) and the alternative target stationary reference value g(Q ₁ ), and assign the value g(Q ₀ )=g(Q), g(Q ₁ )=0, and then jump to step 14). g(Q ₀ ) is used to judge whether the target is still during the calibration process of any row in the horizontal direction, and the alternative target static reference value g(Q ₁ ) is used to judge whether the camera has completed a row of calibration and is turning to prepare for the next row of calibration During the process, whether the target is stationary.

Specific method: Gradually change the duty cycle of the PWM wave sent to the vertically moving stepping motor, so that the camera continues to rotate downward. When the difference between the vertical angle η of the normal vector of the image plane of the camera relative to its initial position camera normal vector and the standard vertical angle η ₀ at this time is equal to the preset vertical learning threshold angle value | _ηε |, the camera Stop; calculate g(Q) at this time, and set the target static reference value g(Q ₀ )=g(Q), and the alternative target static reference value g(Q ₁ )=0. Gradually change the duty cycle of the PWM wave sent to the horizontally moving stepping motor, so that the camera continues to rotate to the left. When the difference between the horizontal angle θ of the normal vector of the image plane of the camera relative to its initial position and the standard horizontal angle θ ₀ is equal to the preset horizontal learning threshold angle value |θ _ε |, start to establish the corresponding At this time, the calibration information table of the g(Q) value jumps to step 14).

The reason for turning the threshold angle before establishing the calibration information table is that the angle information output through the calibration information table is coarse-precision angle information, and the angle that the algorithm needs to output is high-precision when the target and the center point of the image are approximately coincident. angle information. Therefore, when the target is about to coincide with the center point of the image, the calibration information table is no longer used to guide the motor to rotate to obtain high-precision angle information. Therefore, the algorithm does not need the camera calibration information within the threshold angle range, that is, the algorithm does not calibrate the shaded area in Figure 8.

13) At this time, the algorithm will skip the self-calibration process before learning the entrance angle, and directly restore the breakpoint progress of the last self-calibration process;

Gradually change the duty cycle of the PWM slope sent to the vertically moving stepper motor. In the camera coordinate system, first make the camera continue to rotate downward until the angle of the camera’s downward rotation is equal to the learning entrance angle in the vertical direction Δη ₁ , the camera stops. Calculate g(Q) at this time, let the alternative target static reference value g(Q ₁ )=g(Q ₀ ), and set the target static reference value g(Q ₀ )=g(Q)

Gradually change the duty cycle of the PWM wave sent to the horizontally moving stepper motor. In the camera coordinate system, the camera is continuously turned to the left until the angle the camera turns to the left is equal to the learning entrance angle Δθ ₁ in the horizontal direction. Continue to the next step.

14) If g(Q ₁ ) is equal to zero, calculate the f(P) value at this time, and record the Δθ=θ _ε and Δη=η _ε values in the calibration information table corresponding to the f(P) value and In the position of g(Q) value, continue to the next step.

If g(Q1) is not equal to zero, it is judged whether the absolute value of the difference between the alternative target static reference value g(Q ₁ ) and the target static reference value g(Q ₀ ) is smaller than the threshold, that is |g(Q ₁ )-g( Q ₀ )|<ε ₃ . If the inequality is established, calculate the f(P) value at this time, and record the Δθ and Δη values at this time into the calibration information table corresponding to the f(P) value and g(Q) value at this time. The algorithm continues to the next step; if the inequality is not true, record the Δθ and Δη at this time, and set the learning entrance angle Δθ ₁ =Δθ, Δη ₁ =Δη; skip to step 2).

15) In the camera coordinate system, turn the camera to the left by a horizontal angle corresponding to the PWM wave duty cycle unit change step size μ0, and judge whether the geometric center of the target is away from the right side of the camera image at this time; if it has left, turn to Go to step 17); if not leave, then continue the algorithm;

16) Calculate the value of g(Q) at this time, judge whether |g(Q)-g(Q0)|<ε ₂ is true at this time, if it is true, calculate the value of f(P) at this time, and set the value of f(P ), record the Δθ and Δη values corresponding to g(Q) into the calibration information table, and return to step 15); otherwise, record the Δθ and Δη at this time, let the learning entrance angle Δθ ₁ = Δθ, Δη ₁ = Δη; skip Go to step 2);

In step 16) _, it _will be recorded into Calibration information table and return to continue to obtain new calibration information. Otherwise, it can be determined that the target has moved, and the self-calibration process needs to be terminated after saving the breakpoint.

17) In the camera coordinate system, make the camera rotate to the right to Δθ=| _θε |, and then make the camera rotate downwards to a vertical angle corresponding to the PWM wave duty cycle unit change step size _μ0 ; judge at this time Whether the geometric center point of the target leaves the camera image; if it has left, it means that the calibration information table has been established, and go to step 18); if not, calculate g(Q) at this time, and update the static reference value of the alternative target g(Q ₁ )=g(Q ₀ ), then update the target stationary reference value g(Q ₀ )=g(Q), go to step 14);

The tracking algorithm based on the Cam shift algorithm used in the angle information calibration learning algorithm can provide the position of the center point of the target in the camera image and the size of the target in the camera image for the angle information calibration learning algorithm. With the help of these two pieces of information, it can be judged whether the geometric center point of the target has left the camera image. The specific implementation is: when f(P)> _ε4 or g(Q)> _ε5 , it can be judged that the target has left webcam image. The real situation at this time is that the center point of the target is close to the edge of the image. ε ₄ and ε ₅ are threshold values, and the value of ε ₄ in this embodiment is Slightly smaller, the value of ε ₅ is more than slightly smaller.

18) After the calibration information table is established, clear the Marker_learning flag to 0, and go to step 2);

Only when it is confirmed that the self-calibration process of the angle information calibration learning algorithm has been completed, the algorithm will go to step 18, and clear the learning flag Marker_learning to 0. After the learning flag Marker_learning is cleared, no matter whether the algorithm stores a self-calibration process breakpoint or not, it will no longer enter the self-calibration state.

Calibration Information Sheet

As shown in FIG. 10 , the calibration information table mentioned in the angle information calibration learning algorithm is a two-dimensional data table in which camera imaging calibration information is recorded. The table consists of |f(P)|, |g(Q)| at any point in the camera image coordinate system, and the horizontal angle and vertical angle of the camera that moves the target from the origin to the point in the camera coordinate system. angle composition. That is, each position in the table records a set of |Δθ|, |Δη| values corresponding to a set of |f(P)|, |g(Q)| values. The data in each row corresponds to the same |f(P)| value, and the values from the leftmost to the rightmost row correspond to |g(Q)| values that gradually increase in value. The data in each column corresponds to the same |g(Q)| value, and the values from the top to the bottom of the column correspond to |f(P)| values that increase in value gradually. For example: Figure 10 corresponds to |f(P)|=0.25, |g(Q)|=0.33, and a set of Δθ, Δη data is recorded in the table: Δθ=15°, Δη=24.3°.

Since the lens distortion of the used camera is approximately symmetrical, only the calibration information of the I-quadrant image in the camera image is recorded in the calibration information table. The calibration information of images in other quadrants can be obtained through the calculation shown in the calibration information conversion schematic table.

Calibration Information Conversion Table

Calibration information conversion table: The calibration information conversion table assists the work of the calibration information table.

First, the camera image is divided into 4 regions according to the quadrant:

Where imax is the number of lines of the camera image. jmax is the number of columns of the camera image. i refers to the i-th row of the camera image. j refers to the jth column of the camera image. I _ij refers to the pixel point in row i and column j in the camera image.

The calibration information table only records the calibration information of the camera images in the I quadrant, and the calibration information of the camera images in other quadrants can be calculated by following the rules of the calibration information conversion table. The calibration information conversion diagram is as follows: the second column in the table indicates the judgment method for judging different quadrants, and the third and fourth columns indicate the angle and direction that the camera should rotate in the camera coordinate system when it is located in a different quadrant.

I quadrant IF: (f(P)=|f(P)|)∧(g(Q)=|g(Q)|) |Δθ| |Δη| II quadrant IF: (-f(P)＝|f(P)|)∧(g(Q)＝|g(Q)|) -|Δθ| |Δη| III quadrant IF: (-f(P)＝|f(P)|)∧(-g(Q)＝|g(Q)|) -|Δθ| -|Δη| IV quadrant IF: (f(P)＝|f(P)|)∧(-g(Q)＝|g(Q)|) |Δθ| |Δη|

The necessity of self-calibration for the angle information calibration learning algorithm: 1. If the angle information calibration learning algorithm does not perform self-calibration during operation, it can only output the angle information when the target and the center point of the image are approximately coincident, which will As a result, the output of angle information is not real-time, which is not conducive to the use of subsequent algorithms. 2. If the angle information calibration learning algorithm does not perform self-calibration during operation, its ability to track the target will decrease, making the target more likely to be lost.

3. Depth information and world coordinate extraction algorithm

Depth information and world coordinates extraction algorithm steps are as follows:

1) Calibrate the visual system used in the visual coordinate extraction algorithm of the lizard-mimicking suborder evaderidae. After manual calibration, the distance d between the respective geometric centers of the two wide-angle fixed-focus cameras is a known quantity. As shown in Figure 4, in order to describe the calculation process of the algorithm in detail, it is assumed that the included angles between the ray where the plane normal vectors of the two wide-angle fixed-focus cameras are located and the respective geometric center points of the two wide-angle fixed-focus cameras are all acute angles. Let d ₁ be the distance between the geometric center point of the left wide-angle fixed-focus camera and the projection point of the target on the line connecting the respective geometric center points of the two wide-angle fixed-focus cameras. The distance between the projection points on the line connecting the geometric centers of the fixed-focus cameras is d ₂ , then d ₁ +d ₂ =d.

2) The horizontal angles θ ₁ , θ ₂ and the vertical angles η ₁ , η of the respective image plane normal vectors of the two wide-angle fixed-focus cameras received from the angle information calibration learning algorithm relative to the respective initial position camera normal vectors ₂ . Among the angle information from the angle information calibration learning algorithm, there is the angle information obtained by querying the calibration information table, and the angle information obtained by reading the reverse calculation of the control quantity. The former has low precision and high real-time performance, while the latter has high precision and low real-time performance. According to specific needs, the depth information and world coordinate extraction algorithm can choose to shield one type of angle information or receive all of them.

3) According to the obtained calibration value d and angle information θ ₁ and θ ₂ , calculate the depth of the target relative to the midpoint of the line between the respective geometric center points of the two wide-angle fixed-focus cameras, that is, the X coordinate of the target.

Compared with Figure 4, the depth information of the target can be expressed as:

Deduced: $d_1=\frac{\mathrm{the\; tan}{\mathrm{\&theta;}}_1}{\mathrm{the\; tan}{\mathrm{\&theta;}}_2}{d}_2$

Substituting: d ₁ +d ₂ =d, we can get: $(\frac{\mathrm{the\; tan}{\mathrm{\&theta;}}_1}{\mathrm{the\; tan}{\mathrm{\&theta;}}_2}+1)d_2=d$

Organized: $d_2=\frac{d}{(\frac{\mathrm{the\; tan}{\mathrm{\&theta;}}_1}{\mathrm{the\; tan}{\mathrm{\&theta;}}_2}+1)},$ $d_1=d-d_2=d-\frac{d}{(\frac{\mathrm{the\; tan}{\mathrm{\&theta;}}_1}{\mathrm{the\; tan}{\mathrm{\&theta;}}_2}+1)}$

corresponding: $x=\frac{d_2}{\mathrm{the\; tan}{\mathrm{\&theta;}}_2}=\frac{d_1}{\mathrm{the\; tan}{\mathrm{\&theta;}}_1}=\frac{d}{(\frac{\mathrm{the\; tan}{\mathrm{\&theta;}}_1}{\mathrm{the\; tan}{\mathrm{\&theta;}}_2}+1)\mathrm{the\; tan}{\mathrm{\&theta;}}_2}=\frac{d}{\mathrm{the\; tan}{\mathrm{\&theta;}}_1+\mathrm{the\; tan}{\mathrm{\&theta;}}_2}$

That is, the depth information and the X coordinate are: $x=\frac{d}{\mathrm{the\; tan}{\mathrm{\&theta;}}_1+\mathrm{the\; tan}{\mathrm{\&theta;}}_2}.$

4) Further calculated It is the Y coordinate of the system world coordinate system. In step 1), only the angles between the lines between the ray where the plane normal vectors of the heads of the two wide-angle fixed-focus cameras are located and the respective geometric center points of the two wide-angle fixed-focus cameras as shown in Figure 4 are all acute angles Condition. If it is an obtuse angle, the calculation method is similar to the above case.

The Z coordinate of the world coordinate system can also be obtained by a method similar to that of obtaining the Y coordinate. As shown in Figure 5, after the depth information X is obtained, the vertical angle η ₁ of the normal vector of the camera head plane obtained by combining the angle information calibration learning algorithm with respect to the normal vector of the camera at the initial position, through the formula: Z=tan(η ₁ ) ×X, you can get the Z coordinate. So far, the world coordinates of the target relative to the visual system coordinate system have been established.

5) Output the world coordinates of the midpoint of the target relative to the respective geometric center points of the two wide-angle fixed-focus cameras, and exit the algorithm.

Its algorithm flowchart is shown in Figure 11.

The reasons for separating the depth information and world coordinate extraction algorithm from the angle information calibration learning algorithm are as follows: 1. The angle information calibration learning algorithm needs to continuously update the angle information of the target, and its real-time requirements are high. So the brevity of the algorithm is very important. 2. Angle information calibration The angle information output by the learning algorithm is not only provided to the depth information and world coordinate extraction algorithm, but also to other algorithms.

4. Monocular tracking algorithm based on Camshifi algorithm

The angle information calibration learning algorithm uses the monocular tracking algorithm based on the Cam shift algorithm, which is mainly used to provide the angle information calibration learning algorithm with the size of the target in the camera image and the position of the geometric center of the target in the camera image in real time.

Conventional multi-eye vision system tracking algorithms need to comprehensively consider the coordination of multiple cameras during tracking, which requires a large amount of calculation and complex algorithms. Moreover, because they do not have the function of active vision, the tracking range is narrow and the target is prone to loss. The vision system used in the active vision-based algorithm for extracting the visual coordinates of the lizard-like evasionidae organisms separates the tracking algorithms of each camera, which effectively reduces the complexity of the algorithm and reduces the amount of calculation of the algorithm.

The monocular visual tracking algorithm based on the Cam shift algorithm belongs to the prior art, which can be summarized as follows:

Change the HSV color space of the image returned by the camera, extract the hue component, and construct a color histogram and color probability distribution map. According to the color probability distribution map, the Cam_shift algorithm is used to track the target. Establish an appropriate image search window, and extract the coordinates of the center point of the search window. The calculated control amount is sent to the stepping motor to control the camera so that the tracking target is always at the center of the image. The specific control method is the control method described in the angle information calibration learning algorithm.

Claims (2)

1. an imitative Lacertilia Chamaeleontidae biological vision coordinate extraction algorithm, based on by two wide-angle cameras with fixed focus, stepper motor

The physical platform forming, is characterized in that comprising the following steps:

1). make two wide-angle cameras with fixed focus search for respectively target, when A camera is found after target, enable monocular vision track algorithm based on Cam shift algorithm and keep the tracking to target, and while returning in real time this time image planar process vector of A camera with respect to A camera initial position camera as the horizontal sextant angle of planar process vector and vertical angle; First described A camera for to search the camera of target, and B camera is another camera; Described initial position is: make being parallel to each other with surface level as planar process vector of wide-angle cameras with fixed focus, and make the position of the line segment forming perpendicular to two wide-angle cameras with fixed focus geometric center point lines as planar process vector of two wide-angle cameras with fixed focus;

2) .B camera is followed A camera and is carried out target search, and B camera searches after target, and the monocular vision track algorithm of also enabling based on Cam shift algorithm keeps the tracking to target;

3). call angle information and demarcate that learning algorithm calculates respectively A in real time, B camera all traces into after target, picture planar process vector separately with respect to initial position camera separately as horizontal sextant angle and the vertical angle of planar process vector;

Wherein angle information demarcation learning algorithm comprises the following steps:

3.1) initialization study inlet angle Δ θ ₁, Δ η ₁,

Described study inlet angle refers to when in calibration process, target moves, and in camera image, target moves to the position moving from initial point, the level angle that corresponding camera rotates and vertically angle;

3.2) calculate respectively in camera image the horizontal range P of camera image central point and target and vertically number percent f (P) and the g (Q) with respect to camera image principal diagonal length apart from Q;

3.3) judge that whether target is at camera image central point,

If (| f (P) | < ε) ∧ (| g (Q) | < ε), threshold epsilon >0, target, at central point, jumps to step 3.7);

If (| f (P) | > ε) ∨ (| g (Q) | > ε), threshold epsilon >0, target, not at central point, jumps to step 3.4);

3.4) look into demarcation information table,

If exist in demarcation information table corresponding to | f (P) | and | g (Q) | angle information, algorithm jumps to step 3.8);

If do not exist in demarcation information table corresponding to | f (P) | and | g (Q) | angle information, jump to step 3.5);

3.5) progressively rotate camera, target is overlapped with image center in camera image;

3.6) when target overlaps with camera image central point, jump to step 3.7); Otherwise return to step 3.5);

3.7) when target overlaps with image center, read camera picture planar process vector now and also export as the horizontal sextant angle θ of planar process vector and vertical angle η with respect to this camera initial position camera, go to step 3.10);

3.8) if demarcate, in information table, exist corresponding to | f (P) | and | g (Q) | angle information, camera rotates this angle target geometric center point is overlapped with image center in camera image, and rotation direction is determined according to the demarcation information signal table that converts;

3.9), under output world coordinate system, target is the horizontal sextant angle and vertical angle as planar process vector with respect to initial position camera; The time delay T time, wherein T is the amount of delay setting in advance, and returns to step 3.2);

3.10) by default study zone bit, judge whether to learn complete,

If learn completely, go to step 3.2);

If do not learn completely, algorithm continues, and starts to demarcate;

3.11) judge whether it is to demarcate for the first time,

If study inlet angle equals initial value, for demarcating for the first time, now algorithm continues;

If study inlet angle is not equal to initial value,, for demarcating for the first time, now do not jump to step 3.13);

3.12) make camera rotate respectively θ with vertical direction in the horizontal direction _ε, η _ε, θ _ε, η _εfor threshold angle, jump to step 3.14),

3.13) camera turns over the study inlet angle of demarcating record while interrupting for last time, jumps to step 3.14);

3.14) if demarcate for the first time, the threshold angle θ that camera is rotated respectively with vertical direction in the horizontal direction _ε, η _εand the camera image internal object geometric center point corresponding with it | f (P) | and | g (Q) | write demarcation information table;

If not demarcate for the first time, first judge whether target motion has occurred before entering this step; If target travel, renewal learning inlet angle, and jump to step 2.2); If target is motion no, upgrade and demarcate information table;

3.15) keep the vertical angle of camera constant, carry out rower and determine, whenever camera turns over a unit level angle in the horizontal direction, judge now whether target geometric center point leaves camera image; If leave, go to step 3.17); If do not leave, continue algorithm;

3.16) judgement camera often turns over after a unit level angle in the horizontal direction, and whether target moves, if be not moved, upgrade and demarcates information table, and jump to step 3.15); If be moved, renewal learning inlet angle, and jump to step 3.2);

3.17) change the vertical angle of camera, carry out the level of another row and demarcate, if target geometric center point is left camera image, demarcate information table and set up completely, go to step 3.18); If do not leave, go to step 3.14);

3.18) after demarcating information table foundation, remove study zone bit, go to step 3.2);

Described demarcation information table is to record the bivector table that camera imaging is demarcated information; In table, recorded target position information, and in camera image, target moves to target location from initial point, the horizontal sextant angle that camera turns over as planar process vector correspondence and vertically angle; Described target position information is by the horizontal range P of camera image central point and target and vertically number percent f (P) and g (Q) expression with respect to camera image principal diagonal length apart from Q;

Wherein 3.8) described demarcation information conversion signal table can be described as:

First camera image is divided into 4 regions according to place quadrant:

The line number that wherein imax is camera image; Jmax is the columns of camera image; I refers to that camera image i is capable; J refers to camera image j row; I _ijthe pixel that refers to the capable j row of i in camera image;

Demarcation information table only records the demarcation information of I quadrant camera image, and the demarcation information of other quadrant camera image can calculate by following the rule of demarcation information conversion signal table; Demarcation information converts and illustrates that table is as follows:

I quadrant IF:(f(P)＝|f(P)|)∧(g(Q)＝|g(Q)|) |Δθ| |Δη| II quadrant IF:(-f(P)＝|f(P)|)∧(g(Q)＝|g(Q)|) -|Δθ| |Δη| III quadrant IF:(-f(P)＝|f(P)|)∧(-g(Q)＝|g(Q)|) -|Δθ| -|Δη| IV quadrant IF:(f(P)＝|f(P)|)∧(-g(Q)＝|g(Q)|) |Δθ| |Δη|

In table, secondary series represents to judge the determination methods of different quadrants, and third and fourth list is shown while being positioned at different quadrant, angle and direction that in camera coordinate system, camera rotates;

4). according to step 3) result that obtains, use depth information and world coordinates extraction algorithm, calculate in real time and export target depth information and the world coordinates of target in world coordinate system; Described world coordinates is: the mid point of two wide-angle cameras with fixed focus geometric center point lines of take is initial point, take parallel with surface level, perpendicular to two wide-angle cameras with fixed focus geometric center point lines and be x axle positive dirction along the direction of visual pursuit, the right-handed coordinate system that the direction that makes progress perpendicular to surface level of take is z axle positive dirction;

5) if. target is not lost, and returns to step 3), if track rejection returns to step 1).

2. a kind of imitative Lacertilia Chamaeleontidae biological vision coordinate extraction algorithm according to claim 1, is characterized in that described depth information and world coordinates extraction algorithm comprise the following steps:

According to the distance between two wide-angle cameras with fixed focus geometric center point, and according to angle information demarcate two wide-angle cameras with fixed focus that learning algorithm obtains now picture planar process vector separately with respect to initial position camera separately horizontal sextant angle and the vertical angle as planar process vector, utilize trigonometric function to calculate the world coordinates of target.