CN114143421B - Dual-sensor camera system and calibration method thereof - Google Patents

Abstract

A dual-sensor camera system and a calibration method thereof. The dual-sensor camera system includes at least one color sensor, at least one infrared sensor, a storage device, and a processor. The processor is configured to load and execute a computer program stored in the storage device to: control the color sensor and the infrared sensor to respectively acquire multiple color images and multiple infrared images of a camera scene using multiple shooting conditions; calculate multiple color image parameters of the color image acquired under each shooting condition and multiple infrared image parameters of the infrared image acquired under each shooting condition to calculate the difference between the brightness of the color image and the brightness of the infrared image; and determine exposure settings suitable for the color sensor and the infrared sensor according to the calculated difference.

Description

Dual sensor camera system and calibration method thereof

Technical Field

The present disclosure relates to a camera system and method, and more particularly to a dual-sensor camera system and a calibration method thereof.

Background technique

The camera's exposure conditions (including aperture, shutter speed, and brightness) will affect the quality of the captured image, so many cameras will automatically adjust the exposure conditions during the image capture process to obtain a clear and bright image. However, in high-contrast scenes such as low light or backlight, the camera's adjustment of the exposure conditions may result in excessive noise or overexposure in some areas, and the image quality of all areas cannot be taken into account.

In this regard, current technology has adopted a new image sensor architecture, which utilizes the high light sensitivity of infrared (IR) sensors to intersperse IR pixels in the color pixels of the image sensor to assist in brightness detection. For example, FIG1 is a schematic diagram of an existing image sensor for acquiring an image. Referring to FIG1 , in addition to being configured with red (R), green (G), blue (B) and other color pixels, the existing image sensor 10 is also interspersed with infrared (I) pixels. Therefore, the image sensor 10 can combine the color information 12 obtained by the R, G, B color pixels with the brightness information 14 obtained by the I pixel to obtain an image 16 with moderate color and brightness.

However, under the above-mentioned single image sensor architecture, the exposure conditions of each pixel in the image sensor are the same, so only the exposure conditions that are more suitable for color pixels or infrared pixels can be selected to acquire images. As a result, the characteristics of the two pixels cannot be effectively utilized to improve the image quality of the acquired image.

Summary of the invention

The present invention provides a dual-sensor camera system and a calibration method thereof, which utilizes independently configured color and infrared sensors to respectively acquire multiple images under different shooting conditions, and performs image alignment and brightness matching based on the images, and applies the results to subsequently acquired images, thereby improving the image quality of the acquired images.

The dual sensor camera system of the present invention comprises at least one color sensor, at least one infrared sensor, a storage device, and a processor coupled to the color sensor, the infrared light sensor, and the storage device. The processor is configured to load and execute a computer program stored in the storage device to: control the color sensor and the infrared sensor to respectively acquire multiple color images and multiple infrared images of a camera scene using multiple shooting conditions; calculate multiple color image parameters of the color image acquired under each shooting condition and multiple infrared image parameters of the infrared image acquired under each shooting condition to calculate the difference between the brightness of the color image and the brightness of the infrared image; and determine exposure settings suitable for the color sensor and the infrared sensor according to the calculated difference.

The dual-sensor camera system calibration method of the present invention is applicable to a dual-sensor camera system including at least one color sensor, at least one infrared sensor and a processor. The method includes the following steps: controlling the color sensor and the infrared sensor to respectively acquire multiple color images and multiple infrared images of a camera scene using multiple shooting conditions; calculating multiple color image parameters of the color images acquired under each shooting condition and multiple infrared image parameters of the infrared images acquired under each shooting condition to calculate the difference between the brightness of the color image and the brightness of the infrared image; and determining exposure settings suitable for the color sensor and the infrared sensor based on the calculated difference.

Based on the above, the dual-sensor camera system and calibration method thereof of the present invention adopt different shooting conditions on independently configured color sensors and infrared sensors to acquire multiple images, and determine the exposure and alignment settings suitable for the color sensor and infrared sensor according to the positional relationship of corresponding pixels in these images and the brightness difference between these images, so as to perform image alignment and brightness matching on subsequently acquired images, thereby improving the image quality of the captured images.

In order to make the present disclosure more clearly understood, embodiments are specifically cited below and described in detail with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a conventional method of acquiring an image using an image sensor;

FIG2 is a schematic diagram of using an image sensor to acquire an image according to an embodiment of the present invention;

FIG3 is a block diagram of a dual-sensor camera system according to an embodiment of the present invention;

FIG4 is a flow chart of a calibration method for a dual-sensor camera system according to an embodiment of the present invention;

FIG5 is a flow chart of an alignment and calibration method of a dual-sensor camera system according to an embodiment of the present invention;

FIG6 is a flow chart of a brightness matching calibration method for a dual-sensor camera system according to an embodiment of the present invention;

FIG. 7 is a flow chart of a calibration method for a dual-sensor camera system according to an embodiment of the present invention.

Symbol Description

10, 20: Image sensor

12: Color information

14: Brightness information

16: Image

22: Color sensor

22a: Color Image

24: Infrared sensor

24a: Infrared image

26: Scene Image

30: Dual sensor camera system

32: Color sensor

34: Infrared sensor

36: Storage device

38: Processor

R, G, B, I: pixels

S402~S406, S502~S506, S602~S606, S702~S706: Steps

Detailed ways

FIG2 is a schematic diagram of using an image sensor to obtain an image according to an embodiment of the present invention. Referring to FIG2, the image sensor 20 of the embodiment of the present invention adopts a dual sensor architecture of independently configuring a color sensor 22 and an infrared (IR) sensor 24, and uses the respective characteristics of the color sensor 22 and the infrared sensor 24 to respectively obtain multiple images using multiple exposure conditions suitable for the current camera scene, and selects a color image 22a and an infrared image 24a with appropriate exposure conditions, and uses the infrared image 24a to supplement the texture details lacking in the color image 22a through image fusion, thereby obtaining a scene image 26 with good color and texture details.

FIG3 is a block diagram of a dual sensor camera system according to an embodiment of the present invention. Referring to FIG3 , the dual sensor camera system 30 of this embodiment can be configured in electronic devices such as mobile phones, tablet computers, laptop computers, navigation devices, driving recorders, digital cameras, digital video cameras, etc. to provide camera functions. The dual sensor camera system 30 includes at least one color sensor 32, at least one infrared sensor 34, a storage device 36, and a processor 38, and its functions are described as follows:

The color sensor 32 includes, for example, a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS) device, or other types of photosensitive devices, and can sense light intensity to generate an image of the camera scene. The color sensor 32 is, for example, a red, green, and blue (RGB) image sensor, which includes red (R), green (G), and blue (B) color pixels to obtain color information such as red light, green light, and blue light in the camera scene, and synthesize these color information to generate a color image of the camera scene.

The infrared sensor 34 includes, for example, a CCD, a CMOS device, or other types of photosensitive devices, which can sense infrared light by adjusting the wavelength sensing range of the photosensitive device. The infrared sensor 34 uses, for example, the above-mentioned photosensitive devices as pixels to obtain infrared light information in the camera scene, and synthesizes the infrared light information to generate an infrared image of the camera scene.

The storage device 36 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk or similar device or a combination of the above devices, and is used to store computer programs executable by the processor 38. In some embodiments, the storage device 36 can also store, for example, color images acquired by the color sensor 32 and infrared images acquired by the infrared sensor 34.

The processor 38 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessor, microcontroller, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), programmable logic device (PLD) or other similar devices or combinations of these devices, and the present invention is not limited thereto. In this embodiment, the processor 38 can load a computer program from the storage device 36 to execute the calibration method of the dual-sensor camera system of the embodiment of the present invention.

Based on the different characteristics (e.g., resolution, wavelength range, field of view (FOV)) of the color sensor 32 and the infrared sensor 34, an embodiment of the present invention provides a calibration method, which can calibrate the color sensor 32 and the infrared sensor 34 mounted on the dual-sensor camera system 30 in the production stage to balance the differences between the color sensor 32 and the infrared sensor 34 under different shooting conditions. The calibration result will be stored in the storage device 36 and can be used as a basis for adjusting the acquired image in the subsequent run stage.

FIG4 is a flow chart of a calibration method for a dual sensor camera system according to an embodiment of the present invention. Please refer to FIG3 and FIG4 simultaneously. The method of this embodiment is applicable to the dual sensor camera system 30 described above, and is suitable for calibrating the color sensor 32 and the infrared sensor 34 of the dual sensor camera system 30 during the production stage. The following is a detailed description of the calibration method of this embodiment with various components of the dual sensor camera system 30.

In step S402, at least one color sensor 32 and at least one infrared sensor 34 are assembled in the dual sensor camera system 30. The color sensor 32 and the infrared sensor 34 are assembled in the image sensor by a robot, for example, the color sensor 22 and the infrared sensor 24 assembled in the image sensor 20 shown in FIG.

In step S404, the processor 38 performs alignment calibration between the color sensor 32 and the infrared sensor 34. The processor 38 may perform an image alignment algorithm such as brute force, optical flow, homography, or local warping to align the color image acquired by the color sensor 32 with the infrared image acquired by the infrared sensor 34, and its detailed implementation will be described below.

In step S406, the processor 38 performs calibration for brightness matching of the color sensor 32 and the infrared sensor 34 under different shooting conditions. The processor 38, for example, calculates the difference between the color image and the infrared image acquired under different shooting conditions, and determines exposure settings suitable for the color sensor 32 and the infrared sensor 34 accordingly, and its detailed implementation will be described later.

For the above alignment calibration, FIG5 is a flow chart of an alignment calibration method of a dual sensor camera system according to an embodiment of the present invention. Please refer to FIG3 and FIG5 simultaneously. The method of this embodiment is applicable to the above dual sensor camera system 30. The following is a detailed description of the alignment calibration method of this embodiment with various components of the dual sensor camera system 30.

In step S502, the processor 38 controls the color sensor 32 and the infrared sensor 34 to respectively capture a test chart having a special pattern to obtain a color test image and an infrared test image. The special pattern may be a black and white chessboard pattern, or other patterns that can clearly distinguish features, which are not limited here.

In step S504 , the processor 38 detects a plurality of feature points of the special pattern in the color test image and the infrared test image.

In some embodiments, the processor 38, for example, cuts each color test image and infrared test image into multiple blocks, and executes a feature detection algorithm to detect at least one feature point in each block. The processor 38, for example, determines the number of feature points to be detected from each block based on its own computing power, and the feature detection algorithm is, for example, Harris corner detection. In some embodiments, the processor 38, for example, selects edge pixels in each block, or pixels with high local deviation in each block as the detected feature points, which are not limited here.

In step S506, the processor 38 executes an image alignment algorithm to calculate the matching relationship between the color test image and the infrared test image according to the positional relationship between the corresponding feature points in the color test image and the infrared test image, so as to align the color image and the infrared image acquired subsequently. When performing the image alignment, the processor 38, for example, obtains all feature points in the color image and all feature points in the infrared image, and then executes the image alignment algorithm for these feature points.

In some embodiments, when the captured scene is a planar scene, the processor 38 moves a patch including a plurality of pixels at a corresponding position in the infrared test image for a designated feature point among the feature points detected from the color test image, to search for a corresponding feature point in the infrared test image corresponding to the designated feature point. The processor 38, for example, moves the patch around the pixel corresponding to the designated feature point in the infrared test image as the center, and compares the pixels in the patch with the pixels around the designated feature point in the color test image until the pixels in the patch match the pixels around the designated feature point (for example, the sum of the differences in the pixel values of all pixels is less than a predetermined threshold value). Finally, the processor 38 can determine the center point pixel of the patch at the location where the match is achieved as the corresponding feature point corresponding to the designated feature point. The processor 38 will repeat the above matching action until the correspondence between all feature points is obtained.

Afterwards, the processor 38 executes, for example, a RANSAC (random sampling consensus) algorithm to establish a homography transformation matrix, as follows:

Wherein, (x, y) represents the position of the designated feature point in the color test image, (x', y') represents the position of the corresponding feature point in the infrared test image, and a-h represent variables. The processor 38, for example, brings the positions of each feature point in the color test image and the corresponding feature point in the infrared test image into the above-mentioned homography transformation matrix for solution, and uses the obtained solution as the matching relationship between the color test image and the infrared test image.

In some embodiments, when the captured scene is a scene with multiple depths, the image obtained by the color sensor 32 and the infrared sensor 34 will have aberration due to parallax between them, so it is necessary to calculate the matching relationship for the image planes at different depths. At this time, the processor 38 will use the color test image and the infrared test image to calculate the multiple depths of the captured scene, and divide the captured scene into multiple depth scenes of different depths (for example, into near view and distant view). Among them, the processor 38 will, for example, establish a quadratic equation for each depth scene, as follows:

Wherein, (x, y) represents the position of the designated feature point in the color test image, (x', y') represents the position of the corresponding feature point in the infrared test image, and a-f represent variables. The processor 38, for example, substitutes the positions of each feature point in the color test image and the corresponding feature point in the infrared test image into the above quadratic equation for solution, and uses the obtained solution as the matching relationship between the color test image and the infrared test image.

On the other hand, for the calibration of the brightness matching, FIG6 is a flow chart of a brightness matching calibration method of a dual sensor camera system according to an embodiment of the present invention. Please refer to FIG3 and FIG6 simultaneously. The method of this embodiment is applicable to the dual sensor camera system 30 described above. The following is a description of the detailed steps of the brightness matching calibration method of this embodiment with various components of the dual sensor camera system 30.

In step S602, the processor 38 controls the color sensor 32 and the infrared sensor 34 to respectively acquire multiple color images and multiple infrared images of a camera scene using multiple shooting conditions. The shooting conditions include, for example, the wavelength range and brightness of the ambient light, and the distance from the subject and the background in the camera scene, or a combination thereof, which are not limited here.

In step S604, the processor 38 calculates a plurality of color image parameters of the color image acquired under each shooting condition and a plurality of infrared image parameters of the infrared image acquired under each shooting condition, and uses them to calculate the difference between the brightness of the color image and the brightness of the infrared image.

In some embodiments, for images taken under different shooting conditions, the processor 38, for example, calculates 3A (including auto focus (AF), auto exposure (AE), and auto white balance (AWB)) statistics for each image, and uses them to calculate the difference between the brightness of the images (for example, a difference or a ratio).

In some embodiments, the processor 38, for example, cuts each color image and each infrared image into multiple blocks, and calculates the average pixel value of all pixels in each block, thereby calculating the difference in the average pixel values of corresponding blocks, which is used as the difference between the brightness of the color image and the brightness of the infrared image.

In some embodiments, the processor 38 calculates the image histogram of each color image and each infrared image, and calculates the difference between the image histograms of the color image and the infrared image as the difference between the brightness of the color image and the brightness of the infrared image.

Returning to the process of FIG. 6 , in step S606 , the processor 38 determines exposure settings suitable for the color sensor 32 and the infrared sensor 34 based on the calculated differences.

In some embodiments, in order to achieve image synchronization, the processor 38, for example, controls the color sensor 32 and the infrared sensor 34 to respectively acquire the color image and the infrared image of the camera scene using the same exposure time, and calculates the brightness difference between the color image and the infrared image, thereby calculating the gain for adjusting the brightness of the color image and/or the brightness of the infrared image. That is, the processor 38 calculates the gain that can compensate for the brightness difference between the color image and the infrared image, which can be a gain for the color image, a gain for the infrared image, or a gain for both of the above, without limitation.

For example, if the color image acquired with the same exposure time is brighter, the gain for adjusting the brightness of the infrared image can be calculated so that the brightness of the infrared image multiplied by the gain is equivalent to the brightness of the color image. The calculated gain is, for example, stored in the storage device 36 together with the corresponding shooting conditions, so that in the subsequent running time, whenever the color image and the infrared image are acquired, the processor 38 can obtain the gain corresponding to the shooting condition from the storage device 36 by identifying the shooting condition, and multiply the pixel value of the acquired color image or infrared image by the obtained gain, so that the brightness of the color image can match the brightness of the infrared image.

In some embodiments, the dual-sensor camera system 30 may be additionally configured with an infrared projector, so as to cooperate with the infrared sensor 34 to assist the processor 38 in calculating the distance between the dual-sensor camera system 30 and the subject and background in the camera scene.

Specifically, FIG7 is a flow chart of a calibration method for a dual sensor camera system according to an embodiment of the present invention. Please refer to FIG3 and FIG7 simultaneously. The method of this embodiment is applicable to the dual sensor camera system 30 described above. The following is a detailed description of the calibration method of this embodiment with reference to the various components of the dual sensor camera system 30.

In step S702 , the processor 38 controls the infrared projector to project invisible light with a special pattern onto the camera scene.

In step S704, the processor 38 controls two infrared sensors in the infrared sensor 34 to respectively acquire a plurality of infrared images of the camera scene having the special pattern.

In step S706, the processor 38 calculates the distance between the dual-sensor camera system 30 and the subject and background in the camera scene according to the special pattern in the acquired infrared image and the parallax between the two infrared sensors. Since the special pattern projected by the infrared projector to the camera scene is not easily affected by the environment, the above method can obtain a more accurate distance of the subject and/or background, and is used to identify the shooting conditions as a basis for subsequent image compensation.

In summary, the dual-sensor camera system and calibration method of the present invention respectively acquire multiple images using different shooting conditions for the color sensor and infrared sensor configured on the dual-sensor camera system, thereby calibrating the color sensor and the infrared sensor for image alignment and brightness matching, and using the calibration result as a basis for adjusting the subsequently acquired images. Therefore, the image quality of the images acquired by the dual-sensor camera system can be improved.

Although the present disclosure has been disclosed as above by way of embodiments, they are not intended to limit the present disclosure. Any person skilled in the art may make some modifications and improvements without departing from the spirit and scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be determined by the appended claims and their equivalent scope.

Claims (20)

1. A dual-sensor camera system, comprising: at least one color sensor; at least one infrared sensor; a storage device storing a computer program; and A processor, coupled to the at least one color sensor, the at least one infrared sensor and the storage device, is configured to load and execute the computer program to: Controlling the at least one color sensor and the at least one infrared sensor to respectively acquire a plurality of color images and a plurality of infrared images of a camera scene using a plurality of shooting conditions; Calculating a plurality of color image parameters of the color image acquired under each of the shooting conditions and a plurality of infrared image parameters of the infrared image acquired under each of the shooting conditions to calculate a difference between the brightness of the color image and the brightness of the infrared image; and Exposure settings suitable for the at least one color sensor and the at least one infrared sensor are determined according to the calculated difference between the brightness of the color image and the brightness of the infrared image, so that the brightness of the color image matches the brightness of the infrared image. 2. The dual sensor camera system of claim 1, wherein the processor comprises: Cutting each of the color images and each of the infrared images into a plurality of blocks, and calculating an average of pixel values of all pixels in each of the blocks; and The average difference of the pixel values of the corresponding blocks is calculated as the difference between the brightness of the color image and the brightness of the infrared image. 3. The dual sensor camera system of claim 1 , wherein the processor comprises: Calculating an image histogram of each of the color images and each of the infrared images; and A difference between the image histograms of the color image and the infrared image is calculated as a difference between the brightness of the color image and the brightness of the infrared image. 4. The dual sensor camera system of claim 1, wherein the processor comprises: Controlling the at least one color sensor and the at least one infrared sensor to respectively acquire the color image and the infrared image of the camera scene using the same exposure time; and A gain for adjusting the brightness of the color image or the brightness of the infrared image is calculated according to the calculated difference between the brightness of the color image and the brightness of the infrared image. 5. The dual sensor camera system of claim 1 , wherein the processor further comprises: Controlling the at least one color sensor and the at least one infrared sensor to respectively photograph a test image having a special pattern to obtain a color test image and an infrared test image; Detecting a plurality of feature points of the special pattern in the color test image and the infrared test image; and An image alignment algorithm is executed to calculate a matching relationship between the color test image and the infrared test image according to a positional relationship between the corresponding feature points in the color test image and the infrared test image, so as to align the subsequently acquired color image and infrared image. 6. The dual sensor camera system of claim 5, wherein the processor comprises: The color test image and the infrared test image are cut into a plurality of blocks, and a feature detection algorithm is executed to detect at least one feature point in each of the blocks, wherein the feature detection algorithm includes a Harris corner detection method. 7. The dual sensor camera system of claim 5, wherein the processor comprises: For a designated feature point among the feature points detected from the color test image, a patch including a plurality of pixels is moved around a pixel corresponding to the designated feature point in the infrared test image to search for a corresponding feature point in the infrared test image corresponding to the designated feature point; and A random sampling consensus algorithm is executed to establish a homography transformation matrix, and the positions of the feature points in the color test image and the corresponding feature points in the infrared test image are brought into the homography transformation matrix for solution, and the obtained solution is used as the matching relationship between the color test image and the infrared test image. 8. The dual sensor camera system of claim 5, wherein the processor comprises: Calculating multiple depths of a camera scene using the color test image and the infrared test image, thereby dividing the camera scene into multiple depth scenes; and A quadratic equation is established for each of the depth scenes, and the positions of the feature points in the color test image and the corresponding feature points in the infrared test image within each of the depth scenes are substituted into the corresponding quadratic equation for solution, and the obtained solution is used as the matching relationship between the color test image and the infrared test image. 9 . The dual-sensor camera system of claim 5 , wherein the image alignment algorithm comprises a brute force method, an optical flow method, a homography transformation method, or a local warping method. 10. The dual sensor camera system of claim 1, further comprising an infrared projector, wherein the processor further comprises: Controlling the infrared projector to project invisible light with a special pattern onto the camera scene; Controlling two infrared sensors of the at least one infrared sensor to respectively acquire a plurality of infrared images of the camera scene having the special pattern; and According to the special pattern in the acquired infrared image and the parallax of the two infrared sensors, the distances between the dual-sensor camera system and the subject and the background in the camera scene are calculated respectively. 11. A calibration method for a dual-sensor camera system, the dual-sensor camera system comprising at least one color sensor, at least one infrared sensor and a processor, the method comprising the following steps: Controlling the at least one color sensor and the at least one infrared sensor to respectively acquire a plurality of color images and a plurality of infrared images of a camera scene using a plurality of shooting conditions; Calculating a plurality of color image parameters of the color image acquired under each of the shooting conditions and a plurality of infrared image parameters of the infrared image acquired under each of the shooting conditions to calculate a difference between the brightness of the color image and the brightness of the infrared image; and Exposure settings suitable for the at least one color sensor and the at least one infrared sensor are determined based on the calculated difference between the brightness of the color image and the brightness of the infrared image, so that the brightness of the color image matches the brightness of the infrared image. 12. The method according to claim 11, wherein the step of calculating the difference between the brightness of the color image and the brightness of the infrared image comprises: Cutting each of the color images and each of the infrared images into a plurality of blocks, and calculating the average pixel value of all pixels in each of the blocks; and The average difference of the pixel values of the corresponding blocks is calculated as the difference between the brightness of the color image and the brightness of the infrared image. 13. The method according to claim 11, wherein the step of calculating the difference between the brightness of the color image and the brightness of the infrared image comprises: Calculating an image histogram of each of the color images and each of the infrared images; and A difference between the image histograms of the color image and the infrared image is calculated as a difference between the brightness of the color image and the brightness of the infrared image. 14. The method according to claim 11, further comprising: Controlling the at least one color sensor and the at least one infrared sensor to respectively acquire the color image and the infrared image of the camera scene using the same exposure time; and A gain for adjusting the brightness of the color image or the brightness of the infrared image is calculated according to the calculated difference between the brightness of the color image and the brightness of the infrared image. 15. The method according to claim 11, further comprising: Controlling the at least one color sensor and the at least one infrared sensor to respectively photograph a test image having a special pattern to obtain a color test image and an infrared test image; Detecting a plurality of feature points of the special pattern in the color test image and the infrared test image; and An image alignment algorithm is executed to calculate a matching relationship between the color test image and the infrared test image according to a positional relationship between the corresponding feature points in the color test image and the infrared test image, so as to align the subsequently acquired color image and infrared image. 16. The method according to claim 15, wherein the step of detecting a plurality of feature points of the special pattern in the color test image and the infrared test image comprises: The color test image and the infrared test image are cut into a plurality of blocks, and a feature detection algorithm is executed to detect at least one feature point in each of the blocks, wherein the feature detection algorithm includes a Harris corner detection method. 17. The method according to claim 15, wherein the step of calculating the matching relationship between the color test image and the infrared test image comprises: For a designated feature point among the feature points detected from the color test image, a patch including a plurality of pixels is moved around a pixel corresponding to the designated feature point in the infrared test image to search for a corresponding feature point in the infrared test image corresponding to the designated feature point; and A random sampling consensus algorithm is executed to establish a homography transformation matrix, and the positions of the feature points in the color test image and the corresponding feature points in the infrared test image are brought into the homography transformation matrix for solution, and the obtained solution is used as the matching relationship between the color test image and the infrared test image. 18. The method according to claim 15, wherein the step of calculating the matching relationship between the color test image and the infrared test image comprises: Calculating multiple depths of a camera scene using the color test image and the infrared test image, thereby dividing the camera scene into multiple depth scenes; and A quadratic equation is established for each of the depth scenes, and the positions of the feature points in the color test image and the corresponding feature points in the infrared test image within each of the depth scenes are substituted into the corresponding quadratic equation for solution, and the obtained solution is used as the matching relationship between the color test image and the infrared test image. 19. The method of claim 15, wherein the image alignment algorithm comprises a brute force method, an optical flow method, a homography transformation method, or a local warping method. 20. The method of claim 11, wherein the dual sensor camera system further comprises an infrared projector, the method further comprising: Controlling the infrared projector to project invisible light with a special pattern onto the camera scene; Controlling two infrared sensors of the at least one infrared sensor to respectively acquire a plurality of infrared images of the camera scene having the special pattern; and According to the special pattern in the acquired infrared image and the parallax of the two infrared sensors, the distances between the dual-sensor camera system and the subject and the background in the camera scene are calculated respectively.