CN105430298A - Method for Synthesizing HDR Images by Simultaneous Exposure of Stereo Camera System - Google Patents

Abstract

The invention relates to the technical field of image processing, and aims at simultaneously exposing and synthesizing an HDR image and greatly simplifying the computation of an algorithm at the same time. To this end, the technical scheme adopted by the invention is a method for simultaneously exposing and synthesizing the HDR image via a stereo camera system, and the method comprises the following steps: using a plurality of cameras with the same performance or a plurality of vidicons with the same performance to photograph the same scene to obtain LDR images; before photographing, carrying out one-dimensional parallax estimation, calibrating the cameras or the vidicons by an electronic compensation method, and photographing in the same exposure mode; and finally, fusing and synthesizing images photographed by the cameras into the HDR image. The method provided by the invention is mainly applied to image processing.

Description

Method for Synthesizing HDR Images by Simultaneous Exposure of Stereo Camera System

technical field

The invention relates to the technical field of image processing, in particular to a method for simultaneously exposing and synthesizing HDR images through a stereo camera system.

technical background

High dynamic range (HDR: High Dynamic Range) image has become a hot research field at home and abroad because of its advantages of rich layers, strong picture realism, large image dynamic range, and can more realistically restore light and shadow effects close to real scenes. At present, the method of obtaining HDR images is mainly to first obtain a series of LDR (Low Dynamic Range) images with different exposures by adjusting the exposure parameters, and then use image fusion technology to synthesize the LDR image sequence into an HDR image. At present, many researchers have studied this technology, and the existing LDR image acquisition methods are divided into two types, namely simultaneous exposure and time-sharing exposure.

For the method of using simultaneous exposure to obtain LDR images, Nayar et al. [1] proposed a method to change the characteristics of the photosensitive unit according to the different spatial positions. This method covers an optical mask in front of the photosensitive array of the image. The luminosity is different, and adjacent pixels have different transmittances to the incident light, so multiple images with different exposures can be obtained through one shot. However, the LDR image resolution obtained by this method will be reduced.

Another way to acquire a sequence of LDR images with different exposures is to use multiple imaging systems, such as multiple cameras and a beam splitter to generate multiple images of the same scene. Using this method, each image has a separate image sensor, and by using an optical attenuator or changing the exposure time of the photosensitive element to pre-set the exposure parameters of each image separately, multiple LDR images with different exposures can be obtained. The advantage of this type of method is that multiple images with different exposures can be obtained through one shot, which greatly reduces the overall image capture time of the system. However, using multiple image sensors is more difficult in camera setup and calibration than using only one camera, and the number of exposed images as well as the choice of exposure parameters are limited. It is expensive due to the need to use multiple image sensors and delicate optical components. In addition, the incident light in the scene is divided into multiple beams by the beam splitter and projected onto multiple image sensors. The light intensity received by each image sensor is only a part of the incident light intensity. More expensive lenses and larger magnifications are required. Or slow down the shutter speed to compensate for the loss of light intensity.

So far, the simultaneous exposure HDR imaging method is called the HDR imaging method, which is a true HDR imaging method.

Another more common method to obtain image sequences with different exposures is time-sharing exposure, which uses a common camera to rapidly expose multiple times of the same scene. The advantage of this method is that the image capture process and the subsequent image processing process can be performed asynchronously. Based on this idea, Debevec[2] and Mitsunaga et al.[3] proposed to restore the HDR image by calculating the response function of the camera (the function reflecting the relationship between the light intensity and the brightness value of the digital image pixel), combined with the tone mapping algorithm. However, it is difficult to accurately estimate the response curve of the camera, and the stability of the algorithm is not high. Guthier [4] proposed a new exposure method for image sequences. First, determine the exposure time to shoot the scene according to the average light value of the previous frame of the video sequence. This method can analyze and process the previous image while shooting a new LDR image, reducing the time required for capturing the entire image sequence. And it reduces data redundancy when performing image fusion. However, this method of synthesizing a single HDR image by sequentially exposing multiple LDR images has an obvious disadvantage of increasing the overall image capture time of the system. On the other hand, objects in the scene or the camera are likely to move during shooting. At this time, the image sequence for the same scene will no longer display the same content, and the moving object will be dislocated in the image sequence, causing artifacts in the synthesized image.

Although motion estimation and motion compensation can avoid the problem of misalignment in an image sequence in principle, motion estimation is an ill-posed process. For example, not only the speed of motion is unknown, but also the direction of motion cannot be determined, so that motion estimation not only has a large amount of computation, but also has poor reliability of motion vector estimation results.

The HDR imaging method through multiple exposures is called the HDR image synthesis method, which is not a true HDR imaging method.

To sum up, both simultaneous exposure and time-sharing exposure have certain problems in acquiring LDR images to synthesize HDR images.

references

[1] NayarSK, MitsunagaT. Highdynamic range imaging: Spatially varying pixel exposures [C]//ComputerVisionandPatternRecognition, 2000.Proceedings.IEEEConferenceon.IEEE,2000,1:472-479.

[2]DebevecPE, MalikJ.Recoveringhighdynamicrangeradiancemapsfromphotographs[C]//ACMSIGGRAPH2008classes.ACM,2008:31.

[3] MitsunagaT, NayarSK.Radiometricselfcalibration[C]//ComputerVisionandPatternRecognition, 1999.IEEEComputerSocietyConferenceon.IEEE,1999,1.

[4] GuthierB, KopfS, EffelsbergW. Algorithms for real-time HDR videosystem [J]. Pattern Recognition Letters, 2013, 34(1): 25-33.

Contents of the invention

In order to overcome the deficiencies of the existing technology, realize the simultaneous exposure and synthesis of HDR images, and greatly simplify the calculation amount of the algorithm. For this reason, the technical scheme that the present invention takes is, the method for simultaneously exposing and synthesizing an HDR image by a stereo camera system includes the following steps: using a plurality of cameras of the same performance or a plurality of cameras of the same performance to take pictures of the same scene to obtain an LDR image; Before taking pictures, one-dimensional parallax estimation is carried out, and it is calibrated by electronic compensation method, and the pictures are taken by simultaneous exposure; finally, the images taken by each camera are fused to synthesize an HDR image.

The amount of light transmitted by each camera is different, and the resulting LDR image is fused using a computer response function that reflects the relationship between light intensity and digital image pixel brightness values, and combined with tone mapping to synthesize an HDR image.

The amount of light transmitted by each camera is different by adjusting the aperture or exposure time.

Features and beneficial effects of the present invention are:

The present invention adopts a plurality of cameras of a stereo camera system to simultaneously expose and capture a plurality of LDR images, and the performance of the cameras of the system is uniform. Since the parallax can only occur in the horizontal direction, the estimation of the parallax is limited to the horizontal direction, which not only greatly reduces the amount of calculation, but also the estimation result of the parallax is more reliable than the two-dimensional motion vector estimation result.

Stereo camera systems are relatively mature and do not require special manufacturing. In particular, the stereoscopic camera system can be used for dual purposes, and can capture stereoscopic images and synthesize HDR images.

Description of drawings:

Figure 1 is a block diagram of HDR image synthesis using multiple cameras in a stereo camera system.

detailed description

At present, the HDR camera method of simultaneous exposure is expensive, and there are few real HDR camera equipment. The multi-exposure HDR image synthesis method is limited by the problem of motion estimation, especially when shooting video, it is inevitable that the camera will move or the objects in the scene will move. During post-processing, the algorithm has a large amount of computation and motion compensation is difficult. , highly prone to artifacts in the synthesized HDR image.

The present invention proposes a method for synthesizing HDR images by using a stereoscopic imaging system to acquire LDR images in a single simultaneous exposure manner.

The present invention aims at inventing a method for synthesizing HDR images by using a stereo camera system to generate multiple LDR images. In this method, multiple cameras of a stereoscopic camera system are used to simultaneously expose and capture LDR images for the same scene, thereby avoiding the problems caused by the movement of objects in the scene. Due to the multiple cameras of the stereo camera system (from a simple binocular stereo camera to a complex multi-view stereo camera, any camera or video camera with similar performance is fine. Their performance should be as consistent as possible.), only the problem of image displacement , that is to say, the disparity problem. Since the parallax can only occur in the horizontal direction, the estimation of the parallax is limited to the horizontal direction, thereby greatly simplifying the calculation amount of the algorithm.

The present invention is based on a method of simultaneous exposure, using multiple cameras to take pictures of the same scene to obtain LDR images. First of all, multiple cameras with the same performance are required. The same performance can ensure that the captured images have the same parameters, which is convenient for later image fusion. Then, one-dimensional parallax estimation, which is easier to implement, is carried out, and it is calibrated by electronic compensation method, and pictures are taken by simultaneous exposure. Finally, the images taken by each camera are fused to synthesize an HDR image.

The present invention applies multiple cameras using a stereo camera system, tending to have a stereo camera system with more than two cameras (eg six) in order to produce as high a dynamic range as possible. The nature of the stereoscopic imaging system ensures that the geometric positions of the objects are almost in the same direction. Stereoscopic imaging systems inherently require cameras to have fixed geometric positions.

The amount of light transmitted by each camera is different, for example, by adjusting the aperture (Aperture) or the exposure time, so that the amount of light transmitted by each camera is different, so as to take photos with different exposures. The resulting LDR images are preferably fused through the camera response function (a function that reflects the relationship between light intensity and digital image pixel brightness values), combined with tone mapping to synthesize an HDR image.

Claims (3)

1. exposed a method for synthesis HDR image by three-dimensional camera system simultaneously, it is characterized in that, comprise the steps: to use the video camera of the camera of multiple same performance or multiple same performance to take pictures acquisition LDR image to Same Scene; Before taking pictures, carry out one dimension disparity estimation, carry out more accurate by the method for electronic compensation to it, take pictures in the mode exposed simultaneously; Finally, fusion synthesis HDR image is carried out with the image that each camera is taken.

2. exposed the method for synthesis HDR image as claimed in claim 1 by three-dimensional camera system simultaneously, it is characterized in that, make the light transmission capacity of each camera different, the LDR image obtained adopts the computer response function of relation between reflection intensity of illumination and Pixel of Digital Image brightness value to merge, in conjunction with tone mapping synthesis HDR image.

3. being exposed the method for synthesis HDR image as claimed in claim 2 by three-dimensional camera system simultaneously, it is characterized in that, making the light transmission capacity of each camera different by regulating aperture or time for exposure.