CN102509277A - Real-time motion blurred image restoration method for photoelectric hybrid joint transform correlation - Google Patents

Abstract

The invention belongs to the field of image processing, and in particular relates to a restoration method of a real-time motion blurred image related to photoelectric hybrid joint transformation. This method is to use the principle of fast real-time optical correlation to carry out optical correlation calculation on the image sequence collected by high-speed CCD to realize the correlation detection of image motion displacement vector, and the processing speed is hundreds of times higher than that of digital electronic processing. Furthermore, an accurate model of the point spread function (PSF) is established, and then an image restoration algorithm is established through the model to realize high-resolution and real-time motion image restoration of blurred images obtained by the main imaging CCD system. This method has the advantages of high displacement vector detection precision, accurate PSF modeling, and thus high image restoration precision; at the same time, this method has the high speed, large capacity, parallel processing and digital processing technology of optical image processing, flexibility, accuracy, and programmable The advantage, therefore, is that the images are real-time.

Description

Real-time Motion Blurred Image Restoration Method Based on Photoelectric Hybrid Joint Transform Correlation

technical field

The invention relates to a motion blurred image restoration method, especially a real-time motion blurred image restoration method related to photoelectric hybrid joint transformation. field of image processing.

Background technique

High-resolution images are widely used in various fields, especially satellite imaging, aerial reconnaissance imaging, remote sensing and other fields. However, images in these applications are often affected by various factors such as flight movement, various vibrations, and attitude changes, resulting in blurring and a decrease in image spatial resolution. In addition, the image information obtained by aerospace imaging is often massive. It is an inevitable requirement for fast-paced modern high-tech to quickly process these massive information in real time and quickly obtain useful target information in the first time. Although the development of computers has greatly improved the speed of image processing, it is still difficult to process massive amounts of data in real time. The subsequent image processing will consume a lot of time and financial resources, and the real-time performance is poor, which cannot meet the high-tech fast-paced requirements.

At present, the methods for image restoration are mainly through various image restoration algorithms, which can be mainly divided into two types: direct algorithm and blind restoration algorithm (iterative algorithm). Direct algorithms such as inverse filtering and Wiener filtering extract the motion function directly from the image itself. However, due to the complexity and randomness of various vibrations in this method, the image itself often has no sharp edges. It is often necessary to obtain the point spread function PSF The precision is not high enough. In addition, although the blind restoration algorithm does not need to know the point spread function in advance, this iterative algorithm requires an initial estimate of the point spread function, so the effect of image restoration often depends on the estimation accuracy. If the estimation accuracy deviation is large, the restoration of the image The effect is not ideal. Moreover, the above two algorithms are generally more complicated. However, the amount of remote sensing data is often as high as 1000G Mbytes or even massive. If the image processing is performed afterwards, it will take a lot of time and financial resources, which cannot meet the high-tech real-time processing requirements for massive image data, and the real-time performance is poor.

Contents of the invention

In order to overcome the problems of blurred moving images and poor real-time performance in imaging, the real-time motion blurred image restoration method related to photoelectric hybrid joint transformation of the present invention will solve the problems of blurred images and poor real-time performance.

The photoelectric hybrid joint transformation related real-time motion blurred image restoration method of the present invention is characterized in that it adopts a hybrid processing method combining optical correlation processing and digital processing, including the following steps:

Step (1) First, use the high-speed CCD image sensor to obtain the image sequence of the target, store it in the computer 1, and input the adjacent images side by side into the spatial light modulator 1, where the current frame is the reference image, and the next frame is Input image (target image).

Step (2) irradiate the spatial light modulator 1 with the collimated laser (laser) through the lens 1, and pass through the Fourier transform lens 2, record the reference image and the target image with the CCD 1 detector on the output surface of the Fourier lens The joint transformed power spectrum of .

Step (3) Input the power spectrum into the computer 2, process it with a digital processor and then input it into the spatial light modulator 2, irradiate the spatial light modulator 2 and the Fourier transform lens 3 through another laser beam, and detect it by the CCD2 The instrument records its correlation peak.

Step (4) detects and processes the motion vector of the target image relative to the reference image, that is, the motion vector between two adjacent frames.

Step (5) continuously replaces the current frame with the next frame as the reference image, detects a series of motion vectors, and processes the motion vectors of the image within one exposure time of the main CCD imaging system.

Step (6) Process the point spread function of the moving image through the motion vector.

Step (7) Finally, the point spread function is used to establish an image restoration algorithm to realize rapid and real-time restoration of the blurred image obtained by the main CCD imaging system, and obtain a high-resolution, real-time image.

This method also has the advantages of high speed of optical image processing (basically at the speed of light), large capacity, flexibility, accuracy and programmability of parallel processing and digital processing technology, so the image restoration accuracy is high and the real-time performance is good.

Photoelectric hybrid joint transformation related real-time motion blurred image restoration method, which mainly includes: main CCD, high-speed CCD, CCD1, CCD2, laser (laser), lens 1, lens 2, lens 3, spatial light modulator 1, spatial light Composed of modulator 2, computer 1, computer 2 and digital processor.

The high-speed CCD is used to acquire target image sequences.

The detectors CCD1 and CCD2 are used to detect joint image power spectrum and correlation peaks.

The digital processor is used for digital processing and image restoration processing.

The spatial light modulators 1 and 2 are used to display images.

The main CCD imaging is used to acquire target images.

The laser is used as the light source for the system.

Description of drawings

Fig. 1 is the basic principle diagram of the present invention.

Fig. 2 is a correlation peak output diagram of an embodiment of the present invention.

Fig. 3 is a point spread function diagram of an embodiment of the present invention.

Fig. 4 is a comparison of image restoration results of the embodiment of the present invention.

Detailed ways

The following will be described in detail in conjunction with the accompanying drawings and embodiments, but the present invention is by no means limited to the described embodiments.

As shown in Figure 1, the photoelectric hybrid joint transformation related real-time motion blurred image restoration method includes the following steps:

Step (1) First, use the high-speed CCD image sensor to obtain the image sequence of the target, store it in the computer 1, and input the adjacent images side by side into the spatial light modulator 1, where the current frame is the reference image r(x, y), The next frame is the input image (target image) t(x,y), so the joint image is r(x,y)+t(x,y).

Step (2) irradiate the spatial light modulator 1 with the collimated laser light through the lens 1, and pass through the Fourier transform lens 2, and use the CCD1 detector to record the combination of the reference image and the target image on the output surface of the Fourier lens Transform the power spectrum, the mathematical expression of the joint power spectrum S(u, v) is

|S(u, v)| ² = |R(u, v)| ² + |T(u, v)| ²

+R(u,v)T ^* (u,v)exp[-2iπuΔx-2iπv(2a+Δy)]

+T(u,v)R ^* (u,v)exp[2iπuΔx+2iπv(2a+Δy)]

where S(u, v), R(u, v), T(u, v) represent the Fourier transforms of s(x, y), r(x, y) and t(x, y) respectively, (u, v) respectively represent the spatial coordinates of the x and y directions on the Fourier plane, where x=λfu, y=λfv, λ and f represent the working wavelength of the laser and the focal length of the lens 2, respectively.

Step (3) Then, the power spectrum is input into the computer 2, processed by a digital processor and then input into the spatial light modulator 2, and the spatial light modulator 2 and the Fourier transform lens 3 are irradiated by another laser beam, and the CCD2 The detector records its correlation peak, and the output of its correlation peak c(x, y) is

$\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&CircleTimes;</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&CircleTimes;</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>$

$<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&CircleTimes;</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&Delta;x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&Delta;y</span>}<span\; class="notranslate">\; )</span>$

$<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&CircleTimes;</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&Delta;x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&Delta;y</span>}<span\; class="notranslate">\; )</span>$

代表相关操作，δ代表狄拉克符合，Δx，Δy分别代表相邻两帧图像因运动产生的相对位移。In the formula

Represents related operations, δ represents the Dirac coincidence, Δx, Δy represent the relative displacement of two adjacent frames of images due to motion.

Step (4) Detect and process the motion vector of the target image relative to the reference image through the digital processor, that is, the motion vector (Δx, Δy) between two adjacent frames.

Step (5) continuously replace the current frame with the next frame as the reference image, detect the coordinate positions of a series of motion vectors, process the motion vector of the image within one exposure time of the main CCD imaging system, that is, know the trajectory curve of the image motion within the exposure time .

Step (6) Process the point spread function h(x, y) of the moving image through the motion vector.

Step (7) Finally, the point spread function is used to establish an image restoration algorithm to realize rapid and real-time restoration of the blurred image obtained by the main imaging system, and obtain a high-resolution, real-time image. The recovery algorithm expression is as follows:

G(u,v)=H(u,v)F(u,v)+N(u,v)

In the formula, G(u, v), H(u, v), F(u, v) represent the Fourier transform of fuzzy image, clear image and point spread function respectively.

The invention uses the fast real-time optical correlation principle to carry out optical correlation calculation on the image sequence collected by the high-speed CCD to realize the correlation detection of the image motion displacement vector, and the processing speed is hundreds of times higher than that of digital electronic processing. Furthermore, an accurate model of the point spread function (PSF) is established, and then an image restoration algorithm is established through the model to realize high-resolution and real-time motion image restoration of blurred images obtained by the main CCD imaging system. This method has the advantages of high displacement vector detection accuracy, accurate PSF modeling, and thus high image restoration accuracy; at the same time, this method has the high speed of optical image processing (basically at the speed of light), large capacity, parallel processing and flexibility of digital processing technology , Accuracy, programmable advantages, therefore, good real-time performance.

Claims (4)

1. the real-time motion-blurred image restoration method relevant to photoelectric hybrid joint transformation of the present invention is characterized in that comprising the following steps: (1) Firstly, a high-speed CCD image sensor is used to obtain an image sequence of the target, and adjacent images are input side by side into the spatial light modulator 1, wherein the current frame is a reference image, and the next frame is an input image (target image); (2) The laser beam collimated by the lens 1 irradiates the spatial light modulator 1, and passes through the Fourier transform lens 2, and records the joint transformed power spectrum of the reference image and the target image with a CCD1 detector on the output surface of the Fourier transform lens ; (3) Input the power spectrum into the computer 2, and then input it into the spatial light modulator 2 after processing, irradiate the spatial light modulator 2 and the Fourier transform lens 3 through another laser beam, and record its correlation by the CCD2 detector peak; (4) Detect and process the motion vector of the target image relative to the reference image, that is, the motion vector between two adjacent frames; (5) Continuously replace the current frame with the next frame as a reference image, detect a series of motion vectors, and process the motion vector of the image within one exposure time of the main imaging system; (6) process the point spread function of moving image by motion vector; (7) Finally, the image restoration algorithm is established by using the point spread function, and the blurred image obtained by the main imaging system is restored quickly and in real time. 2. According to claim 1, it is characterized in that: the point spread function in the step (6) is to detect the coordinate position of the correlation peak by the principle of optical correlation to determine the image motion vector, and then obtain the point spread function. 3. According to claim 1, it is characterized in that: the image restoration method utilizes the method of optical processing, that is, the principle of optical correlation. 4. According to claim 1, it is characterized in that: the method is an image restoration method using photoelectric hybrid processing.

Images