CN102062572B - JTC-based high-precision photoelectric mixed image motion measuring device and method thereof - Google Patents

Abstract

The invention relates to a JTC-based high-precision photoelectric mixed image motion measuring device, which comprises a laser, a collimating mirror arranged on the light path of the laser, a Spatial Light Modulator (SLM) arranged on the emergent light path of the collimating mirror, a Fourier lens arranged on the emergent light path of the SLM, a CCD (charge coupled device) device for receiving the emergent light of the Fourier lens, and a digital processing unit which is connected with the output end of the CCD device and is used for carrying out local discrete Fourier transform calculation by combining with jointly transformed power spectrum data to obtain an up-sampled image of a cross-correlation peak neighborhood. The invention only needs one set of optical Fourier transform device, which is beneficial to reducing the volume and the cost of the whole detection device and reducing the complexity of the system.

Description

High precision photoelectric mixing image drift measurement mechanism and method thereof based on JTC

Technical field

The invention belongs to photoelectric detection technology field, relate to the optical joint conversion correlation technique and the measurement mechanism that are used for image registration and image drift measurement.

Background technology

The research and development of optical correlation in recent years very rapidly; To joint transform correlator (Joint Transform Correlator-JTC), develop into computer-controlled photoelectricity commingled system from early matched filter (Vander LugtCorrelator) from pure optical system.The parallel processing of the existing optics of joint transform correlator, high capacity and high-speed advantage have the dirigibility and the programmability of Computer Processing again.Therefore, joint transform correlator has been widely used in numerous areas such as machine vision, target following, object monitoring, optical computing.Equally, joint transform correlator also can be used in the remote sensing field, can carry out the image drift of image registration, super-resolution image reconstruction and camera imaging process through this effective technical means and device and measure.But this has also proposed new requirement to optical joint conversion correlation technique, like high-precision image dislocation measurement performance, measurement mechanism miniaturization and property etc. in real time or quasi real time.

The course of work of joint transform correlator can be divided into for two steps, and promptly the record of joint transform power spectrum and the relevant of power spectrum are read.Typical joint transform correlator structure is as shown in Figure 1.When carrying out two width of cloth image image drift measurements or registration, at first will be with reference to image f _r(x is y) with tested image f _t(x y) is loaded on the spatial light modulator SLM13, and the center is respectively (a, 0), and (a, 0) is as shown in Figure 2.For explaining conveniently; (wherein, solid line triangle 14 is represented reference picture, and solid line triangle 15 is represented present image to represent common scenery in two width of cloth images with triangle; The current ideal image of dotted line triangle 16 representatives); Dislocation or image drift along x, y direction between two width of cloth images are respectively δ x, δ y, then the input i of correlator (x y) can be expressed as:

i(x，y)＝f _r(x，y-a)+f _t(x+δx，y+a+δy)

Laser instrument send monochromatic coherent light through shining behind the collimation on the SLM13, through input picture i (x, y) Fourier transform is for the first time accomplished through fourier transform lens FL14 in the modulation back, obtain its frequency spectrum function T (u, v):

T(u，v)＝F _r(u，v)exp(-2iπav)+F _texp{2iπ[uδx+(a+δy)v]}

T (u, power spectrum function v) is:

${\left|T(u,v)\right|}^2={\left|F_r(u,v)\right|}^2+{\left|F_t(u,v)\right|}^2+F_r{F}_t^{*}\exp \{-2\mathrm{i\π}[\mathrm{u\δx}+(2a+\mathrm{\δy})v\left)\right]\}$

$+F_t{F}_r^{*}\exp \left\{2\mathrm{i\π}(\mathrm{u\δx}+(2a+\mathrm{\δy})v]\right\}$

Utilize CCD15 right | T (u, v) | ²Record, and be loaded into it on the SLM27 through numerical control unit 6 (being generally computer system), accomplish for the second time Fourier transform through fourier transform lens FL28 after, on CCD29, obtain its relevant being output as:

$c(x,y)=f_r(x,y)\⊗f_r(x,y)+f_t(x,y)\⊗f_t(x,y)+f_r\⊗f_t*\mathrm{\δ}(x-\mathrm{\δx},y-2a-\mathrm{\δy})$

$+f_t\⊗f_r*\mathrm{\δ}(x+\mathrm{\δx},y+2a+\mathrm{\δy})$

* representes convolution in the formula, represent related operation.Visible from following formula, preceding two is the auto-correlation item of two width of cloth images, and back two is the simple crosscorrelation item.The locus distribution of simple crosscorrelation item relevant peaks is as shown in Figure 3, imaginary circle 17 representation theory relevant peaks positions among the figure (output image when promptly not having dislocation or image drift), the pairing actual output relevant peaks in actual scenery position position among real circle 18 representatives and Fig. 2.Through detection and calculating, can obtain the information that to misplace between two width of cloth images, and then accomplish the purpose of image registration or image drift measurement the simple crosscorrelation peak position.

But be different from Application of pattern recognition; Utilizing joint transform correlator to carry out in the application of image registration and image drift measurement; Be not to carry out the comparison of simple crosscorrelation peak intensity but the position that will confirm the simple crosscorrelation peak accurately, measuring accuracy will reach about 1/10th pixels usually.The relevant output that obtains from joint transform correlator has comprised the auto-correlation peak and the simple crosscorrelation peak of two width of cloth images, and most energy (more than 80%) concentrates on the auto-correlation peak, thereby influences the detection at simple crosscorrelation peak.Therefore; Usually strengthen and compose intensity and the sharpening simple crosscorrelation peak that means such as binaryzation or high-pass filtering slacken the auto-correlation peak with joint Power through the input image being carried out the edge, again simple crosscorrelation peak neighborhood is carried out the inferior pixel position that centroid calculation or curve fitting obtain the simple crosscorrelation peak.But this processing procedure has increased the complexity of whole testing process, is subject to the image of various intermediate links and has limited the further raising of accuracy of detection.

Summary of the invention

To the application demand of above-mentioned optical joint conversion correlation technique, the present invention proposes a kind of new optic-electronic hybrid joint transform relevant measurement method and device.It has the measuring accuracy height, helps reducing system complexity, realizes characteristics such as miniaturization, real-time and saving cost.

A kind of high precision photoelectric mixing image drift measurement mechanism based on JTC; Comprise laser instrument, be arranged on collimating mirror on this laser light path, be arranged on spatial light modulator SLM on the emitting light path of this collimating mirror, be arranged on the fourier transform lens on the emitting light path of spatial light modulator SLM and be used to receive the CCD device of the emergent ray of this fourier transform lens; Its special character is, this device also comprise the output terminal that is connected in said CCD device be used to combine the joint transform power spectrum data to carry out the digital processing element that local DFT calculated, obtained the liter sampling image of simple crosscorrelation peak neighborhood.

Above-mentioned digital processing element is FPGA.

A kind of high precision photoelectric mixing image drift measuring method based on JTC; Its special character is, the method comprising the steps of 1) through confirming the coordinate in relevant output face with tested image in the base attribute of placement coordinate on the SLM and Fourier transform at the next desirable simple crosscorrelation peak of two width of cloth image dislocation-free situation with reference to image.Then, according to the tested image local correlation output data that to calculate with this desirable simple crosscorrelation peak with respect to windowing with reference to the transformation range of the dislocation of image be the center, and tentatively confirm the coordinate of actual simple crosscorrelation peak in relevant output face;

Step 2), be center calculation local correlation output data with actual simple crosscorrelation peak; The k that realizes this relevant output data doubly rises sampling; Thereby obtain the coordinate position of actual simple crosscorrelation peak under 1/k pixel precision; And compare with desirable simple crosscorrelation peak coordinate, obtain with reference to image and registration data or the image drift data of tested image under 1/k pixel precision.

Above-mentioned k doubly rises sampling and realizes through DFT matrix multiplication or fractional Fourier transform.

In sum; High precision photoelectric vision-mix registration and image drift measuring method and device based on JTC proposed by the invention has following characteristics: 1) only need a cover optical Fourier transformation device; Help dwindling volume, the cost of whole detection device, reduce the complicacy of total system.Utilized high speed, the concurrency of optical Fourier transformation to accomplish the main calculation task of whole testing process simultaneously.2) adopt digital signal processor spare (like FPGA) to accomplish from joint Power spectrum data to the calculating of relevant output data; The simple crosscorrelation peak detection technique that is proposed can obtain the high resolving power interpolative data at simple crosscorrelation peak under the situation that does not increase calculated amount, thereby realizes the high Precision Detection to dislocation or image drift between image.Along with improving constantly of digital signal processor calculated performance and concurrency, this system can be used to detect in real time task.

Description of drawings

The typical joint transform correlator structural representation of Fig. 1;

Fig. 2 correlator input picture layout viewing;

Fig. 3 exports the relevant peaks position view;

The novel JTC high precision photoelectric of Fig. 4 vision-mix registration and image drift measurement mechanism structural representation;

Fig. 5 carries out the window's position synoptic diagram of local Fourier transform in relevant output face;

Fig. 6 utilizes the joint Power spectrum to carry out liter process synoptic diagram of the local Fourier transform of sampling.

Embodiment

The JTC that utilizes that the present invention proposes carries out the method for high precision image registration and image drift measurement; The basic structure of its implement device is as shown in Figure 4; This measurement mechanism only uses a spatial light modulator, fourier transform lens and CCD device; Reach the purpose of acquisition, and programmable processor spares such as use FPGA carry out the information that specific computing obtains relevant output with reference to the joint transform power spectrum of image and tested image.This structure will reduce volume, weight and the cost of whole detection system greatly.

Obtain with reference to image and tested image association transform power spectrum through optical Fourier transformation after, iff utilizes digital signal processor spare to accomplish accuracy of detection and speed that the second step Fourier transform process can not effectively improve whole detection system.Here propose to utilize digital signal processor spare to combine the joint transform power spectrum data to carry out local DFT and calculate, obtain the liter sampling image of simple crosscorrelation peak neighborhood.Owing to only need carry out local Fourier transform, can reduce calculated amount greatly, and can according to the needs setting of using in various degree rise sampling rate, effectively improve the inferior pixel position detection accuracy at simple crosscorrelation peak.

Referring to Fig. 5, in concrete implementation procedure, accomplish the high Precision Detection of inferior pixel position, simple crosscorrelation peak in two steps:

The first step is through confirming at two width of cloth image dislocation-free situation next desirable simple crosscorrelation peak coordinate on relevant output face with tested image in the base attribute of placement coordinate on the SLM and Fourier transform with reference to image.Then, according to the tested image local correlation output data that to calculate with this desirable simple crosscorrelation peak with respect to windowing with reference to the transformation range of the dislocation of image be the center, and tentatively confirm the coordinate of actual simple crosscorrelation peak in relevant output face.

Second step; With actual simple crosscorrelation peak is the relevant output data of center calculation local (like 3 * 3 windows); And this relevant output data is carried out k liter doubly sample; Thereby obtain the coordinate position of actual simple crosscorrelation peak under 1/k pixel precision, and compare, obtain with reference to image and the registration data or the image drift data of tested image under 1/k pixel precision with desirable simple crosscorrelation peak coordinate.

The liter of simple crosscorrelation peak neighborhood is sampled local Fourier transform can be through DFT matrix multiplication or fractional Fourier transform realization.Here explain with the DFT matrix multiplication.

Theoretical according to Fourier transform, to be equivalent on frequency field frequency spectrum to this signal slotting in having carried out on the time domain time signal being carried out zero padding.Equally, zero padding on the frequency field be equivalent on time domain this signal has been realized in the desirable band limit slotting.For example, if will carry out 2 times of desirable interpolations that rise sampling to the one-dimensional signal that a length is N, can carry out zero padding by the Fourier transform frequency spectrum to it, making its extended length is 2N, carries out inverse fourier transform again and realizes.To the application here, just need carry out two-dimentional zero padding to the joint transform power spectrum data if hope to obtain the image data that simple crosscorrelation peak k in the relevant output face of JTC doubly rises sampling, expand to k times of original size, carry out Fourier transform again.This will increase calculated amount greatly undoubtedly, even utilize existing equipment to realize.For example,, it is carried out 10 times rise interior the inserting of sampling, just need carry out the inverse fourier transform computing of 10240 * 10240 big minor matrixs its frequency spectrum for one 1024 * 1024 image.Therefore, the present invention adopts the DFT matrix multiplication to solve the problems referred to above.

For the two-dimentional joint transform power spectrum image data f of a M * N size (x, y), the matrix form of its DFT is:

$\mathrm{DFT}\left(f\right)=W_M^u\×f\×W_N^v$

Wherein $W_M^u=\mathrm{Exp}[\mathrm{<figure-callout\; id="2"\; label="i"\; filenames="H2009102189360E00022.png"\; state="\{\{state\}\}">i</figure-callout>}2\mathrm{\&pi;\; Ux}/M],$ $W_N^v=\mathrm{Exp}[\mathrm{<figure-callout\; id="2"\; label="i"\; filenames="H2009102189360E00022.png"\; state="\{\{state\}\}">i</figure-callout>}2\mathrm{\&pi;\; Vy}/N].$ Because with reference to image and the placement coordinate of tested image on SLM is known, so the position at its desirable simple crosscorrelation peak also is known in relevant output face 19.If known tested image is no more than t pixel with respect to the skew range with reference to image; Be that the first big or small subwindow 20 (T=2t+1) of a T * T is opened at the center with any one in two desirable simple crosscorrelation peaks in the relevant output face so; And carry out the relevant data of output face in first subwindow 20 of local DFT acquisition, and then through relatively obtaining actual simple crosscorrelation peak and desirable simple crosscorrelation peak distance in the Pixel-level level.Concrete form of calculation is:

${\mathrm{DFT}}_{\mathrm{window}1}\left(f\right)=W_{\mathrm{window}1}^u\&times;f\&times;W_{\mathrm{window}1}^v---\left(1\right)$

W wherein _Window1 ^uAnd W _Window1 ^vFor calculating the pairing W of DFT with size in the position according to subwindow 1 _M ^uAnd W _N ^vPart coefficient submatrix.

After the pixel position of actual relevant peaks is confirmed; With actual simple crosscorrelation peak is that one second subwindow 21 (for example size is 3 * 3 pixels) is opened at the center; Demand according to inferior pixel accuracy of detection; Can carry out the relevant output face of local DFT acquisition by top form to second subwindow 2 equally and doubly rise the data in second subwindow 2 under the sampling, thereby confirm the coordinate position of actual relevant peaks in 1/k pixel at k.Concrete form of calculation is:

${\mathrm{DFT}}_{\mathrm{window}2}\left(f\right)=W_{\mathrm{window}2}^u\&times;f\&times;W_{\mathrm{window}2}^v---\left(2\right)$

W wherein _Window2 ^u, W _Window2 ^vRise the pairing W of the local DFT of sampling for calculating with size according to the position of subwindow 2 _M ^uAnd W _N ^vPart coefficient submatrix.

At this moment $W_M^u=\mathrm{Exp}[\mathrm{<figure-callout\; id="2"\; label="i"\; filenames="H2009102189360E00022.png"\; state="\{\{state\}\}">i</figure-callout>}2\mathrm{\&pi;\; Ux}/\left(\mathrm{KM}\right)],$ $W_N^v=\mathrm{Exp}[\mathrm{<figure-callout\; id="2"\; label="i"\; filenames="H2009102189360E00022.png"\; state="\{\{state\}\}">i</figure-callout>}2\mathrm{\&pi;\; Vy}/\left(\mathrm{KN}\right)].$

Result of calculation based on above two steps is formula (1) and formula (2), just can realize the coordinate of actual cross-correlation peak under 1/k pixel precision detected.Simultaneously, above-mentioned simple crosscorrelation peak detection technique can effectively reduce complexity of calculation.For example, for the joint Power spectrum image data of M * N size, it is carried out the FFT computation complexity that k doubly rises sampling is o (MNk ²[log (kM)+log (kN)]), the computing method computation complexity above adopting is o (MNk).

Claims (4)

1. high precision photoelectric mixing image drift measurement mechanism based on JTC; Comprise laser instrument, be arranged on collimating mirror on this laser light path, be arranged on a spatial light modulator SLM on the emitting light path of this collimating mirror, be arranged on a fourier transform lens on the emitting light path of spatial light modulator SLM and be used to receive a CCD device of the emergent ray of this fourier transform lens, it is characterized in that: this device also comprise the output terminal that is connected in said CCD device be used to combine the joint transform power spectrum data to carry out the digital processing element that local DFT calculated, obtained the liter sampling image of simple crosscorrelation peak neighborhood.

2. the high precision photoelectric mixing image drift measurement mechanism based on JTC according to claim 1, it is characterized in that: said digital processing element is FPGA.

3. high precision photoelectric mixing image drift measuring method based on JTC is characterized in that: the method comprising the steps of 1) through confirming the coordinate in relevant output face with tested image in the base attribute of placement coordinate on the SLM and Fourier transform at the next desirable simple crosscorrelation peak of two width of cloth image dislocation-free situation with reference to image; Then, according to the tested image local correlation output data that to calculate with this desirable simple crosscorrelation peak with respect to windowing with reference to the transformation range of the dislocation of image be the center, and tentatively confirm the coordinate of actual simple crosscorrelation peak in relevant output face;

Step 2), be center calculation local correlation output data with actual simple crosscorrelation peak; The k that realizes this relevant output data doubly rises the local Fourier transform of sampling; Thereby obtain the coordinate position of actual simple crosscorrelation peak under 1/k pixel precision; And compare with desirable simple crosscorrelation peak coordinate, obtain with reference to image and registration data or the image drift data of tested image under 1/k pixel precision.

4. the high precision photoelectric mixing image drift measuring method based on JTC according to claim 3 is characterized in that: said k doubly rises the local Fourier transform of sampling and realizes through DFT matrix multiplication or fractional Fourier transform.

Images