CN114170192B - Fine detection method of focal plane jitter of spaceborne optical camera - Google Patents
Fine detection method of focal plane jitter of spaceborne optical camera Download PDFInfo
- Publication number
- CN114170192B CN114170192B CN202111502706.4A CN202111502706A CN114170192B CN 114170192 B CN114170192 B CN 114170192B CN 202111502706 A CN202111502706 A CN 202111502706A CN 114170192 B CN114170192 B CN 114170192B
- Authority
- CN
- China
- Prior art keywords
- tremor
- error
- relative error
- frequency
- amplitude
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 31
- 230000003287 optical effect Effects 0.000 title claims abstract description 23
- 206010044565 Tremor Diseases 0.000 claims abstract description 194
- 238000003384 imaging method Methods 0.000 claims abstract description 25
- 238000004458 analytical method Methods 0.000 claims abstract description 22
- 238000000034 method Methods 0.000 claims description 18
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 238000012886 linear function Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000017525 heat dissipation Effects 0.000 claims description 3
- 238000005259 measurement Methods 0.000 abstract description 7
- 230000000737 periodic effect Effects 0.000 abstract description 5
- 230000006870 function Effects 0.000 description 29
- 230000000694 effects Effects 0.000 description 8
- 238000005096 rolling process Methods 0.000 description 8
- 238000013211 curve analysis Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a fine detection method for focal plane tremor of a satellite-borne optical camera, which comprises the following steps: s1, high-precision dense matching: determining an image overlapping region between adjacent frames of the planar array CMOS sequence image, and then performing pixel-by-pixel matching to obtain homonymous points of all pixel points in the overlapping region, so as to generate a parallax image; s2, tremor relative error analysis: combining a plurality of tremor relative error curves of the single parallax images with time to form tremor relative error curves of the sequence images, and filling gaps; performing Fourier transform analysis to obtain the frequency and the amplitude of the tremor relative error curve and obtain the frequency, the amplitude and the initial phase of tremor relative error; s3, tremor absolute error modeling: modeling the tremor absolute error, and combining the frequency, amplitude and initial phase of the tremor relative error to obtain the frequency, amplitude and initial phase of the tremor absolute error. The invention realizes the accurate measurement of the periodic tremor error of the imaging focal plane and can provide a reliable data basis for the tremor error compensation of the remote sensing image.
Description
Technical Field
The invention belongs to the technical field of imaging of satellite-borne optical cameras, and particularly relates to a fine detection method for focal plane tremor of a satellite-borne optical camera.
Background
The satellite platform tremble refers to a vibration phenomenon with smaller amplitude and higher frequency, which is generated by the influence of internal and external factors on a satellite. The high-resolution optical satellite is affected by the tremble of the platform, so that the focal plane of an imaging system carried on the satellite platform is tremble simultaneously, and the quality of satellite images is directly affected. As known from imaging geometry principles, for satellites with orbit heights of 500km, platform tremors of 1 degree/second cause distortion errors of push-broom images of 2-3 meters, and with the improvement of orbit heights and spatial resolution, the influence of the platform tremors becomes more remarkable. In order to realize high-precision processing and application of high-resolution optical satellites, platform tremor is a critical problem to be solved urgently.
At present, research on tremor treatment of a high-resolution optical satellite image platform is mainly divided into platform tremor detection and compensation based on an attitude sensor and based on remote sensing images. The platform tremble measurement based on the attitude sensor mainly utilizes sensors such as angular displacement, angular accelerometer, angular vibration and the like to acquire information such as high-frequency angular displacement, high-frequency angular acceleration, high-frequency angular vibration and the like of the platform, and performs information fusion with the absolute attitude acquired by the star sensor to acquire an attitude measurement result with high precision and high frequency. For example, the method for measuring the satellite platform flutter based on the linear accelerometer can realize the reconstruction of high-resolution satellite flutter distortion images, and can effectively eliminate the distortion of high-resolution satellite image imaging under the platform flutter. The satellite platform tremor error detection and compensation based on the remote sensing image mainly utilizes the time-sharing imaging characteristic of the optical satellite or reference data to acquire the platform tremor rule, compensates the tremor error and improves the geometric quality of the remote sensing image.
In recent years, along with the continuous perfection of the area array CMOS (Complementary Metal Oxide Semiconductor ) process, the area array CMOS image sensor has the advantages of high imaging quality, low cost, low power consumption, high speed, strong radiation resistance and the like, and gradually plays a certain role in the aerospace field. Furthermore, the area array CMOS has flexible imaging modes, and can be provided with various imaging modes such as global imaging, rolling shutter imaging, windowing imaging and the like. The rolling shutter adopts a progressive exposure mode, has the capability of stable perception of a super-strong platform, and can become a new platform tremor detection means.
Disclosure of Invention
The invention aims to provide a fine detection method for focal plane tremor of an on-board optical camera, which is used for realizing accurate measurement of periodic tremor errors of an imaging focal plane.
In order to achieve the above purpose, the invention provides a fine detection method for focal plane tremor of an on-board optical camera, which comprises the following steps:
s1, high-precision dense matching
Determining an image overlapping region between adjacent frames of the planar array CMOS sequence image, and then performing pixel-by-pixel matching to obtain homonymous points of all pixel points in the overlapping region, so as to generate a parallax image;
s2, tremor relative error analysis
Generating a single parallax map tremor relative error curve based on the single parallax map;
Combining a plurality of tremor relative error curves of the single parallax images through time correlation to form tremor relative error curves of the sequence images;
Gaps exist between tremble relative error curves of adjacent images, and a fitting interpolation method is adopted to fill the gaps;
Carrying out Fourier transform analysis on the tremor relative error curve of the filled sequence image to obtain the frequency and the amplitude of the tremor relative error curve;
Taking the obtained frequency and amplitude as initial values, and adopting a sine function model to perform integral least square fitting on tremor relative error curves of the sequence images to obtain the frequency, amplitude and initial phase of tremor relative errors;
S3, tremor absolute error modeling
Modeling the tremor absolute error, and combining the frequency, amplitude and initial phase of the tremor relative error to obtain the frequency, amplitude and initial phase of the tremor absolute error.
In some alternative embodiments, determining the image overlap region between adjacent frames of an area array CMOS sequence image comprises:
matching at least one homonymy point between two adjacent frames of images of the planar array CMOS sequence image by adopting a SIFT algorithm;
counting the coordinate difference of the same-name points between two adjacent frames of images to obtain offset;
According to the offset, calculating the initial position of each pixel point of the front sequence image on the rear sequence image; if the initial position of a pixel point exceeds the region of the rear sequence image, the pixel point is not in the overlapping region.
In some alternative embodiments, pixel-by-pixel matching is performed using a correlation coefficient matching and least squares matching algorithm.
In some alternative embodiments, generating the disparity map includes: and calculating coordinate differences of the same name points of all pixel points in the overlapping area of two adjacent frames of images in the row direction and the column direction, and generating a coordinate difference map in the row direction and the column direction by taking the coordinate differences as DN values.
In some alternative embodiments, generating a single disparity map tremor relative error curve based on the single disparity map comprises:
Calculating the average value of each row of the single parallax map as the relative error of the row, and further obtaining a tremor relative error curve along the scanning row; and converting the scanning line into time to obtain a tremor relative error curve.
In some alternative embodiments, the method of fitting interpolation includes straight line function interpolation fitting and spline function interpolation fitting. Further, when the absolute value of the slope of the samples at the two ends of the gap is larger than 300, adopting linear function interpolation fitting or spline function interpolation fitting; when the absolute value of the slope of the samples at the two ends of the gap is between 100 and 300, adopting linear function interpolation fitting; and when the absolute value of the slope of the samples at the two ends of the gap is smaller than 100, adopting spline function interpolation fitting.
In some alternative embodiments, the sine function model is as follows:
Wherein, f k,Ark is used for the treatment of the heat dissipation, The frequency, amplitude and initial phase of the kth sinusoidal component of the tremor relative error, respectively.
In some alternative embodiments, modeling the tremor absolute error, and combining the frequency, amplitude, and initial phase of the tremor relative error to obtain the frequency, amplitude, and initial phase of the tremor absolute error comprises:
The frequency of the tremor absolute error is the same as the frequency of the tremor relative error;
when there is only one tremor frequency, the absolute tremor amplitude is calculated as follows:
Ad＝Ar/(2sin(πfΔt))
Wherein Ad is the amplitude of tremor absolute error, ar is the amplitude of tremor relative error, f is tremor frequency, and Δt is the time interval of homonymous point imaging of adjacent images;
the initial phase calculation formula of tremor absolute error is as follows:
wherein, Representing the initial phase of the absolute error of tremor,Representing the initial phase of tremor relative error, mod (fΔt/2, 2) represents the remainder of fΔt/2 divided by 2.
In some alternative embodiments, the tremor absolute error is expressed as follows:
where D (t) represents a function of tremor absolute error over time.
Compared with the prior art, the invention has the following advantages:
the invention realizes the accurate measurement of the periodic tremor error of the imaging focal plane through three main steps of dense matching of the planar array CMOS sequence images, relative tremor error analysis and absolute tremor error modeling, and can provide a reliable data base for tremor error compensation of the remote sensing images.
Drawings
Fig. 1 is a flowchart of a fine detection method for focal plane tremor of an on-board optical camera.
FIG. 2 is a single disparity map at 100Hz, a tremor relative error curve and a global fit map, and a result map of a tremor error curve Fourier analysis according to the present invention.
Fig. 3 is a plot of tremor relative error, a plot of local linear interpolation fit, a plot of local spline interpolation fit, a plot of error from two interpolation methods, a plot of tremor curve fourier analysis results, and a plot of overall tremor fit for the sequence disparity plot at 10Hz of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. In addition, the technical features of the embodiments of the present invention described below may be combined with each other as long as they do not collide with each other.
Aiming at the limitations of the traditional tremor detection method based on the attitude sensor and the remote sensing image platform, the invention utilizes the rolling shutter area array CMOS detector to carry out the fine detection of the focal plane tremor of the satellite-borne optical camera. The rolling shutter area array CMOS sensor and the linear array CCD (Charge Coupled Device ) are arranged on the same imaging focal plane, and the tremble error at the imaging focal plane can be obtained by utilizing the RS effect. The three main steps of rolling shutter area array CMOS sequence image dense matching, relative tremor error analysis and absolute tremor error modeling are adopted to realize the accurate measurement of the periodic tremor error of the imaging focal plane, and a reliable data base can be provided for the tremor error compensation of the remote sensing image.
The invention discloses a fine detection method for focal plane tremor of a satellite-borne optical camera based on a rolling shutter area array CMOS, which comprises the following steps:
step1, high precision dense matching
And performing dense matching on the planar array CMOS sequence images according to the imaging time. The method comprises the steps of determining the image overlapping degree between adjacent frames, determining the range available for tremor detection, and then performing pixel-by-pixel matching to obtain homonymous points of all image points in an overlapping area, so as to generate a corresponding parallax map.
Specifically, firstly, a tremor detection area is determined, and a small number of homonymous points are matched between two frames of images by adopting a SIFT algorithm. And calculating the initial position of each pixel point of the front sequence image on the subsequent image by counting the coordinate difference of the same name points. And carrying out pixel-by-pixel matching by adopting a correlation coefficient matching and least square matching algorithm to obtain homonymous points of all image points, calculating coordinate differences of homonymous points of all the pixel points in the overlapping area obtained by the pixel-by-pixel matching in two directions of rows and columns, and generating a coordinate difference map in two directions by taking the coordinate differences as DN values, namely a so-called parallax map.
Step 2, relative tremor error analysis
1) And generating a single parallax map tremor error curve. The tremor relative error curve along the scan line is derived from the average of each line of the single disparity map.
2) And generating a tremor error curve of the sequence disparity map. Because tremor frequency is low, a single set of data is difficult to show tremor change trend, so that analysis of sequence images is performed. And combining a plurality of groups of single parallax map tremor error curves with the same frequency by using time to obtain tremor relative error curves of the sequence images.
3) Fitting interpolation. Due to the size of the matching window, a gap is formed between detection results of adjacent images. Therefore, the gap is filled, namely, interpolation is carried out before the tremor detection curve of the sequence image is analyzed.
4) And (5) Fourier transform analysis. And carrying out Fourier transform analysis on the tremor relative error curve after the local interpolation to obtain the frequency and the amplitude of the tremor relative error curve.
5) And (5) integral least squares fitting. And taking the frequency and the amplitude obtained in the last step as initial values, and adopting a sine function model to perform integral least square fitting on the tremor relative error curve, so as to accurately estimate the frequency, the amplitude and the initial phase of the relative error.
Step 3, absolute tremor error modeling
And establishing a tremor absolute error model, and obtaining the frequency, amplitude and initial phase of the absolute tremor error according to a corresponding formula. The amplitude of the absolute tremor is the same as the frequency of the tremor relative error, and according to the amplitude and the phase of the tremor relative error, the amplitude and the phase of the absolute tremor can be calculated according to a formula.
One specific example is given below:
fig. 1 shows a schematic flow chart of fine detection of focal plane tremor of a satellite-borne optical camera based on a rolling shutter array CMOS, which mainly comprises the following steps:
Step 1: high precision dense matching. And performing dense matching on the planar array CMOS sequence images according to the imaging time.
Since the overlapping degree of adjacent frames of the area array CMOS image sequence is closely related to the flight state, the overlapping degree is not a fixed value. For each set of sequence effects, therefore, the degree of overlap of images between adjacent frames is first determined, i.e., the area available for tremor detection is first determined. The invention adopts SIFT algorithm to match a small number of homonymous points between two frames of images. And calculating the offset by counting the coordinate differences of the same-name points, namely calculating the initial position of each pixel point of the front sequence image on the rear sequence image. The image points beyond the post-sequence image coordinate area indicate that they are not in the overlapping area.
And then carrying out pixel-by-pixel matching by adopting a correlation coefficient matching and least square matching algorithm, calculating coordinate differences of homonymous points of all pixel points in an overlapping area obtained by the pixel-by-pixel matching in two directions of rows and columns, and generating a coordinate difference map in two directions by taking the coordinate differences as DN values, namely a so-called parallax map. Fig. 2 (a) is a 100HZ parallax map.
Step 2: tremor relative error analysis. The frequency, amplitude and initial phase of the tremor relative error are accurately estimated.
1) A single disparity map error curve is generated. Calculating the average value of each line of the single parallax map as the line relative error, further obtaining a tremor relative error curve along the scanning line, converting the scanning line into time, and obtaining a certain group of tremor relative error curves, see dark gray solid-point curves (tremor frequency of 100HZ of FIG. 2) in FIG. 2 (b), wherein the black solid-line curves in FIG. 2 (b) are integrally fitted curves, and light gray circular curves are fitting errors. Fourier transform analysis was performed on this set of error curves to obtain the fourier analysis result of fig. 2 (c).
If the platform tremble exists, an error with time-varying characteristics is presented in the parallax map, namely, the relative error of two frames of images caused by the platform tremble is presented, and the relative error of the platform tremble can be expressed by a formula:
g(t)＝f(t+Δt)-f(t) (1)
Wherein g (t) represents a function of the relative error of the platform tremor with time, f (t) represents a function of the absolute error of tremor with time, and Δt is the imaging time interval of the homonymous features of the two frames of images.
2) And generating a sequence disparity map error curve. Because tremor frequency is low, a single set of data is difficult to show tremor change trend, so that analysis of sequence images is performed. And combining the tremor relative error curves of the single parallax images through time correlation to form a tremor relative error curve of the sequence image. Due to the size of the matching window, a "gap" appears between the detection results of adjacent images, see fig. 3 (a) (the tremble frequency is 10HZ in fig. 3).
3) Fitting interpolation. Because the size of the matching window causes a gap between the detection results of adjacent images, the data of the gap section is filled, i.e. interpolated, before analyzing the tremor detection curve of the sequence image. The invention adopts two different methods to carry out local fitting interpolation, one is local straight line fitting interpolation, and the other is local spline function fitting interpolation. Knowing the data of two end points of the gap section between adjacent images, the line segment function between two points can be obtained by the data of the two points, the function is generated to fill the gap section, then the corresponding function value is obtained by the known self-variable data, and the function value is inserted into the gap, namely, the so-called linear interpolation method. The spline function is composed of polynomials, each polynomial being determined by two adjacent points. Therefore, a sample function is constructed according to the existing data of the function, and fitting interpolation is further carried out.
The linear interpolation function formula is:
y＝ax+b (2)
Slope a= (vx (k) -vx (k-1))/(t (k) -t (k-1)), b = vx (k) -a x t (k), where vx (k), vx (k-1), t (k), t (k-1) are known function values and arguments.
The cubic spline interpolation function is:
Si(x)＝ai+bi(x-xi)+ci(x-xi)2+di(x-xi)3,i＝0,1,...,n (3)
Where a i,bi,ci,di represents an unknown number. The function S (x) can be calculated from the continuity of the second derivative of S (x) over [ a, b ], satisfying the continuity condition and its interpolation condition at node x i.
Wherein fig. 3 (b) (tremor frequency 10HZ in fig. 3) and fig. 3 (c) are effect graphs of local straight line interpolation fitting and local spline function interpolation fitting, respectively. Meanwhile, the two small images in fig. 3 (b) and fig. 3 (c) are partial enlarged images of the two selected sections with representative fitting results, the first partial enlarged image is the first section of gap (with larger absolute value of slope), and compared with fig. 3 (b) and fig. 3 (c), the effect of straight line fitting can be found to be better, and meanwhile, the first error in fig. 3 (d), namely, the error of the two is larger. The second partial enlarged graph is the third segment of gap (with small absolute value of slope), and comparing fig. 3 (b) and fig. 3 (c), it can be found that the effects of the two interpolation methods are similar, and the second partial enlarged graph corresponds to the third error in fig. 3 (d), that is, the errors of the two interpolation methods are small.
After the method provided by the invention is used for experiments, a proper interpolation method can be selected according to the slope of samples at two ends of a gap, and when the absolute value of the slope is more than 300, the effect of the two interpolation methods is not great, so that partial fitting of a broken curve can be better carried out; when the absolute value of the slope is between 100 and 300, compared with the fitting effect of the spline function, the fitting result of the linear function is closer to the original tremor curve; when the absolute value of the slope is smaller than 100, the spline function is adopted to achieve a better fitting effect.
4) And (5) Fourier transform analysis. And carrying out Fourier transform analysis on the tremor relative error curve after the partial fitting to obtain the frequency and the amplitude of the tremor relative error curve, wherein the frequency and the amplitude are shown in fig. 3 (e) in the drawing.
5) And (5) integral least squares fitting. The frequency and the amplitude of the obtained tremor relative error curve are used as initial values, and the platform tremor can be regarded as harmonic motion superposed by one or more sine functions in a short time, so that the integral least square fitting is carried out on the tremor relative error curve by adopting a sine function model, and the frequency, the amplitude and the initial phase of the relative error are accurately estimated. The implementation mode is as formula (4):
Wherein, f k,Ark is used for the treatment of the heat dissipation, The frequency, amplitude and initial phase of the kth sinusoidal component of the tremor relative error, respectively. Typically, only a single primary frequency of platform tremors is present in the satellite for a short period of time, so only the case of k taking 1 will be discussed in this disclosure.
Step 3: absolute tremor error modeling. And modeling the tremor absolute error, and obtaining the amplitude and the initial phase of the absolute tremor according to a corresponding calculation formula.
According to the motion synthesis principle, the frequency of tremor absolute error is the same as that of tremor relative error. In combination with formulas (1) and (4), it is known that when there is only one tremor frequency, the amplitude of absolute tremor can be calculated according to formula (5):
Ad＝Ar/(2sin(πfΔt)) (5)
Where Ad is the tremor absolute error amplitude, ar is the tremor relative error amplitude, f is tremor frequency, and Δt is the time interval for homonymous point imaging of adjacent images.
The tremor absolute error initial phase is calculated by the formula (6):
where mod (fDeltat/2, 2) represents the remainder of fDeltat/2 divided by 2.
And the absolute error of tremor can be expressed by formula (7):
In summary, the invention provides a tremor detection method for recovering tremor frequency, amplitude and initial phase of a platform with high precision, aiming at the problem that tremor errors of ultra-high resolution images are difficult to accurately compensate due to small difference between tremor parameters at the installation position of an attitude sensor and the imaging focal plane of a camera.
The invention provides a satellite-borne optical camera focal plane tremor fine detection method based on a rolling shutter array CMOS, which comprises three steps of high-precision dense matching, tremor relative error curve analysis and tremor absolute error modeling. The high-precision dense matching comprises the steps of matching homonymous points between two frames of images by adopting a SIFT algorithm, and then carrying out pixel-by-pixel matching by adopting a correlation coefficient matching and least square matching algorithm to obtain homonymous points of all image points in an overlapping area, so as to obtain a parallax image of the images. The tremor relative error analysis comprises single parallax map error curve analysis and sequence image parallax map error curve analysis, gaps of the sequence image curves are filled before the sequence image tremor error curve is analyzed, and local fitting of the curves is performed by using two interpolation methods of linear interpolation fitting and spline function interpolation fitting. And then carrying out Fourier transform analysis on the tremor error curve after the local fitting to obtain the frequency and the amplitude of the tremor relative error curve. And taking the obtained frequency and amplitude as initial values, and adopting a sine function model to perform integral least square fitting on the error curve to obtain the frequency, amplitude and initial phase of tremble relative error. The tremor absolute error modeling, namely according to the amplitude and the phase of tremor relative error, the amplitude and the phase of the absolute tremor can be calculated according to a formula.
Meanwhile, the invention develops a simulation experiment based on real remote sensing data, realizes accurate measurement of periodic tremor errors of an imaging focal plane, and can directly provide a reliable data basis for tremor error compensation of remote sensing images.
It should be noted that each step/component described in the present application may be split into more steps/components, or two or more steps/components or part of operations of the steps/components may be combined into new steps/components, according to the implementation needs, to achieve the object of the present application.
It will be readily appreciated by those skilled in the art that the foregoing is merely a preferred embodiment of the invention and is not intended to limit the invention, but any modifications, equivalents, improvements or alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (8)
1. The fine detection method for the focal plane tremor of the satellite-borne optical camera is characterized by comprising the following steps of:
s1, high-precision dense matching
Determining an image overlapping region between adjacent frames of the planar array CMOS sequence image, and then performing pixel-by-pixel matching to obtain homonymous points of all pixel points in the overlapping region, so as to generate a parallax image;
s2, tremor relative error analysis
Generating a single parallax map tremor relative error curve based on the single parallax map;
Combining a plurality of tremor relative error curves of the single parallax images through time correlation to form tremor relative error curves of the sequence images;
Gaps exist between tremble relative error curves of adjacent images, and a fitting interpolation method is adopted to fill the gaps;
Carrying out Fourier transform analysis on the tremor relative error curve of the filled sequence image to obtain the frequency and the amplitude of the tremor relative error curve;
Taking the obtained frequency and amplitude as initial values, and adopting a sine function model to perform integral least square fitting on tremor relative error curves of the sequence images to obtain the frequency, amplitude and initial phase of tremor relative errors;
S3, tremor absolute error modeling
Modeling the tremor absolute error, and combining the frequency, the amplitude and the initial phase of the tremor relative error to obtain the frequency, the amplitude and the initial phase of the tremor absolute error, wherein the tremor absolute error modeling comprises the following steps:
The frequency of the tremor absolute error is the same as the frequency of the tremor relative error;
when there is only one tremor frequency, the absolute tremor amplitude is calculated as follows:
Ad＝Ar/(2sin(πfΔt))
Wherein Ad is the amplitude of tremor absolute error, ar is the amplitude of tremor relative error, f is tremor frequency, and Δt is the time interval of homonymous point imaging of adjacent images;
the initial phase calculation formula of tremor absolute error is as follows:
wherein, Representing the initial phase of the absolute error of tremor,Representing the initial phase of tremor relative error, mod (fΔt/2, 2) represents the remainder of fΔt/2 divided by 2;
the absolute error of tremors is expressed as follows:
Where D (t) represents the function of tremor absolute error over time.
2. The fine detection method of focal plane tremor of an on-board optical camera according to claim 1, wherein determining an image overlap region between adjacent frames of an area array CMOS sequence image comprises:
matching at least one homonymy point between two adjacent frames of images of the planar array CMOS sequence image by adopting a SIFT algorithm;
counting the coordinate difference of the same-name points between two adjacent frames of images to obtain offset;
According to the offset, calculating the initial position of each pixel point of the front sequence image on the rear sequence image; if the initial position of a pixel point exceeds the region of the rear sequence image, the pixel point is not in the overlapping region.
3. The fine detection method of focal plane tremor of an on-board optical camera according to claim 1, wherein pixel-by-pixel matching is performed by adopting correlation coefficient matching and least squares matching algorithms.
4. The method for fine detection of focal plane tremor of an on-board optical camera of claim 1, wherein generating a disparity map comprises: and calculating coordinate differences of the same name points of all pixel points in the overlapping area of two adjacent frames of images in the row direction and the column direction, and generating a coordinate difference map in the row direction and the column direction by taking the coordinate differences as DN values.
5. The method of claim 1, wherein generating a single disparity map tremor versus error curve based on a single disparity map comprises:
Calculating the average value of each row of the single parallax map as the relative error of the row, and further obtaining a tremor relative error curve along the scanning row; and converting the scanning line into time to obtain a tremor relative error curve.
6. The fine detection method of focal plane tremor of an on-board optical camera according to claim 1, wherein the method of fitting interpolation includes straight line function interpolation fitting and spline function interpolation fitting.
7. The fine detection method for focal plane tremor of an on-board optical camera according to claim 6, wherein when the absolute value of the slope of samples at two ends of a gap is greater than 300, straight line function interpolation fitting or spline function interpolation fitting is adopted; when the absolute value of the slope of the samples at the two ends of the gap is between 100 and 300, adopting linear function interpolation fitting; and when the absolute value of the slope of the samples at the two ends of the gap is smaller than 100, adopting spline function interpolation fitting.
8. The fine detection method of focal plane tremor of an on-board optical camera according to claim 1, wherein the sine function model is as follows:
Wherein, f k,Ark is used for the treatment of the heat dissipation, The frequency, amplitude and initial phase of the kth sinusoidal component of the tremor relative error, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111502706.4A CN114170192B (en) | 2021-12-10 | 2021-12-10 | Fine detection method of focal plane jitter of spaceborne optical camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111502706.4A CN114170192B (en) | 2021-12-10 | 2021-12-10 | Fine detection method of focal plane jitter of spaceborne optical camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114170192A CN114170192A (en) | 2022-03-11 |
| CN114170192B true CN114170192B (en) | 2024-11-15 |
Family
ID=80485227
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111502706.4A Active CN114170192B (en) | 2021-12-10 | 2021-12-10 | Fine detection method of focal plane jitter of spaceborne optical camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114170192B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117475087B (en) * | 2023-12-22 | 2024-04-26 | 武汉工程大学 | Satellite-borne TDI CCD image tremor imaging distortion modeling method and device |
| CN118134837B (en) * | 2024-01-05 | 2024-09-20 | 武汉工程大学 | Remote sensing satellite multi-frequency tremor detection and modeling method based on non-collinear CCD |
| CN118310635B (en) * | 2024-06-11 | 2024-08-13 | 山西创芯光电科技有限公司 | Focal plane detection method and system for infrared detector |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016106959A1 (en) * | 2014-12-30 | 2016-07-07 | 华中科技大学 | Multi-parameter model guided-method for iteratively extracting movement parameter of angiography image of blood vessel |
| CN110796641A (en) * | 2019-10-08 | 2020-02-14 | 武汉大学 | High-resolution satellite image tremor detection method based on continuous snapshot model |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101905716B1 (en) * | 2017-03-06 | 2018-10-11 | 순천향대학교 산학협력단 | Video-oculography nystagmus analytics apparatus using infrared camera and method thereof |
-
2021
- 2021-12-10 CN CN202111502706.4A patent/CN114170192B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016106959A1 (en) * | 2014-12-30 | 2016-07-07 | 华中科技大学 | Multi-parameter model guided-method for iteratively extracting movement parameter of angiography image of blood vessel |
| CN110796641A (en) * | 2019-10-08 | 2020-02-14 | 武汉大学 | High-resolution satellite image tremor detection method based on continuous snapshot model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114170192A (en) | 2022-03-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114170192B (en) | Fine detection method of focal plane jitter of spaceborne optical camera | |
| Saurer et al. | Rolling shutter stereo | |
| US8350922B2 (en) | Method to compensate the effect of the rolling shutter effect | |
| US8111294B2 (en) | Hybrid image stabilization method and apparatus | |
| CN102478759A (en) | An integrated measurement method for space camera wavefront distortion and optical axis jitter | |
| CN104864853B (en) | A kind of high-resolution three line scanner satellite along the attitude flutter of rail direction detection method | |
| Pan et al. | Precise georeferencing using the rigorous sensor model and rational function model for ZiYuan-3 strip scenes with minimum control | |
| KR101890612B1 (en) | Method and apparatus for detecting object using adaptive roi and classifier | |
| CN112050806B (en) | Positioning method and device for moving vehicle | |
| US20180075614A1 (en) | Method of Depth Estimation Using a Camera and Inertial Sensor | |
| US9794483B1 (en) | Video geolocation | |
| CN115330641B (en) | Image restoration method, system and device based on MHD angular velocity sensor | |
| CN102576459B (en) | Method for detecting shifts in line images obtained by a sensor that is airborne or moving in space | |
| JP5162412B2 (en) | Tunnel wall surface photographing device | |
| JP2017059998A (en) | Image processing apparatus and method, and imaging device | |
| KR101770745B1 (en) | Method for evaluation of fusion feasibility on multi-sensor satellite images and Apparatus Thereof | |
| EP4235574A1 (en) | Measuring device, moving device, measuring method, and storage medium | |
| CN104296726B (en) | Two-sided battle array pushes away sweeps stereo mapping formation method and imaging system | |
| CN113870321B (en) | A method and system for simulating vibration changes of remote sensing images | |
| Türkmenoğlu et al. | On-orbit modulation transfer function estimation For RASAT | |
| Zhu et al. | Rigorous parallax observation model-based remote sensing panchromatic and multispectral images jitter distortion correction for time delay integration cameras | |
| JP6207435B2 (en) | Moving body mounted photography system | |
| US20250078312A1 (en) | System and method for depth determination | |
| JP5409451B2 (en) | 3D change detector | |
| Reulke et al. | Improvement of spatial resolution with staggered arrays as used in the airborne optical sensor ADS40 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |