CN112393807B - Infrared image processing method, device, system and computer readable storage medium - Google Patents
Infrared image processing method, device, system and computer readable storage medium Download PDFInfo
- Publication number
- CN112393807B CN112393807B CN202011323560.2A CN202011323560A CN112393807B CN 112393807 B CN112393807 B CN 112393807B CN 202011323560 A CN202011323560 A CN 202011323560A CN 112393807 B CN112393807 B CN 112393807B
- Authority
- CN
- China
- Prior art keywords
- column
- correction
- infrared image
- data
- black body
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 19
- 238000003860 storage Methods 0.000 title abstract description 10
- 238000012937 correction Methods 0.000 claims abstract description 148
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000005457 optimization Methods 0.000 claims description 27
- 238000007781 pre-processing Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 7
- 238000009499 grossing Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 abstract description 5
- 230000004927 fusion Effects 0.000 abstract description 5
- 230000000694 effects Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000001931 thermography Methods 0.000 description 4
- 238000013500 data storage Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000005855 radiation Effects 0.000 description 3
- 241000282414 Homo sapiens Species 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000036760 body temperature Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003331 infrared imaging Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/10—Radiation pyrometry, e.g. infrared or optical thermometry using electric radiation detectors
- G01J5/12—Radiation pyrometry, e.g. infrared or optical thermometry using electric radiation detectors using thermoelectric elements, e.g. thermocouples
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/10—Radiation pyrometry, e.g. infrared or optical thermometry using electric radiation detectors
- G01J5/12—Radiation pyrometry, e.g. infrared or optical thermometry using electric radiation detectors using thermoelectric elements, e.g. thermocouples
- G01J2005/126—Thermoelectric black plate and thermocouple
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Radiation Pyrometers (AREA)
Abstract
The invention relates to the technical field of infrared image processing, and provides an infrared image processing method, an infrared image processing device, an infrared image processing system and a computer readable storage medium. The method comprises the steps of performing row-column stripe correction based on row-average data of black body data, namely judging the temperature of the row stripe to obtain the row stripe by utilizing the row-average data of the black body data according to the gray value of an infrared image; and then removing the column stripes from the infrared image output by the non-uniformity correction, and outputting the infrared image corrected by the column stripes. The method is based on the fusion correction of the combination of the multi-section two-point and the column fringe correction in different temperature intervals, has fewer data points compared with the existing multi-point correction, reduces the complexity of operation and the correction accuracy, does not need to increase the DDR read-write function, reduces the memory and the complexity, and can further improve the correction precision and the correction accuracy.
Description
Technical Field
The invention relates to the technical field of infrared image processing, in particular to correction of an infrared image based on FPGA (field programmable gate array) memory optimization, and specifically relates to an infrared image processing method, device and system and a computer readable storage medium.
Background
The infrared thermal imaging technology effectively widens the vision range of human beings, and has wide application in military and civil fields. Infrared thermal imaging uses a photoelectric technology to detect infrared specific wave band signals of object thermal radiation, and the signals are converted into images and graphs which can be distinguished by human vision.
In an infrared thermal imaging detection system, radiation is mainly detected through a detector, signal processing is carried out through a processing circuit at the rear end, thermal radiation signals are converted into distribution of temperature/temperature fields, and visual images and graphs and dynamic change processes of the images and the graphs are formed. Irregular stripes in the infrared image are the embodiment of the nonuniformity of the infrared focal plane array, and the generated nonuniformity is a key factor for restricting the imaging quality of the infrared imaging system. However, the reason for generating the non-uniformity is very complicated, it is not practical to completely eliminate the non-uniformity in the production process of the imaging system/instrument, and the currently common mode is to improve the performance of the imaging system/instrument by non-uniformity correction, effectively remove the non-uniformity of the image, compensate the non-uniformity of the response of the detector, remove the low-frequency fixed mode noise, and improve the image quality.
In the prior art, two-point correction is most commonly used in a correction method based on black body calibration. At present, a mature infrared thermal imaging detection system (infrared camera) sold in the market usually calculates the correction coefficient of each detection element at the temperature of a plurality of focal plane arrays in a two-point correction mode, then stores the correction coefficient into a register of an FPGA (field programmable gate array), and performs interpolation calculation according to the current temperature of the FPGA in actual acquisition to obtain a corresponding correction coefficient so as to eliminate noise.
In the further optimization process, the problem of column stripes existing in two-point correction is improved, and the multi-point correction can solve the problem of column stripes which do not completely appear in the two-point correction, but needs to store (n +1) × imax*jmaxA data memory, wherein n is the number of temperature intervals, imax，jmaxThe number of rows and columns of the pixel points needs to be stored by the FPGA, so that more parameters are stored, a large amount of FPGA memory is occupied, and the method inevitably increases power consumption.
Disclosure of Invention
The first aspect of the present invention is to provide an infrared image processing method based on FPGA memory optimization, which is based on the fusion correction of column fringe correction combining multiple segments of two points and different temperature intervals, and compared with the existing multipoint correction, the method has the advantages of selecting fewer temperature points, reducing the complexity of operation and the accuracy of correction, and being used as a supplement or a substitute for the multipoint correction (multiple segments of two points).
Furthermore, on the basis of the first aspect of the invention, the column stripe correction is optimized, and the column stripe optimization correction is also judged based on the black body column mean value data, so that the memory occupation of the FPGA is reduced, meanwhile, the memory can be performed in the chip of the FPGA, the DDR read-write function is not required to be added, the memory use and complexity are reduced, and the correction accuracy and precision are further improved.
According to a first aspect of the present invention, an infrared image processing method is provided, comprising the steps of:
step 1, black body data of different temperatures collected by an infrared detector are obtained;
step 2, preprocessing an image;
step 4, correcting the blackbody data based on the gain coefficient and the offset coefficient to obtain corrected blackbody data at different temperatures, and acquiring corresponding row stripes at different temperatures;
and 6, judging the temperature of the row stripes by using the temperature data of the black body according to the gray value of the infrared image, obtaining the row stripes at the temperature, and correcting the row stripes of the infrared image to obtain the infrared image output without the row stripes.
According to a second aspect of the present invention, an infrared image processing method based on FPGA memory optimization is provided, which includes the following steps:
step 1, black body data of different temperatures collected by an infrared detector are obtained;
step 2, preprocessing an image;
step 4, correcting the blackbody data based on the gain coefficient and the offset coefficient to obtain corrected blackbody data at different temperatures, and acquiring corresponding row stripes at different temperatures;
step 6, calculating the column mean value data of the black body data, and then judging the temperature of the column stripes according to the gray value of the infrared image and by using the column mean value data of the black body data to obtain the column stripes; and then removing the column stripes from the infrared image output by the non-uniformity correction, and outputting the infrared image corrected by the column stripes.
In a preferred embodiment, in step 6, instead of performing the column stripe correction based on the blackbody data in the first aspect, the performing the optimization correction by using the column average of the blackbody data specifically includes:
1) calculating blackbody line mean value data E 'after removing blind pixels'TAccording to YnThe gradation value of (i, j) is E'TJudging the temperature T of the row stripe to obtain a row stripe DT。
2) Based on Y'n(i,j)＝Yn(i,j)-DTObtaining a scene graph Y 'from which the column stripes are removed'n(i,j)。
Therefore, in the infrared image processing method based on the FPGA memory optimization provided by the second aspect of the invention, the column stripe correction is optimized on the basis of multi-segment two-point correction and column stripe correction in different temperature intervals, and the column stripe optimization correction is also judged based on the black body column mean value data, so that the memory occupation of the FPGA is reduced, meanwhile, the memory can be performed in the FPGA chip, the DDR read-write function does not need to be added, the memory use and complexity are reduced, and the correction accuracy and precision are further improved.
According to a third aspect of the present invention, an infrared image processing apparatus based on FPGA memory optimization is provided, including:
the module is used for acquiring black body data of different temperatures acquired by the infrared detector;
a module for image pre-processing;
a module for obtaining a gain coefficient and an offset coefficient by adopting a multi-point correction (including two-point correction) mode based on the preprocessed black body data;
the module is used for correcting the blackbody data based on the gain coefficient and the offset coefficient to obtain the corrected blackbody data at different temperatures and obtain corresponding row stripes at different temperatures;
a module for non-uniformly correcting the input infrared image based on the gain coefficient and the bias coefficient; and
the module is used for correcting the column stripes based on the column mean data of the black body data, namely judging the temperature of the column stripes to obtain the column stripes according to the gray value of the input infrared image and by using the column mean data of the black body data; and then removing the column stripes from the infrared image output by the non-uniformity correction, and outputting the infrared image corrected by the column stripes.
According to a fourth aspect of the present invention, there is provided a computer system comprising:
one or more processors;
a memory storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising performing processes of the infrared image processing method.
According to a fifth aspect of the object of the present invention, a computer-readable medium storing software is proposed, the software comprising instructions executable by one or more computers, the instructions, when executed by the one or more computers, performing a process of an infrared image processing method.
Compared with the prior art, the technical scheme of the invention has the following remarkable advantages that:
1) compared with the multi-point correction scheme in the prior art, the multi-point correction method has the advantages that the number of temperature points required to be selected is small, the calculation amount and the calculation process are reduced, the memory occupation is reduced, the power consumption is reduced, new fusion correction is formed by combining the combination of the column stripe correction in different temperature intervals, and the coordination accuracy of the column stripes is improved;
2) the invention further provides the method for judging based on the black body array mean value instead of the black body data, so that the memory occupation of the FPGA is reduced, meanwhile, the memory can be performed in the chip of the FPGA, the DDR read-write function does not need to be added, the memory and the complexity are reduced, and the correction accuracy is further improved.
It should be understood that all combinations of the foregoing concepts and additional concepts described in greater detail below can be considered as part of the inventive subject matter of this disclosure unless such concepts are mutually inconsistent. In addition, all combinations of claimed subject matter are considered a part of the presently disclosed subject matter.
The foregoing and other aspects, embodiments and features of the present teachings can be more fully understood from the following description taken in conjunction with the accompanying drawings. Additional aspects of the present invention, such as features and/or advantages of exemplary embodiments, will be apparent from the description which follows, or may be learned by practice of specific embodiments in accordance with the teachings of the present invention.
Drawings
The drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures may be represented by a like numeral. For purposes of clarity, not every component may be labeled in every drawing. Embodiments of various aspects of the present invention will now be described, by way of example, with reference to the accompanying drawings, in which:
FIG. 1 is a flowchart of an FPGA-based optimized memory based non-uniform correction de-striping IR image processing in an exemplary embodiment of the present invention;
fig. 2 is an infrared image of a black body at a temperature T of 15 ° corrected for two points;
fig. 3 is an infrared image of a black body at a temperature T of 15 ° subjected to two-point correction + de-striping;
fig. 4 is an infrared image of a black body at a temperature T of 15 ° subjected to two-point correction + column fringe optimization;
fig. 5 is an infrared image of a black body at a temperature T of 15 ° subjected to multipoint correction;
FIG. 6 shows C at a temperature T of 15 °T、CT'、DT；
FIG. 7 is a two-point corrected scene graph;
FIG. 8 is a scene graph after two-point correction + de-alignment of stripes;
FIG. 9 is a scene graph after two-point correction + de-column streak optimization;
FIG. 10 is a view of a multi-point corrected scene.
Detailed Description
In order to better understand the technical content of the present invention, specific embodiments are described below with reference to the accompanying drawings.
In this disclosure, aspects of the present invention are described with reference to the accompanying drawings, in which a number of illustrative embodiments are shown. Embodiments of the present disclosure are not necessarily intended to include all aspects of the invention. It should be appreciated that the various concepts and embodiments described above, as well as those described in greater detail below, may be implemented in any of numerous ways, as the disclosed concepts and embodiments are not limited to any one implementation. In addition, some aspects of the present disclosure may be used alone, or in any suitable combination with other aspects of the present disclosure.
The infrared image processing method proposed in conjunction with the exemplary embodiment shown in fig. 1 is intended to implement column fringe optimization and uniform correction on an input infrared image, implement correction on the infrared image, and provide image quality. The infrared image processing method as illustrated in fig. 1 includes the steps of:
step 1, black body data of different temperatures collected by an infrared detector are obtained;
step 2, preprocessing an image;
step 4, correcting the blackbody data based on the gain coefficient and the offset coefficient to obtain corrected blackbody data at different temperatures, and acquiring corresponding row stripes at different temperatures;
and 6, judging the temperature of the row stripes by using the temperature data of the black body according to the gray value of the infrared image, obtaining the row stripes at the temperature, and correcting the row stripes of the infrared image to obtain the infrared image output without the row stripes.
Optionally, in the step 1, the interval T is collected at a set temperature0For the period, black body data E corresponding to different temperatures T are collectedT. The temperature acquisition interval T0Set to 5 °, the range of the collected different temperatures T is 5 ° to 40 °, i.e.:
T＝5°,10°,15°,20°,25°,30°,35°,40°。
in step 2, image preprocessing is performed to perform preprocessing on the input black body image, such as blind pixel removal, median filtering, noise reduction, and the like. In the embodiment of the invention, the dead pixel removal is taken as an example to remove the dead pixel in the image.
In step 3, the gain coefficient and the bias coefficient are determined according to the multipoint correction (including two-point correction), including the following operations:
step S31 according to
Calculating a gain factor an(i,j)；
Step S32 according to
Calculating a bias coefficient bn(i,j)；
Wherein, VHRepresenting the average gray value of all pixel points of black body data collected at high temperature;
VLrepresenting the average gray value of all pixel points of black body data collected at low temperature;
Preferably, in the step 4, the column stripes D corresponding to different temperatures T are obtained by the following operationsT：
S41, taking the average value of each column of the preprocessed black body data to obtain a column average value vector C of the black body data at the corresponding temperatureT；CT＝CT1,CT2,...,CTj；CTjExpressed as the mean value of the jth column of the image at temperature T, jmaxThe column number of the pixel points of the black body data corresponding to the black body diagram is shown;
step S42, the obtained column mean vector CTSmoothing is carried out to remove the stripe quantity, and the column mean vector after smoothing is named as CT'，CT'＝CT1',CT2',...,CTj'；CTj' is the value of the smoothed jth point; and
step S43, subtracting the column mean vector of the black body data at the corresponding temperature from the smoothed column mean vector to obtain the column stripe D at the corresponding temperatureT，DT＝CT-CT'，D＝D1,D2,...,Dj。
In an alternative embodiment, the non-uniformity correction of the infrared image in said step 5 comprises the following operations:
step S51, according to Xn(i, j) judging the temperature interval of the gray value of the black body data by using the temperature data of the black body data to obtain the corresponding an(i,j)、bn(i,j)；
Step S52, based on Yn(i,j)＝an(i,j)×Xn(i,j)+bn(i, j) non-uniformity correcting the infrared image and outputting Yn(i,j)；
Wherein, Xn(i, j) represents the output of the detecting element (i, j) before correction, namely an infrared image before correction;
Yn(i, j) represents the corrected output, i.e., the non-uniformity corrected infrared image;
an(i, j) and bn(i, j) respectively represent the corresponding gain coefficient and bias coefficient determined from the input infrared image.
Preferably, in the step 6, the temperature of the row of stripes is determined by using the temperature data of the black body according to the gray value of the infrared image, the row of stripes at the temperature is obtained, and the infrared image is subjected to row stripe correction to obtain the infrared image output with the row of stripes removed, and the specific processing procedure includes:
step S61, outputting infrared image Y according to non-uniformity correctionn(i, j), judging the temperature T of the row stripe by using the temperature data of the black body data to obtain a row stripe DT；
Step S62, according to Y'n(i,j)＝Yn(i,j)-DTRemoving the column stripes DTAnd outputs Y'n(i, j); wherein, Y'n(i, j) represents an infrared image output after the column stripes are removed.
The above implementation will be described in more detail with reference to the drawings.
In the following embodiment of the invention, on one hand, a fusion correction method of multi-point correction (including two-point correction) and column fringe correction is explained, the fusion correction is further optimized, and the black body column mean value is used for replacing black body data for judgment, so that the memory occupation of the FPGA is reduced, the power consumption is reduced, and the correction precision and accuracy are improved.
On the other hand, we compare the correction method optimized for multi-point and column fringe blending with the multi-point correction (8-point correction) of the prior art, and illustrate the significant benefits and advances of the present invention from memory footprint and correction effects (in 15 ° bold and scene).
S1, collecting blackbody data E of different temperatures T by using a detectorT. The blackbody data collected by the detector in this embodiment is 512 x 640, 14 bits, [5 ° -40 ° ]]The temperature interval can be 5 degrees for once collection; the average of 100 plots for each temperature was taken as the black body data at that temperature.
Blackbody data E of T is acquired in this embodimentTThe data amount is 8 × 512 × 640, and the data is used as subsequent storage data.
S2, image preprocessing, such as blind pixel removal and the like, and dead pixels in the image are removed. The present embodiment employs blind pixel replacement.
S3, obtaining a gain coefficient a by adopting a multi-section two-point correction moden(i, j), bias coefficient bn(i, j). In the embodiment, a two-point correction and a row stripe correction algorithm in a plurality of corrections are adopted to compare with a multi-point correction algorithm (8 points are selected for multi-point correction in the embodiment, and 8 points are selected for correction); the two point corrected temperatures were 5 ° and 40 °; and a multipoint correction mode is adopted to compare with the scheme of the invention; the temperatures for the multi-point calibration were:
T＝5°,10°,15°,20°,25°,30°,35°,40°
in the operation of S3, the multipoint correction (including two-point correction) is performed as follows:
s31, according to
Calculating an(i,j)。
S32, according to
Calculation of bn(i,j)。
VHRepresenting the average gray value of all pixel points of black body data collected at high temperature;
VLrepresenting the average gray value of all pixel points of black body data collected at low temperature;
n represents the number of intervals of the multipoint correction.
In step S3, the corresponding correction coefficients in the present embodiment are a (i, j) and b (i, j); for multipoint correction corresponds to a8(i, j) and b8(i,j)。
S4, utilizing a obtained after correctionn(i,j),bn(i, j) correcting the blackbody image to obtain corrected blackbody images at different temperatures; obtaining corresponding row stripes D at different temperatures TT. Column stripe D of T in this embodimentTThe data amount is 8 × 640.
Specifically, in S4, the following stripes corresponding to different temperatures T are obtained, and the specific processing steps are as follows:
and S41, preprocessing the blackbody image (blackbody data), such as blind pixel replacement, multipoint correction and blind pixel removal processing. The embodiment adopts two-point correction and blind pixel removal processing, fig. 2 is a blackbody diagram with T being 15 degrees and subjected to two-point correction, and a in the embodimentn(i,j),bn(i, j) corresponds to a (i, j), b (i, j). While figure 5 is given as a comparison a multipoint corrected bold image of T-15 deg..
S42, taking the average value of each row of the preprocessed black body to obtain a row average value vector C of the black body at the temperature TT；CT＝CT1,CT2,...,CTj；CTjIs the average value of j in the j th column of the image when the black body temperature is TmaxThe number of columns of image pixels. In the present embodiment, jmax＝640。
S43, the obtained column mean value vector C is subjected toTSmoothing is carried out, the stripe amount is removed, and the smoothing mode is not limited; such as median fitting, gaussian fitting, etc. The processed vector is named CT'，CT'＝CT1',CT2',...,CTj'；CTj' is the value of the smoothed jth point.
S44, subtracting to obtain the column stripe D at the temperatureTThe method comprises the following specific operations: dT＝CT-CT'，D＝D1,D2,...,Dj。
In the present embodiment, fig. 6 shows that C corresponding to the temperature T of 15 ° is obtained in accordance with steps S42, S43, and S44T、CT'、DT。
And S5, carrying out non-uniform correction on the picture. The method comprises the following specific steps:
s51, according to XnThe gray value of (i, j) is judged in the temperature interval by using the temperature data of the black body to obtain an(i,j)，bn(i, j). In the present embodiment, two points are corrected, and only 1 group a (i, j), b (i, j) is provided. The multipoint correction has 8 groups a8(i, j) and b8(i,j)。
S52, obtaining a corrected scene graph Y according to the scene graphn(i,j)。
And S6, correcting the column stripes of the picture. The specific steps for removing the column stripes are as follows:
s61, according to Yn(i, j) grayscale value using temperature data E of black bodyTJudging the temperature T of the row stripe to obtain a row stripe DT. Wherein the judging method is not limited; for example, the temperature range is judged according to the bisection method.
S62. according to Y'n(i,j)＝Yn(i,j)-DTSubtracting the data to obtain an output Y 'from which the column stripes are removed'n(i,j)。
It should be understood that the non-uniformity correction and the column-fringe correction sequence corresponding to the foregoing steps S5 and S6 can be performed sequentially or reversely.
According to S61 and S62, Y is utilized in the present embodimentnAnd (i, j) determining the gray value of the (i, j), wherein the closer the gray value is, the Y isnThe column stripe D corresponding to the j column is subtracted from (i, j) in (i, j)T。
In the present embodiment, two-point correction, multi-point correction, and two-point + column streak correction are performed using a blackbody diagram and a scene diagram of T15 °, respectively. Corresponding to figures 2, 5, 3 and figures 7, 10, 8 of the accompanying description.
In another embodiment, a further improvement is made on the basis of the previous embodiment, and data judgment is performed by changing the temperature data of the black body into the average value of the black body column after preprocessing such as blind pixel removal and the like; compared with the traditional two-point correction and multi-point correction, the method can further reduce the data storage amount and reduce the memory occupation of the FPGA.
As an embodiment of the second aspect of the present invention, the foregoing embodiment is optimized to replace the blackbody data adopted in step 6 of the foregoing embodiment to perform column fringe correction, and the specifically optimized column fringe correction steps are as follows:
s71, calculating column mean value data E 'of blackbody data'TAccording to YnThe gradation value of (i, j) is E'TJudging the temperature T of the row stripe to obtain a row stripe DT。
The black body column average E 'of 5 °,10 °,15 °,20 °,25 °,30 °,35 °,40 ° is obtained in this example'TThe data volume is 8 × 640; this data is used as data for storage. In contrast to blackbody data E for optimized S61TThe data volume is 8 × 512 × 640, and the memory occupation is obviously reduced by adopting the effect obtained by the optimization.
S72. finally, according to Y'n(i,j)＝Yn(i,j)-DTTo obtain output Y 'with the column stripes removed'n(i, j), namely the infrared image output after the column fringe correction.
According to S71 and S72, Y is utilized in the present embodimentn(i, j) is judged to be the black body line average value E'T(of 5 °,10 °,15 °,20 °,25 °,30 °,35 °,40 °) Ti, j), the closer the grey value is, YnThe column stripe D corresponding to the j column is subtracted from (i, j) in (i, j)T。
For the convenience of comparison with the steps S6(S61, S62) of the previous embodiment, steps S71, S72 are used herein for comparative explanation and to prevent confusion. It should be understood that in the present embodiment, on the basis of the foregoing embodiment, the column streak optimization correction processing of steps S71 and S72 is adopted to replace the foregoing implemented S6, and a solution of the second aspect of the present invention is formed.
Therefore, in the infrared image processing method based on the FPGA memory optimization provided by the second aspect of the invention, the column stripe correction is further optimized on the basis of the multi-segment two-point correction and the column stripe correction in different temperature intervals, the column stripe optimization correction processing is judged based on the black body column mean value data, the occupation of the FPGA memory can be further reduced, meanwhile, the FPGA on-chip memory can be used, the DDR read-write function does not need to be added, the memory use and complexity are reduced, and the correction accuracy and precision are further improved.
In the present embodiment, two-point + column fringe correction optimization correction is performed using a blackbody diagram and a scene diagram of T-15 °. Fig. 4 and 9 are explained for the drawings.
To illustrate that the optimization is more accurate, the column stripes D are compared by means of a corresponding temperature lookup tableTAccuracy is expressed in terms of the black body data and the black body column mean. The method comprises the following specific steps:
s81, steps S61 and S71 are paired to last Y'n(i, j) the temperature of each point (i, j) is recorded, with the following steps:
s811 according to Yn(i, j), judging the temperature T of the row stripes by using the temperature data of the black body, and recording the T corresponding to each point (i, j); a correspondence table T (i, j) relating to the temperature can be obtained;
s812, according to YnThe gradation value of (i, j) is E'TJudging the temperature T of the row stripes; the corresponding T of each point (i, j) is recorded to obtain a corresponding table T related to the temperaturec(i,j)；
S82, selecting blackbody data test at a certain temperature T(ii) a Looking up T (i, j) and T corresponding to each point (i, j)c(i, j) accuracy.
Coordination is performed in this embodiment example using a black body at a temperature T of 15 °; counting the number of T (15 degrees) in the corresponding table T (i, j); statistical correspondence table TcThe number of T in (i, j) is 15 °.
In the present embodiment, the two-point + column fringe correction and the two-point + column fringe correction optimization effects are compared by using the black body data with the black body T of 15 °; the correction effect graphs are the graphs in figure 3 and figure 4 in the description of the figures; the accuracy of the correction is shown in table 1 below:
table 1 comparison of accuracy of two-point + column fringe correction and two-point + column fringe correction using black body data with a black body T of 15 °
| 15 ° blackbody data | Black body (first aspect of the invention) | Black body row stripe (second aspect of the invention) |
| The number of T ═ 15 ° in the correspondence table | T(i,j)＝304313 | Tc(i,j)＝325365 |
| Rate of accuracy | 92.87％ | 99.29％ |
The two-point correction + fringe correction map in fig. 3 and the two-point correction + fringe optimization correction map in fig. 4 both remove the fringes caused by the two-point correction in fig. 2; meanwhile, the accuracy of the optimized correction by the blackbody column stripes is higher than that of the correction by the blackbody data alone according to the table data.
A pretreatment step: preprocessing can obtain a corrected gain coefficient an(i, j) and bias coefficient bn(i, j) and the column fringe coefficients D at different temperaturesTAnd black body data ETAnd blackbody column mean data E'TFor the memory to store parameters.
In which temperature T is 15 ° corresponding to C according to fig. 6T、CT'、DTIt can be seen that step S4 is a method for obtaining different temperature interval row stripes; to CTSmoothening to obtain CT'；CT-CT' the streaks at this temperature are obtained.
The processing steps are as follows: the preprocessing is applied to obtain data, and the preprocessing data can be directly applied to the scene graph to perform multipoint correction and column fringe correction.
Preprocessing can result in corrected gain and offset coefficients and column fringe coefficients at different temperatures and blackbody data and blackbody column mean data for the memory storage parameters.
It is worth mentioning that to reduce the amount of memory, a is calculated from the blackbody datan(i, j) and bn(i, j). In this embodiment, the parameters stored in the FPGA memory are divided into four parts: for two-point correction, the data are stored as a (i, j), b (i, j); the amount of data stored for the two-dot + column stripe correction is the black body data ETAnd column stripes DT(ii) a Data amounts a (i, j), b (i, j) and blackbody column mean value data E 'stored after optimization of two-point + column stripes by S71.S72'TRow stripe DT(ii) a The amount of data stored for the multi-point correction is blackbody data ET。
The following table 2 shows the storage data and the storage amount corresponding to the four correction methods, and in the implementation example, the data storage amount after the two-point + column stripe correction optimization is far smaller than the data storage amount of the multiple points, so that the effect is obviously enhanced. According to the two-point correction + column stripe optimization images 4 and 9, the effect is obviously better than that of the two-point image 2 and 7, and the column stripes in the black body and scene images are removed; the storage capacity is only 1.4% more than that of two points; ratio of effects at the same timeThe stripes of two points and rows are good, the accuracy rate is high, and meanwhile, the memory occupation is small; for multiple points, the comparison of scene graphs 9 and 10 shows that the effects are basically consistent, and compared with the storage capacity of the following table, the memory of the two-point + row stripe optimization occupies 2.53% of the memory of the multiple points; therefore, the method of the invention reduces the memory occupation of the FPGA and the power consumption, and can be used as supplement or substitute for multipoint correction. 6 temperature T15 ° corresponding to CT、CT'、DT。
TABLE 2 comparison of stored data and memory space corresponding to four correction methods
From the above exemplary embodiments, the infrared image processing method proposed by the present invention can also be configured to be implemented in the following manner.
Infrared image processing device
An infrared image processing apparatus as an example includes:
the module is used for acquiring black body data of different temperatures acquired by the infrared detector;
a module for image pre-processing;
a module for obtaining a gain coefficient and a bias coefficient by adopting a multipoint correction mode based on the preprocessed black body data;
the module is used for correcting the blackbody data based on the gain coefficient and the offset coefficient to obtain the corrected blackbody data at different temperatures and obtain corresponding row stripes at different temperatures;
a module for non-uniformly correcting the infrared image based on the gain coefficient and the bias coefficient; and
the module is used for correcting the column stripes based on the column mean data of the black body data, namely judging the temperature of the column stripes to obtain the column stripes according to the gray value of the infrared image and the column mean data of the black body data; and then removing the column stripes from the infrared image output by the non-uniformity correction, and outputting the infrared image corrected by the column stripes.
Computer system
An exemplary computer system comprises:
one or more processors;
a memory storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising performing processes of the infrared image processing method of any of the preceding embodiments.
In an optional embodiment, the computer system is, in particular, an infrared image processing computer system based on FPGA memory optimization.
In further embodiments, the computer system may be configured to be implemented in the form of a locally deployed client or a cloud deployed server.
Computer readable medium
An exemplary computer-readable medium stores software including instructions executable by one or more computers, the instructions when executed by the one or more computers performing the processes of the infrared image processing method of any one of the foregoing embodiments.
Although the present invention has been described with reference to the preferred embodiments, it is not intended to be limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, the protection scope of the present invention should be determined by the appended claims.
Claims (6)
1. An infrared image processing method based on FPGA memory optimization is characterized by comprising the following steps:
step 1, obtaining black body images collected by an infrared detector, wherein the black body images comprise black body data with different temperatures;
step 2, preprocessing the blackbody data, wherein blind pixel removal processing is adopted to remove dead pixels;
step 3, based on the preprocessed blackbody data, obtaining a gain coefficient and a bias coefficient by adopting a multipoint correction mode;
step 4, correcting the blackbody data based on the gain coefficient and the offset coefficient to obtain corrected blackbody data at different temperatures, and acquiring corresponding row stripes at different temperatures;
step 5, performing non-uniform correction on the input infrared image based on the gain coefficient and the bias coefficient; and
and 6, performing column fringe correction based on column mean data of the blackbody data, namely: judging the temperature of each column of the infrared image according to the gray value of the infrared image and the column mean vector of the black body data to obtain the column stripes of each column of the infrared image; then removing the column stripes from the infrared image output by the non-uniformity correction, and outputting the infrared image corrected by the column stripes;
in the step 4, the column stripes D corresponding to different temperatures T are obtained by the following operationsT：
S41, taking the average value of each column of the preprocessed black body data to obtain a column average value vector C of the black body data at the corresponding temperatureT；CT＝CT1，CT2，...，CTj(ii) a Symbol CTjThe average value of the jth column of the image is represented when the temperature is T, and j is the column number of pixel points of the black body image corresponding to the black body data;
step S42, the obtained column mean vector CTSmoothing is carried out to remove the stripe quantity, and the column mean vector after smoothing is named as CT'，CT'＝CT1',CT2',...,CTj'；CTj' is the value of the smoothed jth column; and
step S43, subtracting the column mean vector of the black body data at the corresponding temperature from the smoothed column mean vector to obtain the column stripe D at the corresponding temperatureT，DT＝CT-CT'，DT＝D1，D2，...，Dj。
2. The infrared image processing method as set forth in claim 1, characterized in thatIn the step 1, the interval T is collected at a set temperature0For the period, black body data E corresponding to different temperatures T are collectedT。
3. The infrared image processing method according to claim 1, wherein in the step 3, the obtaining the gain coefficient and the offset coefficient by means of multipoint correction includes the following operations:
step S31 according to
Calculating a gain factor an(i,j)；
Step S32 according to
Calculating a bias coefficient bn(i,j)；
Wherein, VHRepresenting the average gray value of all pixel points of black body data collected at high temperature;
VLrepresenting the average gray value of all pixel points of black body data collected at low temperature;
n represents the number of intervals of the multipoint correction.
4. The method for processing infrared image based on FPGA memory optimization according to claim 3, wherein in the step 5, the non-uniform correction of the input infrared image based on the gain coefficient and the bias coefficient comprises the following operations:
step S51. According to Xn(i, j) judging the temperature interval of the gray value of the black body data by using the temperature data of the black body data to obtain the corresponding an(i,j)、bn(i,j)；
Step S52, based on Yn(i,j)＝an(i,j)×Xn(i,j)+bn(i, j), non-uniformity correcting the infrared image, outputting Yn(i,j)；
Wherein, Xn(i, j) represents the output of the detecting element (i, j) before correction, namely an infrared image before correction;
Yn(i, j) denotes a corrected output, i.e., a non-uniformity corrected infrared image.
5. A computer system, comprising:
one or more processors;
a memory storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising performing processes of the method of any of claims 1-4 above.
6. A computer-readable medium storing software, the software comprising instructions executable by one or more computers to perform the process of any one of the methods of claims 1-4 when executed by the one or more computers.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011323560.2A CN112393807B (en) | 2020-11-23 | 2020-11-23 | Infrared image processing method, device, system and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011323560.2A CN112393807B (en) | 2020-11-23 | 2020-11-23 | Infrared image processing method, device, system and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112393807A CN112393807A (en) | 2021-02-23 |
| CN112393807B true CN112393807B (en) | 2021-11-23 |
Family
ID=74607705
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011323560.2A Active CN112393807B (en) | 2020-11-23 | 2020-11-23 | Infrared image processing method, device, system and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112393807B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113489907B (en) * | 2021-07-16 | 2023-07-14 | 北京富吉瑞光电科技股份有限公司 | Imaging control method and device of thermal infrared imager and infrared imaging equipment |
| CN113432723B (en) * | 2021-08-29 | 2021-11-05 | 北方夜视科技(南京)研究院有限公司 | Image processing method, system and computer system for weakening stray radiation |
| CN114066759B (en) * | 2021-11-18 | 2023-08-01 | 电子科技大学 | A method and system for real-time distortion correction of infrared images based on FPGA |
| CN116112776A (en) * | 2023-01-03 | 2023-05-12 | 广东家祥智能科技有限公司 | A behavior analysis method and device based on AI intelligent image analysis and processing |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103164846A (en) * | 2013-03-15 | 2013-06-19 | 上海海事大学 | Infrared fringe non-uniformity correction method based on steering kernel estimation |
| CN104251742A (en) * | 2014-09-30 | 2014-12-31 | 成都市晶林科技有限公司 | Two-point linear and target and environment-based binary non-linear infrared detector heterogeneity correction method |
| CN107767346A (en) * | 2017-09-08 | 2018-03-06 | 湖北久之洋红外系统股份有限公司 | A kind of infrared image fringes noise filtering method |
| CN109509158A (en) * | 2018-11-19 | 2019-03-22 | 电子科技大学 | Method based on the removal of amplitude constraint infrared image striped |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR3020735B1 (en) * | 2014-04-30 | 2017-09-15 | Ulis | METHOD FOR PROCESSING AN INFRARED IMAGE FOR NON-UNIFORMITY CORRECTION |
-
2020
- 2020-11-23 CN CN202011323560.2A patent/CN112393807B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103164846A (en) * | 2013-03-15 | 2013-06-19 | 上海海事大学 | Infrared fringe non-uniformity correction method based on steering kernel estimation |
| CN104251742A (en) * | 2014-09-30 | 2014-12-31 | 成都市晶林科技有限公司 | Two-point linear and target and environment-based binary non-linear infrared detector heterogeneity correction method |
| CN107767346A (en) * | 2017-09-08 | 2018-03-06 | 湖北久之洋红外系统股份有限公司 | A kind of infrared image fringes noise filtering method |
| CN109509158A (en) * | 2018-11-19 | 2019-03-22 | 电子科技大学 | Method based on the removal of amplitude constraint infrared image striped |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112393807A (en) | 2021-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112393807B (en) | Infrared image processing method, device, system and computer readable storage medium | |
| CA2758724C (en) | Image processing method for surpressing spatio-temporal column or row noise | |
| CN110211056B (en) | Adaptive infrared image stripping algorithm based on local median histogram | |
| CN101442608B (en) | A Method of Improving Relative Radiation Correction of CCD Camera | |
| CN103335724B (en) | Calibration-based scene self-adaption IRFPA heterogeneity correction method | |
| CN106373094B (en) | The asymmetric correction method and device of a kind of infrared image | |
| CN101666682B (en) | Neural network non-uniformity correction method based on scene statistics | |
| CN103268594B (en) | A kind of blind element replacement method of infrared thermal imagery instrument system | |
| CN106197690B (en) | Image calibrating method and system under the conditions of a kind of wide temperature range | |
| Lee et al. | Dual-branch structured de-striping convolution network using parametric noise model | |
| CN105869129A (en) | Residual heterogeneous noise elimination method for aiming at thermal infrared image after heterogeneous correction | |
| CN112798654A (en) | Fast Gradient Method and Adaptive Jacobian Reconstruction Method for Electrical Impedance Tomography | |
| Bai et al. | MRFormer: Multiscale retractable transformer for medical image progressive denoising via noise level estimation | |
| CN111161182A (en) | A non-local mean-guided partial volume correction method for PET images constrained by MR structure information | |
| CN111161183A (en) | Method for correcting bad pixel points of flat panel detector in oral CT | |
| CN111652820B (en) | Method, device and image processing system for correcting non-uniformity of infrared image stripes | |
| CN105005967A (en) | Method and apparatus for correcting non-uniformity of infrared imaging based on combined space-time filtering | |
| CN106815820B (en) | A kind of infrared image strip noise cancellation method | |
| CN111932478A (en) | Self-adaptive non-uniform correction method for uncooled infrared focal plane | |
| CN114037636B (en) | Multi-frame blind restoration method for correcting image by self-adaptive optical system | |
| Hakim et al. | Statistical-based Stripe Noise Correction on LAPAN Microsatellite Imagery | |
| CN111076815B (en) | A method for correcting non-uniformity of hyperspectral images | |
| Pagot et al. | Automated processing of lasco coronal images: Spurious point-source-filtering and missing-blocks correction | |
| Jia et al. | An unsupervised underwater image enhancement method based on generative adversarial networks with edge extraction | |
| Zhou et al. | Realized multipoint method for real-time correction of nonuniformity of uncooled IRFPA |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |