CN118129907B - Medium-long wave infrared spectrum modulation snapshot imaging spectrometer and spectrum reconstruction method - Google Patents

Abstract

The invention relates to the technical field of spectral imaging, in particular to a middle-long wave infrared spectrum modulation snapshot imaging spectrometer and a spectrum reconstruction method, wherein a target light field enters an infrared spectrum modulation device formed by super-surface units through a telescopic system, an aperture diaphragm and an imaging objective lens to perform spectrum modulation of middle-long wave infrared bands; the emergent light of the infrared spectrum modulation device is divided into medium-wave infrared light and long-wave infrared light through a dichroic mirror, and the medium-wave infrared light and the long-wave infrared light are respectively emitted into a corresponding imaging objective lens and an area array detector; the infrared spectrum modulation device based on the super surface is matched with the multimode space division multiplexing to realize the multimode spectrum modulation, so that the spectrum resolution is effectively improved; the spectrum reconstruction method adopts spectrum coding aperture matched with the spectrum information demodulation technology of the neural network, effectively improves the spatial resolution, and realizes the snapshot type integrated high-resolution detection of the light field spectrum information and the image information.

Description

Mid- and long-wave infrared spectral modulation snapshot imaging spectrometer and spectrum reconstruction method

Technical Field

The present invention relates to the technical field of spectral imaging, and in particular to a medium- and long-wave infrared spectral modulation snapshot imaging spectrometer and a spectrum reconstruction method.

Background Art

In the process of detecting and identifying targets, spectral imaging detection technology can detect target images and spectral information on fine spectrum bands, thereby achieving a more comprehensive, accurate and scientific understanding of the target. By using the spectral imaging data of the target surface radiation information, the target image and spectral feature information are comprehensively obtained, effectively improving the contrast between the target and the background, highlighting the detailed features of the target, enhancing the effect of target identification, and more comprehensively and deeply understanding the properties and behaviors of the target, and realizing target detection and identification under complex conditions.

Imaging spectrometers can obtain a three-dimensional data cube that integrates the target scene and the spectrum. Currently, time scanning is mostly used to obtain the third-dimensional spatial or spectral information. However, when the target exhibits rapid spatial motion or rapid spectral changes (such as rockets, missiles, aircraft, etc.), its time scanning process will greatly limit the effectiveness of spectrum detection. The snapshot imaging spectrometer uses a static structure to obtain the entire three-dimensional image spectrum data cube of the target within a single integration time of the detector, greatly improving the spectrum measurement capability for dynamic scenes or rapidly changing targets. Since the spectrum cube is three-dimensional data, and the array detector is only a two-dimensional device, the snapshot imaging spectrometer must simultaneously measure all elements of the data cube on the two-dimensional array detector, which requires the use of a special array device structure optical design to provide the ability to simultaneously obtain three-dimensional data.

The traditional spectral imaging detection system uses a two-dimensional array detector to obtain the light intensity distribution in two-dimensional space. The acquisition of information in the third dimension needs to be achieved through multiple discrete optical and mechanical devices, which has problems such as complex system, large volume, and difficulty in integration. At the same time, most of the detection bands of snapshot imaging spectral modulation are mostly in the visible light band, and there are few studies on the infrared band. In the mid- and long-wave infrared bands, the array detector uses a special cooling structure, which encapsulates the focal plane array and the cold screen aperture together in the Dewar flask through the infrared window. The infrared window and the cold screen aperture limit the integration of the filter array and the focal plane.

Summary of the invention

In order to solve the above problems, the present invention provides a medium- and long-wave infrared spectral modulation snapshot imaging spectrometer and a spectrum reconstruction method, adopts a broadband infrared spectral modulation device based on a metasurface, realizes multimodal spectral modulation through the Fano resonance of a subwavelength structure, and constructs an arrayed "space-spectrum" modulation strategy by jointly optimizing the metasurface material, structure and size parameters with a reconstruction algorithm, and adopts an evaluation standard of "space-spectrum" coded modulation based on a correlation criterion to improve the spectral reconstruction accuracy.

The medium- and long-wave infrared spectral modulation snapshot imaging spectrometer provided by the present invention comprises a telescope system, an aperture diaphragm, an imaging objective lens, an infrared spectral modulation device, a dichroic mirror, a medium-wave infrared imaging spectral channel and a long-wave infrared imaging spectral channel; wherein the target light field is converged by the telescope system, and then sequentially imaged through the aperture diaphragm and the imaging objective lens onto the infrared spectral modulation device to obtain a spectral modulation imaging light field; the spectral modulation imaging light field is incident on the dichroic mirror, and the dichroic mirror divides the spectral modulation imaging light field into long-wave infrared light and medium-wave infrared light, the medium-wave infrared light enters the medium-wave infrared imaging spectral channel, and the long-wave infrared light enters the long-wave infrared imaging spectral channel;

The medium-wave infrared imaging spectrum channel includes a medium-wave infrared relay imaging mirror and a medium-wave infrared array detector. The medium-wave infrared light is imaged onto the medium-wave infrared array detector through the medium-wave infrared relay imaging mirror. The medium-wave infrared array detector includes a medium-wave infrared cold screen diaphragm and a medium-wave infrared focal plane array. The medium-wave infrared cold screen diaphragm is located at the image focal plane of the medium-wave infrared relay imaging mirror and coincides with the exit pupil of the medium-wave infrared relay imaging mirror. The medium-wave infrared focal plane array is located at the image plane of the medium-wave infrared relay imaging mirror, and each pixel of the medium-wave infrared focal plane array corresponds to an image of a metasurface unit on the infrared spectrum modulation device after passing through the medium-wave infrared relay imaging mirror. After the medium-wave infrared light is incident on the medium-wave infrared cold screen diaphragm to suppress stray light, it is imaged in the medium-wave infrared focal plane array.

The long-wave infrared imaging spectrum channel includes a long-wave infrared relay imaging mirror and a long-wave infrared array detector. The long-wave infrared light is imaged onto the long-wave infrared array detector through the long-wave infrared relay imaging mirror; the long-wave infrared array detector includes a long-wave infrared cold screen diaphragm and a long-wave infrared focal plane array; wherein, the long-wave infrared cold screen diaphragm is located at the image focal plane of the long-wave infrared relay imaging mirror and coincides with the exit pupil of the long-wave infrared relay imaging mirror; the long-wave infrared focal plane array is located at the image plane of the long-wave infrared relay imaging mirror, and each pixel of the long-wave infrared focal plane array corresponds to the image of a metasurface unit after passing through the long-wave infrared relay imaging mirror; after the long-wave infrared light is incident on the long-wave infrared cold screen diaphragm for stray light suppression, it is imaged in the long-wave infrared focal plane array.

Furthermore, the infrared spectrum modulation device includes arrayed metasurface units, each metasurface unit corresponds to a pixel of a medium-wave infrared array detector or a long-wave infrared array detector through a medium-wave infrared relay imaging mirror or a long-wave infrared relay imaging mirror, and the absolute value of the correlation coefficient between the spectral transmittance functions of every two adjacent metasurface units is less than 0.1; no less than two adjacent metasurface units constitute a spectrum modulation unit, the spectrum modulation units are arrayed and repeated to obtain an infrared spectrum modulation device, and adjacent spectrum modulation units randomly share the same metasurface unit.

Furthermore, the metasurface unit includes no less than two micro-nanostructure arrays with sub-wavelength periods; each micro-nanostructure array is a structure with four-fold rotational symmetry, and different micro-nanostructure arrays have different spectral modulation characteristics; the micro-nanostructure arrays of every two adjacent metasurface units are different.

Furthermore, the medium-wave infrared array detector also includes a dewar flask with a window, and the medium-wave infrared cold screen aperture and the medium-wave infrared focal plane array are packaged in the dewar flask.

Furthermore, the long-wave infrared array detector comprises a dewar flask with a window, and the long-wave infrared cold screen aperture and the long-wave infrared focal plane array are packaged in the dewar flask.

Furthermore, the imaging objective lens adopts an image-side telecentric optical path structure, and the medium-wave infrared relay imaging lens and the long-wave infrared relay imaging lens both adopt an object-side telecentric optical path structure.

Furthermore, the telescope system includes a telescope objective, a field diaphragm and a collimator; wherein the target light field is imaged on the field diaphragm through the telescope objective, the field diaphragm limits the target light field and then becomes parallel light through the collimator and is incident on the aperture diaphragm.

Based on the provided medium- and long-wave infrared spectral modulation snapshot imaging spectrometer, the present invention also proposes a medium-wave infrared spectral encoding aperture spectrum reconstruction method, which specifically includes the following contents:

Get all 2D measurements from the MWIR focal plane array;

At each wavelength, each metasurface unit in the spectral modulation unit is regarded as a medium-wave infrared aperture coding plate unit. At this time, the spectral modulation unit is regarded as a medium-wave infrared spatial aperture coding plate. The two-dimensional measurement values obtained by the medium-wave infrared spatial aperture coding plate corresponding to the medium-wave infrared focal plane array are spliced column by column into a one-dimensional vector to obtain the medium-wave infrared measurement vector ;

The MWIR measurement vector Input the trained autoencoder neural network to obtain the medium-wave infrared three-dimensional atlas vector ;

The mid-wave infrared three-dimensional atlas vector The matrix is restored into a medium-wave infrared three-dimensional spectrum cube according to the wavelength order.

Furthermore, each MWIR spatial aperture coding plate at each wavelength has its own spatial modulation characteristics.

Based on the provided mid- and long-wave infrared spectral modulation snapshot imaging spectrometer, the present invention also proposes a long-wave infrared spectral encoding aperture spectrum reconstruction method, which specifically includes the following contents:

Get all 2D measurements from the LWIR focal plane array;

At each wavelength, each metasurface unit in the spectral modulation unit is regarded as a long-wave infrared aperture coding plate unit. At this time, the spectral modulation unit is regarded as a long-wave infrared spatial aperture coding plate. The two-dimensional measurement values obtained by the long-wave infrared spatial aperture coding plate corresponding to the long-wave infrared focal plane array are spliced into a one-dimensional vector column by column to obtain the long-wave infrared measurement vector ;

The long-wave infrared measurement vector Input the trained autoencoder neural network to obtain the long-wave infrared three-dimensional atlas vector ;

The long-wave infrared three-dimensional atlas vector The long-wave infrared three-dimensional atlas cube is restored by matrixing the wavelength sequence.

Furthermore, each long-wave infrared spatial aperture coding plate at each wavelength has its own spatial modulation characteristics.

Compared with the prior art, the present invention can achieve the following beneficial effects:

1) The present invention realizes the rapid measurement of image spectrum while realizing the lightweight and miniaturization of the instrument through spectral modulation of the metasurface infrared spectrum modulation device. The lightweight and miniaturized static measurement can improve the stability and reliability of information detection in harsh environments, while improving the real-time performance of information detection, thereby obtaining higher optical performance, smaller optical system volume and stronger environmental adaptability, solving the problems of relatively single function, complex system, large volume, difficult integration, difficult information fusion and poor coordination of traditional photoelectric detection equipment;

2) When using an infrared spectrum modulation device containing multiple metasurface units, multi-mode space division multiplexing is used to adjust the configuration mode of the metasurface units, so that the number of spectral sampling points is increased and the spectral resolution is improved;

3) Spectral coded aperture technology is used for encoding and neural network is used for decoding to improve spatial resolution. At the same time, remote relay imaging technology is used to achieve snapshot integrated detection of spectral information and image information, which improves the real-time detection of spectral information and the stability of the system.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is an overall structural diagram of a mid- and long-wave infrared spectral modulation snapshot imaging spectrometer provided according to an embodiment of the present invention;

FIG2 is a schematic diagram of the structure of an infrared spectrum modulation device provided according to an embodiment of the present invention;

FIG3 is a schematic diagram of a metasurface combination method of a spectrum modulation unit provided according to an embodiment of the present invention;

FIG4 is a schematic diagram of a configuration mode of a metasurface unit provided according to an embodiment of the present invention;

FIG5 is a schematic diagram of vectorization of a three-dimensional atlas data cube provided by an embodiment of the present invention;

FIG6 is a schematic diagram of diagonalization of a spectrally coded aperture cube vector provided by an embodiment of the present invention;

FIG7 is a network structure diagram of a graph reconstruction autoencoder neural network provided in an embodiment of the present invention.

Figure numerals: telescope objective 101, field diaphragm 102, collimator 103, aperture diaphragm 104, imaging objective 105, infrared spectrum modulation device 106, metasurface unit 1061, micro-nano structure array 1062, dichroic mirror 107, medium-wave infrared relay imaging mirror 108, medium-wave infrared cold shield diaphragm 109, medium-wave infrared focal plane array 110, long-wave infrared relay imaging mirror 111, long-wave infrared cold shield diaphragm 112, long-wave infrared focal plane array 113, Dewar flask 114, window 115.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

The medium- and long-wave infrared spectral modulation snapshot imaging spectrometer and spectrum reconstruction method provided by the present invention adopt infrared spectral modulation technology based on metasurface in combination with a planar array detector with a cold screen aperture, thereby solving the problems of complex system, large volume and difficult integration existing in traditional imaging spectral instruments. The infrared spectral modulator based on metasurface utilizes multi-mode space division multiplexing technology to increase the number of spectral sampling points and improve the spectral resolution, and cooperates with spectral coded aperture technology in combination with a neural network algorithm to improve the spatial resolution.

FIG1 shows the overall structure of a mid- and long-wave infrared spectral modulation snapshot imaging spectrometer provided according to an embodiment of the present invention.

As shown in FIG1 , the mid- and long-wave infrared spectral modulation snapshot imaging spectrometer provided in an embodiment of the present invention includes a telescope system, an aperture stop 104 , an imaging objective lens 105 , an infrared spectral modulation device 106 , a dichroic mirror 107 , a mid-wave infrared imaging spectral channel, and a long-wave infrared imaging spectral channel.

The telescope system includes a telescope objective lens 101, a field stop 102 and a collimator lens 103. The target light field is imaged on the field stop 102 through the telescope objective lens 101, and the field stop 102 limits the target light field and then becomes parallel light through the collimator lens 103. After the target light field passes through the telescope system, the beam diameter is reduced and then incident on the aperture stop 104. The aperture stop 104 is located on the front focal plane of the imaging objective lens 105. The imaging objective lens 105 adopts an image-side telecentric optical path structure so that the main light of the light beam generated by the imaging objective lens 105 is parallel to the optical axis. The infrared spectrum modulation device 106 is located on the back focal plane of the imaging objective lens 105, perpendicular to the optical axis, receives the light beam from the imaging objective lens 105 and generates a spectrally modulated imaging light field. The spectrally modulated imaging light field is incident on the dichroic mirror 107, which divides the spectrally modulated imaging light field into long-wave infrared light and medium-wave infrared light. The medium-wave infrared light enters the medium-wave infrared imaging spectral channel for medium-wave infrared imaging, and the long-wave infrared light enters the long-wave infrared imaging spectral channel for long-wave infrared imaging.

The medium-wave infrared imaging spectrum channel includes a medium-wave infrared relay imaging mirror 108 and a medium-wave infrared array detector. The medium-wave infrared relay imaging mirror 108 adopts an object-side telecentric optical path structure, so that the main ray of the medium-wave infrared light is parallel to the optical axis after the medium-wave infrared light passes through the medium-wave infrared relay imaging mirror 108. The medium-wave infrared light is imaged onto the medium-wave infrared array detector through the medium-wave infrared relay imaging mirror 108. The long-wave infrared imaging spectrum channel includes a long-wave infrared relay imaging mirror 111 and a long-wave infrared array detector. The long-wave infrared relay imaging mirror 111 also adopts an object-side telecentric optical path structure, so that the main ray of the medium-wave infrared light is parallel to the optical axis after the long-wave infrared light passes through the long-wave infrared relay imaging mirror 111. The long-wave infrared light is imaged onto the long-wave infrared array detector through the long-wave infrared relay imaging mirror 111.

The outside of the medium-wave infrared array detector and the long-wave infrared array detector are both equipped with a Dewar flask 114 with a window 115, and the inside of the medium-wave infrared array detector includes a medium-wave infrared cold screen diaphragm 109 and a medium-wave infrared focal plane array 110. The medium-wave infrared cold screen diaphragm 109 is located at the image focal plane of the medium-wave infrared relay imaging mirror 108 and coincides with the exit pupil of the medium-wave infrared relay imaging mirror 108. The medium-wave infrared focal plane array 110 is located at the image plane of the medium-wave infrared relay imaging mirror 108. After the medium-wave infrared light is injected into the medium-wave infrared cold screen diaphragm 109 by the window 115 to suppress stray light, it is imaged in the medium-wave infrared focal plane array 110. The inside of the long-wave infrared array detector includes a long-wave infrared cold screen diaphragm 112 and a long-wave infrared focal plane array 113. The long-wave infrared cold screen aperture 112 is located at the image focal plane of the long-wave infrared relay imaging mirror 111 and coincides with the exit pupil of the long-wave infrared relay imaging mirror 111, and the long-wave infrared focal plane array 113 is located at the image plane of the long-wave infrared relay imaging mirror 111. After the long-wave infrared light is injected into the long-wave infrared cold screen aperture 112 by the window 115 to suppress stray light, it is imaged in the long-wave infrared focal plane array 113. In the embodiment of the present invention, the material of the focal plane array of the medium-wave infrared array detector includes but is not limited to HgCdTe and InSb, and the material of the focal plane array of the long-wave infrared array detector includes but is not limited to HgCdTe and type II superlattice.

FIG. 2 shows the structure of an infrared spectrum modulation device provided according to an embodiment of the present invention.

As shown in FIG2 , the infrared spectrum modulation device 106 provided in an embodiment of the present invention includes arrayed metasurface units 1061, each of which corresponds to a pixel of a medium-wave infrared focal plane array 110 or a long-wave infrared focal plane array 113 after being passed through a medium-wave infrared relay imaging mirror 108 or a long-wave infrared relay imaging mirror 111. A metasurface unit 1061 is composed of a micro-nanostructure array 1062 with a sub-wavelength period, and different metasurface units 1061 have different structural parameters, thereby achieving different spectral transmittance modulations. In order to improve the accuracy of spectral reconstruction, it is necessary to have a low correlation coefficient between the spectral transmittance functions of each metasurface unit 1061. The correlation coefficient refers to the covariance between two spectral transmittance functions divided by their respective standard deviations. It is preferred that the absolute value of the correlation coefficient between the spectral transmittance functions of each metasurface unit 1061 is less than 0.1.

Among them, the design principle of each micro-nano structure unit in the micro-nano structure array 1062 on a single super surface unit 1061 is to meet the four-fold rotational symmetry to ensure that the spectral modulation characteristics are independent of polarization, that is, the spectral modulation characteristics are the same for each polarization state, thereby ensuring the effectiveness of polarization detection. Therefore, the design of the micro-nano structure array 1062 includes but is not limited to structures with four-fold rotational symmetry such as a circular hole array, a square hole array, a right-angled diamond hole array, a cross hole array, an X-shaped hole array, and a ring hole array. By adjusting the array period of the micro-nano structure array 1062 and the structural parameters such as the diameter, side length, and thickness of each micro-nano structure unit, different spectral transmittance modulations in the band are achieved. In order to make the spectral transmittance functions of each super surface unit 1061 have a low correlation coefficient, in an embodiment of the present invention, it is preferred that the micro-nano structure arrays 1062 of every two adjacent super surface units 1061 are different.

No less than two adjacent metasurface units 1061 constitute a spectral modulation unit, and then the spectral modulation units are arrayed and repeated to obtain an infrared spectral modulation device 106. In a specific embodiment, assuming that the number of arrays of the medium-wave infrared focal plane array 110 or the long-wave infrared focal plane array 113 is M×N, the number of metasurface units 1061 is also M×N. Different and adjacent m×n metasurface units 1061 constitute a spectral modulation unit. At this time, the entire infrared spectral modulation device 106 includes (M/m)×(N/n) spectral modulation units, and each spectral modulation unit corresponds to a target object point in space. By integrating (M/m)×(N/n) spectral modulation units on an infrared spectral modulation device 106, snapshot spectral imaging is achieved, and it also has the advantages of miniaturization and lightweight.

FIG. 3 shows a metasurface assembly method of a spectral modulation unit provided according to an embodiment of the present invention.

In a single spectrum modulation unit, multiple types of metasurface units 1061 are used to form a spectrum modulation unit. Typically, when 4, 8, 9, or 16 types of metasurface units 1061 form a spectrum modulation unit, their arrangement may be as shown in FIG. 3 .

Among them, the spectral resolution of the 4-channel spectral modulation unit is the lowest, the spectral resolution of the 8-channel spectral modulation unit will be increased by 2 times, the spectral resolution of the 9-channel spectral modulation unit will be increased by 9/4 times, and the spectral resolution of the 16-channel spectral modulation unit will be increased by 4 times. However, the spatial resolution of the 16-channel spectral modulation unit is the lowest, the spatial resolution of the 9-channel spectral modulation unit is 16/9 times, the spatial resolution of the 8-channel spectral modulation unit is 2 times, and the spatial resolution of the 4-channel spectral modulation unit is 4 times.

Assuming that the number of metasurface units 1061 contained in a spectral modulation unit is m×n=L, then for each spectral modulation unit, its measurement matrix T is:

;

The spectral modulation unit measurement matrix has a total of L rows, indicating that it contains a total of L metasurface units 1061, each row of which represents the spectral transmittance of a metasurface unit 1061, K represents the number of sampling points of the wavelength, It represents the transmittance of the Lth metasurface unit at the Kth wavelength sampling point.

For spectral reconstruction, the measurement equation of a spectral modulation unit can be expressed as:

;

represents the measured light intensity of the Lth metasurface unit, Represents the spectral intensity at the Kth wavelength sampling point. Since the number of metasurface units 1061 in each spectral modulation unit is much smaller than the number of wavelength sampling points (L<<K), the measurement equation is an underdetermined linear equation system, which needs to be optimized and solved in combination with the prior information of the spectrum to be measured in order to achieve high-resolution spectral reconstruction on the chip. In order to improve the accuracy of spectral reconstruction, the absolute value of the correlation coefficient between the spectral transmittance functions of each metasurface unit 1061 is less than 0.1, that is, the correlation between each row vector of the measurement matrix Ｔ and other row vectors is as small as possible, thereby ensuring high spectral resolution.

For any metasurface unit 1061 in a spectrum modulation unit, any two or more metasurface units 1061 with different spectral transmittances are selected near the metasurface unit 1061 to form a spectrum modulation unit, that is, a spectrum modulation unit of any shape is constructed by dynamically combining adjacent metasurface units 1061 to perform spectrum measurement, and adjacent spectrum modulation units randomly share the same metasurface unit 1061, that is, adjacent spectrum modulation units may share the same metasurface unit 1061 or may not share the same metasurface unit 1061 according to actual needs. By reasonably selecting the number of metasurface units 1061 and the configuration mode of the shape of the spectrum modulation unit, and reasonably selecting whether adjacent spectrum modulation units need to share the same metasurface unit 1061 according to actual conditions, multi-mode space division multiplexing of the metasurface unit 1061 is realized, thereby reducing the multicollinearity of the spectrum transmittance while ensuring the resolution of the reconstructed spectrum.

FIG. 4 is a configuration diagram of the metasurface unit 1061 according to an embodiment of the present invention.

As shown in Figure 4, taking a spectral modulation area composed of an m×n=4×4 array of metasurface units 1061 as an example, a spectral modulation area has a total of 16 metasurface units 1061 with different spectral transmittances. At this time, a spectral modulation unit can be formed by arranging multiple metasurface units 1061 such as 4×4, 3×3, 2×4, 4×2, 2×2, etc.

When 4×4 metasurface units 1061 are used to form a spectral modulation unit, there is only one configuration mode;

When 3×3 metasurface units 1061 are used to form a spectral modulation unit, there are 16 configuration modes;

When 2×4 metasurface units 1061 are used to form a spectral modulation unit, there are two configuration modes;

When 4×2 metasurface units 1061 are used to form a spectral modulation unit, there are two configuration modes;

When 2×2 metasurface units 1061 are used to form a spectral modulation unit, there are a total of 4 configuration modes.

When the application requires the system to have a higher spectral resolution, a 4×4 array configuration mode can be used; when the application requires the system to have a higher spatial resolution, a 2×2 array configuration mode can be used; for the application target scene, when it is high-frequency target information in one image dimension and low-frequency background information in another image dimension, a 2×4 array or 4×2 array configuration mode can be used.

Since different metasurface units 1061 have different spectral modulation characteristics, the entire infrared spectrum modulation device 106 has different spatial modulation characteristics at different wavelengths. In order to improve the accuracy of spatial image reconstruction, the correlation between the spatial distribution of metasurface array transmittance at each wavelength is as small as possible, and it is also necessary to measure that the correlation between each column vector of the matrix T and other column vectors is as small as possible, so as to ensure high image resolution, that is, in the embodiment of the present invention, the micro-nano structure arrays 1062 of every two adjacent metasurface units 1061 are also different.

FIG. 5 and FIG. 6 respectively illustrate the process of vectorization of the three-dimensional atlas data cube and the process of vector diagonalization of the spectrally coded aperture cube provided in the embodiments of the present invention.

For spatial image reconstruction, in the process of spectrally coded aperture algorithm, the 3D atlas data cube to be restored is first vectorized, that is, the 2D image array under each single wavelength is sequentially spliced into a vector column by column, and the vectors of each single wavelength are then serially spliced into a longer vector, denoted as vector X(M, N, K). The process is shown in Figure 5. , , and Indicates different wavelengths. Among them, M and N represent the size of a single two-dimensional image, K represents the number of spectral wavelengths of the two-dimensional image, Represents the two-dimensional image corresponding to the Kth wavelength.

The measurement matrix H is decomposed from the spectrally coded aperture cube into coded aperture patterns at each single wavelength, and the coded aperture patterns at each single wavelength are The diagonal pattern is then arranged in order of wavelengths in the horizontal direction, as shown in Figure 6. The diagonal elements of each diagonal pattern are generated by the vectorized elements of the coded aperture pattern corresponding to the single wavelength.

The measurement vector data Y is the two-dimensional measurement value obtained by the detector. The vectorization is obtained by sequentially concatenating columns.

Therefore, the measurement equation Y=HX of the entire metasurface spectral imaging chip can be expressed as:

;

Since the number of arrays of the medium-wave infrared focal plane array 110 or the long-wave infrared focal plane array 113 is much smaller than the number of sampling points of the three-dimensional atlas data cube (M×N<<M×N×K), the measurement equation is an underdetermined linear equation system, which requires a large amount of prior knowledge for optimization solution.

In order to avoid using a large amount of prior knowledge and improve the effect of spatial image reconstruction, the embodiment of the present invention provides a medium- and long-wave infrared spectral modulation snapshot imaging spectrometer based on the embodiment of the present invention, and also proposes two spectrally coded aperture map reconstruction methods, which can improve the spatial resolution of image reconstruction without losing spectral resolution.

Specific embodiment 1:

The method for reconstructing a spectrally coded aperture spectrum of a medium-wave infrared provided by an embodiment of the present invention specifically includes the following contents:

All two-dimensional measurements obtained by the MWIR focal plane array 110 are acquired.

At each wavelength, each metasurface unit 1061 in the spectral modulation unit is regarded as a medium-wave infrared aperture coding plate unit, and the spectral modulation unit array is regarded as a medium-wave infrared spatial aperture coding plate. At this time, the medium-wave infrared spatial aperture coding plate at each wavelength has its own spatial modulation characteristics. Therefore, for the entire spectral band, the entire spectral detection unit array is a spectrally modulated medium-wave infrared spatial aperture coding plate.

The two-dimensional measurement values obtained by the medium-wave infrared spatial aperture code plate corresponding to the medium-wave infrared focal plane array 110 are spliced into one-dimensional vectors in order, and the medium-wave infrared measurement vector is obtained. .

The MWIR measurement vector Input the trained autoencoder neural network to obtain the corresponding medium-wave infrared three-dimensional atlas vector .

In the embodiment of the present invention, in order to avoid introducing too much prior knowledge and increasing the amount of calculation, a deep learning neural network is used to perform an optimization solution. The spectral aperture coding is used to obtain the medium-wave infrared measurement vector , and then reconstruct the three-dimensional map through neural network decoding. It is preferred to use the trained map reconstruction autoencoder neural network to respectively decode the medium-wave infrared measurement vector Perform calculation prediction to obtain the corresponding medium-wave infrared three-dimensional spectrum vector .

FIG. 7 shows the network structure of the graph reconstruction autoencoder neural network provided by an embodiment of the present invention.

Among them, the low-dimensional discretized medium-wave infrared measurement vector Input the spectrum reconstruction autoencoder neural network shown in Figure 7 to obtain a high-dimensional medium-wave infrared three-dimensional spectrum vector , by optimizing and adjusting the parameters of the reconstruction network, the reconstructed spectrum is constantly approaching the original spectrum. The spectrum reconstruction network is a multi-layer perceptron with one-dimensional input and one-dimensional output, which is fully connected by multiple hidden layers (i.e., H in Figure 7). For the spectrum reconstruction network, the length of the input data is short, and the length of the output data is long due to the need for higher spectral resolution. In the actual optimization process, the predicted value of the reconstructed spectrum is first obtained by forward propagation, and then the loss value and its gradient to the neural network parameters are calculated. According to the gradient, the network parameters are updated by the back propagation algorithm. Repeating this process makes the loss value drop close to zero, thereby achieving the optimization of the network.

The mid-wave infrared three-dimensional atlas vector The matrix is restored into a medium-wave infrared three-dimensional spectrum cube according to the wavelength order.

Specific embodiment 2:

The method for reconstructing a long-wave infrared spectral coded aperture spectrum provided by an embodiment of the present invention specifically includes the following contents:

Obtain all two-dimensional measurements obtained by the long-wave infrared focal plane array 113;

At each wavelength, each metasurface unit 1061 in the spectral modulation unit is regarded as a long-wave infrared aperture coding plate unit, and the spectral modulation unit is regarded as a long-wave infrared spatial aperture coding plate. At this time, the long-wave infrared spatial aperture coding plate at each wavelength has its own spatial modulation characteristics. Therefore, for the entire spectral band, the entire long-wave infrared spectral detection unit array is a spectrally modulated long-wave infrared spatial aperture coding plate.

The two-dimensional measurement values obtained by the long-wave infrared spatial aperture code plate corresponding to the long-wave infrared focal plane array 113 are spliced into a one-dimensional vector in sequence column by column to obtain a long-wave infrared measurement vector .

The long-wave infrared measurement vector Input the trained autoencoder neural network to obtain the corresponding long-wave infrared three-dimensional atlas vector .

The autoencoder neural network in this specific embodiment uses the spectrum reconstruction autoencoder neural network in specific embodiment 1 to measure the long-wave infrared vector Perform calculation prediction to obtain the long-wave infrared three-dimensional map vector , I will not go into details here.

The long-wave infrared three-dimensional atlas vector The long-wave infrared three-dimensional atlas cube is restored by matrixing the wavelength sequence.

The medium- and long-wave infrared spectral modulation snapshot imaging spectrometer achieves the improvement of information dimension while realizing the lightness and miniaturization of the instrument through the organic combination of infrared spectral modulation technology based on metasurface and array detector with cold screen aperture. Lightweight and small static measurement can improve the stability, reliability and real-time performance of information detection in harsh environments. The array detector with cold screen aperture can further suppress background noise, increase detection distance and obtain detailed features. The synergy of software and hardware ultimately achieves higher optical performance, smaller optical system volume and stronger environmental adaptability, thus solving the problems of relatively single function, complex system, large volume, difficult integration, difficult information fusion and poor synergy of traditional photoelectric detection equipment. In the infrared spectral modulation based on metasurface, the spectral resolution is improved based on multi-mode space division multiplexing, and the spatial resolution is improved based on spectral coding aperture and neural network, breaking through the resolution constraints of spatial information and spectral information under the limited spatial bandwidth product of the detector, and realizing the maximum extraction of spectral information.

It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (9)

1. A medium- and long-wave infrared spectral modulation snapshot imaging spectrometer, characterized in that it specifically comprises a telescope system, an aperture diaphragm, an imaging objective lens, an infrared spectral modulation device, a dichroic mirror, a medium-wave infrared imaging spectral channel and a long-wave infrared imaging spectral channel; wherein the target light field is converged by the telescope system, and then sequentially passes through the aperture diaphragm and the imaging objective lens to be imaged onto the infrared spectral modulation device to obtain a spectral modulation imaging light field; the spectral modulation imaging light field is incident on the dichroic mirror, and the dichroic mirror divides the spectral modulation imaging light field into long-wave infrared light and medium-wave infrared light, the medium-wave infrared light enters the medium-wave infrared imaging spectral channel, and the long-wave infrared light enters the long-wave infrared imaging spectral channel; The medium-wave infrared imaging spectrum channel includes a medium-wave infrared relay imaging mirror and a medium-wave infrared array detector. The medium-wave infrared light is imaged onto the medium-wave infrared array detector through the medium-wave infrared relay imaging mirror. The medium-wave infrared array detector includes a medium-wave infrared cold screen diaphragm and a medium-wave infrared focal plane array. The medium-wave infrared cold screen diaphragm is located at the image focal plane of the medium-wave infrared relay imaging mirror and coincides with the exit pupil of the medium-wave infrared relay imaging mirror. The medium-wave infrared focal plane array is located at the image plane of the medium-wave infrared relay imaging mirror, and each pixel of the medium-wave infrared focal plane array corresponds to an image of a metasurface unit on the infrared spectrum modulation device after passing through the medium-wave infrared relay imaging mirror. After the medium-wave infrared light is incident on the medium-wave infrared cold screen diaphragm for stray light suppression, it is imaged in the medium-wave infrared focal plane array. The long-wave infrared imaging spectrum channel includes a long-wave infrared relay imaging mirror and a long-wave infrared array detector, and the long-wave infrared light is imaged onto the long-wave infrared array detector through the long-wave infrared relay imaging mirror; the long-wave infrared array detector includes a long-wave infrared cold screen diaphragm and a long-wave infrared focal plane array; wherein the long-wave infrared cold screen diaphragm is located at the image focal plane of the long-wave infrared relay imaging mirror and coincides with the exit pupil of the long-wave infrared relay imaging mirror; the long-wave infrared focal plane array is located at the image plane of the long-wave infrared relay imaging mirror, and each pixel of the long-wave infrared focal plane array corresponds to an image of a metasurface unit after passing through the long-wave infrared relay imaging mirror; after the long-wave infrared light is incident on the long-wave infrared cold screen diaphragm for stray light suppression, it is imaged in the long-wave infrared focal plane array; At least two adjacent metasurface units are dynamically combined to construct a spectrum modulation unit of any shape for spectrum measurement, the spectrum modulation unit is arrayed and repeated to obtain the infrared spectrum modulation device, and adjacent spectrum modulation units randomly share the same metasurface unit; The infrared spectrum modulation device comprises arrayed metasurface units, each metasurface unit corresponds to a pixel of the medium-wave infrared array detector or the long-wave infrared array detector through the medium-wave infrared relay imaging mirror or the long-wave infrared relay imaging mirror, and the absolute value of the correlation coefficient between the spectral transmittance functions of every two adjacent metasurface units is less than 0.1; The metasurface unit includes no less than two micro-nanostructure arrays with sub-wavelength periods; each micro-nanostructure array is a structure with four-fold rotational symmetry, and different micro-nanostructure arrays have different spectral modulation characteristics; the micro-nanostructure arrays of every two adjacent metasurface units are different. 2. The medium-wave infrared spectral modulation snapshot imaging spectrometer according to claim 1 is characterized in that the medium-wave infrared array detector also includes a Dewar flask with a window, and the medium-wave infrared cold screen aperture and the medium-wave infrared focal plane array are encapsulated in the Dewar flask. 3. The medium- and long-wave infrared spectral modulation snapshot imaging spectrometer according to claim 1 is characterized in that the long-wave infrared array detector also includes a Dewar flask with a window, and the long-wave infrared cold screen aperture and the long-wave infrared focal plane array are encapsulated in the Dewar flask. 4. The medium- and long-wave infrared spectral modulation snapshot imaging spectrometer according to claim 1 is characterized in that the imaging objective lens adopts an image-side telecentric optical path structure, and the medium-wave infrared relay imaging mirror and the long-wave infrared relay imaging mirror both adopt an object-side telecentric optical path structure. 5. The medium- and long-wave infrared spectral modulation snapshot imaging spectrometer according to claim 1 is characterized in that the telescopic system includes a telescopic objective, a field diaphragm and a collimator; wherein the target light field is imaged on the field diaphragm through the telescopic objective, and the field diaphragm limits the target light field and then converts it into parallel light through the collimator and is incident on the aperture diaphragm. 6. A method for reconstructing a medium-wave infrared spectral coded aperture spectrum, applicable to the medium- and long-wave infrared spectral modulation snapshot imaging spectrometer according to any one of claims 1 to 5, characterized in that it specifically comprises the following contents: Obtaining all two-dimensional measurements obtained by the medium-wave infrared focal plane array; At each wavelength, each metasurface unit in the spectral modulation unit is regarded as a medium-wave infrared aperture coding plate unit. At this time, the spectral modulation unit is regarded as a medium-wave infrared spatial aperture coding plate. The two-dimensional measurement values obtained by the medium-wave infrared spatial aperture coding plate corresponding to the medium-wave infrared focal plane array are sequentially spliced into a one-dimensional vector column by column to obtain a medium-wave infrared measurement vector ; The medium wave infrared measurement vector Input the trained autoencoder neural network to obtain the medium-wave infrared three-dimensional atlas vector ; The medium-wave infrared three-dimensional atlas vector The matrix is restored into a medium-wave infrared three-dimensional spectrum cube according to the wavelength order. 7. The method for reconstructing the spectral coded aperture spectrum of medium-wave infrared according to claim 6, characterized in that the medium-wave infrared spatial aperture coding plate at each wavelength has its own spatial modulation characteristics. 8. A method for reconstructing a long-wave infrared spectral coded aperture spectrum, applicable to the medium- and long-wave infrared spectral modulation snapshot imaging spectrometer as claimed in any one of claims 1 to 5, characterized in that it specifically comprises the following contents: Obtaining all two-dimensional measurements obtained by the long-wave infrared focal plane array; At each wavelength, each metasurface unit in the spectral modulation unit is regarded as a long-wave infrared aperture coding plate unit. At this time, the spectral modulation unit is regarded as a long-wave infrared spatial aperture coding plate. The two-dimensional measurement values obtained by the long-wave infrared spatial aperture coding plate corresponding to the long-wave infrared focal plane array are sequentially spliced into a one-dimensional vector column by column to obtain a long-wave infrared measurement vector ; The long-wave infrared measurement vector Input the trained autoencoder neural network to obtain the long-wave infrared three-dimensional atlas vector ; The long-wave infrared three-dimensional atlas vector The long-wave infrared three-dimensional atlas cube is restored by matrixing the wavelength sequence. 9 . The long-wave infrared spectral coded aperture spectrum reconstruction method according to claim 8 , wherein the long-wave infrared spatial aperture coding plate at each wavelength has its own spatial modulation characteristics.