CN112528869B - Phase-free data imaging method based on complex neural network - Google Patents

Abstract

The invention discloses a phase-free data imaging method based on a complex neural network, and belongs to the technical field of imaging of electromagnetic inverse problems. Firstly, a real sample image is acquired, and phase-free total field data and ideal scattered field data are generated by the real sample image to form a training set. The method comprises the steps of establishing a complex domain Unet network, wherein the complex domain Unet network comprises a first branch and a second branch, the first branch comprises an imaginary part generating module and a first complex network, the second branch is a second complex network, and the complex network comprises a complex convolution module, a complex batch processing module and a complex activation function module. Training the first branch and the second branch respectively, cascading the first branch and the second branch after the training is finished, and further fine tuning. And generating a corresponding image by utilizing the trained complex domain Unet network aiming at the non-phase data to be imaged. The invention utilizes two branches in the complex domain Unet network to process the complex domain problem of electromagnetic backscattering, and has high precision and quick response.

Description

A No Phase Data Imaging Method Based on Complex Neural Network

technical field

The invention belongs to the technical field of imaging of electromagnetic inverse problems, and in particular relates to a phase-free data imaging method based on a complex neural network.

Background technique

Electromagnetic inverse scattering is widely used in microwave remote sensing, medical imaging, geological exploration and other fields. Specifically, the physical and geometric feature information of the unknown object is reconstructed through the reflected electromagnetic wave data, mainly including the position, shape, and dielectric constant of the detected object. The biggest challenge of the electromagnetic inverse scattering imaging problem comes from its ill-posedness and high nonlinearity. The obtained electromagnetic wave information is the result of multiple scattering, refraction and diffraction, and usually does not propagate in a simple path within the imaging area. Quantified imaging results need to solve all nonlinear equations, and these equations are ill-conditioned, have non-unique and unstable solutions, that is, small changes in data may lead to huge deviations, these factors make the electromagnetic inverse scattering problem difficult solve.

In practice, on the one hand, it is difficult to accurately measure the phase information of the scattered field data in the high-frequency range; on the other hand, in order to obtain the phase information, the hardware cost will be greatly increased, and new noise will inevitably be introduced . Therefore, the imaging research of the inverse problem based on phase-free data has important engineering practical application significance.

Imaging methods for inverse problems mainly include learning-based methods and model-based methods. Among them, traditional model-based iterative algorithms include Iterative Shrinkage/Thresholding Algorithm (ISTA), Alternating Direction Method of Multipliers (ADMM) and Primal Dual Hybrid Gradient (PDHG) . With the continuous progress in the field of machine learning, especially the application of deep learning, neural networks are increasingly used to solve nonlinear fitting problems in inverse imaging. The inverse imaging solution using deep learning is mainly based on a complete learning method, that is, only through the fitting of a large amount of training data, to learn the mapping from measurement data to image data, such an end-to-end learning network in the form of a "black box" To solve inverse imaging, this requires the network model to learn all the physical rules of inverse imaging, so the learned network is less portable; the second is the method based on image preprocessing. The main reason is that the input of the network model is the initial image processed by an iterative algorithm. Although this increases the imaging time, it increases the interpretability and operability of the model; the third is a partially model-based deep network. The neural network not only replaces the imaging optimization stage in Method 2, but also directly replaces the initial imaging stage of the iterative algorithm. That is, another neural network is used to replace the traditional iterative algorithm, such as the learnable iterative threshold shrinkage algorithm.

When the existing technical solutions face the detection of objects with large target areas, traditional high-precision calculation electromagnetic methods (such as the method of moments, finite element method, finite difference method in time domain, etc.) need to consume a lot of computing resources and take a long time. Although various approximate numerical methods (for example, high-frequency approximation method, Born approximation method, etc.) improve the calculation speed to a certain extent, these methods often ignore the electromagnetic mutual coupling between targets, resulting in low calculation accuracy. On the other hand, for electromagnetic inverse scattering problems, such as radar imaging, many reconstruction algorithms have been applied in the past few decades. For example, back projection algorithm, Newton iterative algorithm, genetic algorithm, etc. These algorithms are usually limited to point-by-point scanning imaging or iterative imaging processes. When applied to large-scale imaging scenarios, the calculations are often huge and cannot meet the real-time requirements.

In recent years, deep learning has been widely used in many fields to complete classification and regression tasks, and achieved good results. At present, most mainstream networks are built based on the real number domain, while the electromagnetic scattering and inverse scattering problems are in the complex number domain; secondly, due to the "black box" property of the deep learning network, the lack of accurate physical support results in the existing deep It is difficult to directly apply the learning framework to electromagnetic scattering and inverse scattering problems, and even if the training results are good, it is difficult to adapt to other different data sets.

Contents of the invention

In order to overcome the problem that the high-precision electromagnetic calculation method in the prior art has a large amount of calculation and cannot meet the application and real-time requirements in large-scale imaging scenarios, and the existing deep learning network is extremely dependent on the training set, which has poor interpretability and adaptability To solve the problem, the present invention proposes a phase-free data imaging method based on a complex neural network, using two branches in the complex domain Unet network to deal with the complex domain problem of electromagnetic inverse scattering, with high precision and fast response.

In order to achieve the above object, the present invention adopts the following technical solutions:

A phase-free data imaging method based on a complex neural network, comprising the following steps:

1) Collect real sample images, generate phase-free total field data and ideal scattered field data from real sample images, and form a training set with phase-free total field data, ideal scattered field data and real sample images;

2) Establish a two-branch complex field Unet network, wherein branch one includes an imaginary part generation module and the first complex network, branch two is the second complex network, and the first complex network and the second complex network are composed of complex convolution modules , complex batch processing module and complex activation function module;

3) Training branch one:

Input the phase-free total field data as the real part into branch 1 of the Unet network in the complex field, first generate the imaginary part corresponding to the real part through the imaginary part generation module, and combine the real part and the generated imaginary part as the first complex number The input of the network is to obtain the predicted scattering field data; the loss is calculated according to the predicted scattering field data and the ideal scattering field data, and branch one is trained;

4) Training branch two:

Input the ideal scattering field data into the branch 2 of the Unet network in the complex number field, process it through the second complex network, output the complex number data and take the modulus to obtain the predicted image; calculate the loss according to the predicted image and the real sample image, and train the branch 2;

5) Perform fine-tuning training on the complex domain Unet network:

The trained branch 1 and branch 2 are connected, and the phase-free total field data is predicted to output the predicted scattered field data after the branch 1 is used as the input of the branch 2, and the predicted image data is output; the loss is calculated according to the predicted image and the real sample image, and the The cascaded branches 1 and 2 in the complex domain Unet network are fine-tuned to obtain a well-trained complex domain Unet network;

6) For the phase-free data to be imaged, use the trained complex domain Unet network to generate the corresponding image.

Compared with the prior art, the present invention has the advantages of:

1) Due to the inadaptability and nonlinearity of the inverse scattering problem, an iterative optimization algorithm is usually used, such as the FISTA algorithm. time to complete the reconstruction, but the accuracy is not high, especially for objects with high permittivity. Algorithms based on artificial neural networks are often only aimed at objects of specific shapes, positions, and sizes, and their adaptability is not strong.

The total field data phase recovery algorithm based on deep learning mainly uses real number network, it is difficult to achieve a good fitting effect for the scattered field data whose output is in the form of complex value; for the complex valued data of the scattered field, it is processed in separate channels, ignoring the complex value Intrinsic link between data real part and imaginary part; Therefore, the present invention proposes the imaging algorithm based on the Unet model of complex number neural network, the Unet core of complex number neural network is to adopt the real part and real part, real part based on complex number basic algorithm The complex convolutional neural network of sum imaginary part, imaginary part and real part, imaginary part and imaginary part replaces real number convolutional neural network. In addition, real number data is used to generate imaginary number data, and complex number output results are modulo processed to improve image quality.

2) This method adopts step-by-step training, and introduces scattered field data in the training process of the network to guide the optimal descending direction, which is more controllable and interpretable for the end-to-end training method in which the total field data is directly restored to the image; Finally, better phase recovery and data imaging are achieved through cascade fine-tuning.

3) Finally, the Unet network based on the complex neural network can well complete the restoration of the total field data to the scattered field data and the mapping of the scattered field data to the image data, while retaining better contour information and permittivity prediction, with stronger robustness.

Description of drawings

Fig. 1 is the flow chart of the phase-free data imaging method based on complex neural network of the present invention;

Fig. 2 is a schematic flow chart of a branch in the complex domain Unet network adopted by the present invention;

Fig. 3 is a schematic diagram of branch two flow charts in the complex domain Unet network adopted by the present invention;

Fig. 4 is the Unet network schematic diagram of the multiple domain Unet after cascading in the present invention;

Figure 5 is a test result diagram with Emnist as the test set;

Figure 6 is a test result diagram with Austria as the test set;

Figure 7 is the result of testing different dielectric constants in Austria;

Fig. 8 is a schematic structural diagram of a branch in this embodiment;

FIG. 9 is a schematic diagram of the structure of branch 2 in this embodiment.

Detailed ways

The present invention will be further elaborated and described below in combination with specific embodiments. The technical features of the various implementations in the present invention can be combined accordingly on the premise that there is no conflict with each other.

A kind of phaseless data imaging method based on complex neural network proposed by the present invention, its sorting flow chart is as shown in Figure 1, and main steps are:

Step 1: Collect real sample images, generate phase-free total field data and ideal scattered field data from the real sample images, and use phase-free total field data, ideal scattered field data and real sample images to form a training set;

Step 2: Establish a two-branch complex field Unet network, wherein branch one includes an imaginary part generation module and the first complex network, branch two is the second complex network, and the first complex network and the second complex network are both composed of complex convolutions module, complex batch processing module and complex activation function module;

Step 3: Train Branch 1:

Input the phase-free total field data as the real part into branch 1 of the Unet network in the complex field, first generate the imaginary part corresponding to the real part through the imaginary part generation module, and combine the real part and the generated imaginary part as the first complex number The input of the network is to obtain the predicted scattering field data; the loss is calculated according to the predicted scattering field data and the ideal scattering field data, and branch one is trained;

Step 4: Train branch 2:

Input the ideal scattering field data into the branch 2 of the Unet network in the complex number field, process it through the second complex network, output the complex number data and take the modulus to obtain the predicted image; calculate the loss according to the predicted image and the real sample image, and train the branch 2;

Step 5: Perform fine-tuning training on the complex domain Unet network:

The trained branch 1 and branch 2 are connected, and the phase-free total field data is predicted to output the predicted scattered field data after the branch 1 is used as the input of the branch 2, and the predicted image data is output; the loss is calculated according to the predicted image and the real sample image, and the The cascaded branches 1 and 2 in the complex domain Unet network are fine-tuned to obtain a well-trained complex domain Unet network;

Step 6: For the phase-free data to be imaged, use the trained complex domain Unet network to generate the corresponding image.

The specific implementation steps of the present invention will be described in detail below.

The dimensions of the real sample images in the training set are uniform, and the dimensions of the phase-free total field data and scattered field data generated from the real sample images are uniform. In this embodiment, step 1 collects a total of 5000 real sample images, wherein the phase-free total field data and ideal scattered field data are generated by real sample images under ideal conditions, and the dimensions are (32, 16) and ( 2,32,16); the dielectric constant range of real image data is 1.0-2.0.

In this embodiment, the generation process of phase-free total field data and ideal scattered field data adopts the existing technical means in this field. The specific operation is first to establish a model, that is, define the solution equation, create the model geometry, define material properties, and establish metal boundaries and the radiation boundary and determine the positional relationship between the model and the model, and determine the number of transmitting and receiving antennas; among them, the material properties include relative magnetic permeability, relative permittivity and electrical conductivity; followed by subdividing the mesh, using the finite element method to convert the model Space discretization, when solving electromagnetic wave problems, the grid resolution is very important when dealing with wave type-related problems. Only when the grid is fine enough can the wavelengths in all media be resolved; finally, a set of linear equations describing the electric field is solved in the simulation domain. Equations, and finally extract useful information from the calculated electric field results. The phase-free total field data, ideal scattered field data and real sample images constitute the training set.

In this embodiment, the complex domain Unet network established in step 2 includes branch one and branch two; said branch one first generates imaginary part data through the imaginary part generation module, combines with real part data to obtain complex number data, and converts the complex number The data is input into the complex network for processing to obtain the predicted data; the second branch also adopts the same complex network as the branch one. The complex network includes a complex convolution module, a complex batch processing module and a complex activation function module. The complex convolution module performs convolution operations on the real and imaginary parts of the input complex data. The convolution process follows the complex basic operations as a criterion, and the final result is output in the form of complex numbers.

The schematic diagrams of the structure of the branch 1 and the branch 2 in this embodiment are shown in FIG. 2 and FIG. 3 .

Branch 1 is mainly used for phase recovery. The input is phase-free data with dimensions (1, 32, 16). After the imaginary part generation module is combined with the real part input, the data with dimensions (2, 32, 16) is obtained as a complex number. The input of the Unet network, the output is the scattered field data predicted by the dimension (2,32,16).

Branch 2 is mainly used for scattering field imaging. The input is the scattering field data under ideal conditions with dimensions (2,32,16). After passing through the complex UNet network, the output is complex data with dimensions (1,32,16). After Take the modulus of complex numbers to obtain image data with dimensions (1,32,32). The cascaded network is to cascade the trained branch one and branch two networks, that is, the input of branch two is the output of branch one.

In a specific implementation of the present invention, both the real number activation function module and the complex number activation function module use the Relu activation function, wherein the complex number activation function module applies activation functions to the real number part and the generated imaginary number part respectively.

The complex convolution module adopts a Unet network, including a downsampling encoding structure composed of several downsampling layers, an upsampling decoding structure composed of several upsampling layers and a skip connection; the number of the downsampling layer and the upsampling layer Equal; the number of downsampling layers and upsampling layers in the complex convolution module of branch one is recorded as n, and the number of downsampling layers and upsampling layers in the complex convolution module of branch two is recorded as m;

Among them, downsampling is the process of compression, that is, Encoder. The image size is reduced by convolution and downsampling, and some shallow features are extracted. Upsampling is the process of decoding, namely Decoder. Get some deep features by convolution and upsampling. The Unet network in the complex domain replaces the real Unet network by constructing a complex convolution module, a complex batch processing module and a complex activation function.

In the complex convolution module of branch 1, the real part corresponding to the phase-free total field data and the generated imaginary part are combined as the input of the first down-sampling layer, and then the output of the previous down-sampling layer is used as the down-sampling layer The input of one layer of downsampling layer, the output of the last layer of downsampling layer is used as the input of the first layer of upsampling; and the output of the i-th downsampling layer and the output of the n-i-th layer of upsampling layer are fused by skip connection , use the fusion result as the input of the next upsampling layer, until the output of the last upsampling layer is obtained as the output of the complex convolution module; in this embodiment, as shown in FIG. 8, n is 3.

In the complex convolution module of branch two, the scattered field data is firstly used as the input of the first layer of downsampling layer, then the output of the previous layer of downsampling layer is used as the input of the next layer of downsampling layer, and the last layer of downsampling layer The output of the layer is used as the input of the first layer of upsampling; and the output of the i-th down-sampling layer is fused with the output of the m-i-th up-sampling layer by skip connection, and the fusion result is used as the input of the next up-sampling layer , until the output of the last upsampling layer is obtained as the output of the complex convolution module. In this embodiment, as shown in FIG. 9 , m is set to 3.

In the complex convolution module, the downsampling layer and the upsampling layer use a complex two-dimensional convolutional network. The operation of two-dimensional complex convolution is mainly by first defining the complex weight matrix W=A+iB, and the input h=x+iy of the complex convolution layer, which is different from the real-number field convolution network. The complex convolution network requires the input to be a complex number ; Then the complex weight matrix is convolved with the input matrix, as shown in the following formula, where * represents the convolution operation.

w*h=(A+iB)*(x+iy)=(A*x-B*y)+i(B*x+A*y)

Written in matrix form, it is shown in the following formula, where R(W*h), I(W*h) represent the real and imaginary parts of the output:

Among them, R(W*h) represents the real part of the output of the complex two-dimensional convolutional network, I(W*h) represents the imaginary part of the output of the complex two-dimensional convolutional network, W is the complex weight matrix, and A is the complex weight The real part of , B is the imaginary part of the complex weight, h represents the input of the complex two-dimensional convolutional network, x and y represent the real and imaginary parts of h, respectively, and * represents the convolution operation.

The branch 1 and branch 2 in the above network structure are trained separately, and the overall structure is fine-tuned. The specific implementation process is as follows:

1) The first stage of network training is phase recovery.

The input is the phase-free total field data, and the output is the predicted scattered field data, specifically:

Input the phase-free total field data as the real part to the imaginary part generation module, generate the corresponding imaginary part, combine the real part with the generated imaginary part, and use it as the input of the complex network, and pass through the complex convolution module and complex batch processing respectively Obtain the predicted scattered field data after the optimization module and the complex activation function module.

The loss function of branch one training is the square of the difference between the real value and the predicted value (MSE Loss); the number of training iterations is 200, the number of input samples per round is 20, and the optimizer is Adam.

2) The second stage is the scattering field imaging stage.

The input is the scattered field data under ideal conditions, and the complex output of the complex Unet is modulo-processed as the final predicted sample imaging data. Specifically, the ideal scattering field data is input into the second branch of the Unet network in the complex domain, and the ideal scattering field data in the complex domain are respectively processed by the complex convolution module, the complex batch processing module and the complex activation function module, and the branch two The output complex data is moduloed to obtain the predicted image.

The loss function of branch two training consists of the square of the difference between the real value and the predicted value (MSE) and the structural similarity loss function (SSIM Loss). When calculating the SSIM of the real value and the predicted value of the sample, it is normalized separately so that the pixel value is in the range of 0-1. The training hyperparameters are the same as above.

3) The third stage is cascade fine-tuning training.

The models trained in stage 1 and stage 2 are cascaded, that is, the phase-free data passes through branch 1 and predicts the output scattering field data as the input of branch 2, and the output is image data.

During the training process, when the error of the test set is less than the threshold 0.001, it is considered that the stage three training has achieved the purpose of fine-tuning, the training is ended, and the trained complex domain Unet network is obtained. The number of training iterations is 20, the number of input samples per round is 20, the optimizer is SGD, the learning rate is 0.01, and the momentum coefficient is 0.9.

4) The model after cascade training is the final prediction model. For the phase-free data to be imaged, as shown in Figure 4, the phase-free data is used as the input of branch 1 in the trained complex domain Unet network, and The scattered field data is predicted and used as the input of branch two, and the complex data output by branch two is moduloed to obtain the generated image.

In one embodiment of the present invention, the test sets Emnist and Austria are used in sequence to test the generalization performance of the network; the test sets of Emnist and Austria are respectively 100 and 10, and the dielectric constant range is 1-2. The test results of this embodiment are compared with the real number network, the BPS algorithm, and the BPS+Unet network, as shown in Figures 5, 6, and 7. Figure 5 shows the test set Emnist results, Figure 6 shows the test set Austria results, and Figure 7 shows the results of different permittivity tests in Austria.

Among them, the first line in Figure 5 and Figure 6 is the output image of the real number network; the second line is the output image of the BPS algorithm; the third line is the output image of BPS+UNet, and the bps and the scattering field of the second line are used as input; the fourth line is complex Network image; the fifth row is a real sample image;

The numbers above the images represent structural similarity SSIM loss, mean square error MSE loss, mean absolute error MAE loss, respectively.

The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be based on the appended claims.

Claims (7)

1. The phase-free data imaging method based on the complex neural network is characterized by comprising the following steps of:

1) Collecting a real sample image, generating phase-free total field data and ideal scattered field data by the real sample image, and forming a training set by the phase-free total field data, the ideal scattered field data and the real sample image;

2) Establishing a double-branch complex domain Unet network, wherein a branch I comprises an imaginary part generating module and a first complex network, and a branch II is a second complex network, and the first complex network and the second complex network are composed of a complex convolution module, a complex batch processing module and a complex activation function module;

the complex convolution module adopts a Unet network and comprises a downsampling coding structure formed by a plurality of downsampling layers, an upsampling decoding structure formed by a plurality of upsampling layers and jump connection; the number of the downsampling layers is equal to that of the upsampling layers; the number of the downsampling layers and the upsampling layers in the complex convolution module of the first branch is recorded as n, and the number of the downsampling layers and the upsampling layers in the complex convolution module of the second branch is recorded as m;

in a complex convolution module of the first branch, combining a real part corresponding to the phase-free total field data with a generated imaginary part to serve as an input of a first-layer downsampling layer, taking an output of a last downsampling layer as an input of a next downsampling layer, and taking an output of a last downsampling layer as an input of a first-layer upsampling; the output of the i-th layer lower sampling layer and the output of the n-i-th layer upper sampling layer are fused in a jump connection mode, and the fusion result is used as the input of the next layer upper sampling layer until the output of the last layer upper sampling layer is obtained as the output of the complex convolution module;

in the complex convolution module of the second branch, firstly taking scattered field data as input of a first layer of downsampling layer, then taking output of a last layer of downsampling layer as input of a next layer of downsampling layer, and finally taking output of the last layer of downsampling layer as input of the first layer of upsampling; the output of the ith layer of lower sampling layer and the output of the mth-ith layer of upper sampling layer are fused in a jump connection mode, and the fusion result is used as the input of the next layer of upper sampling layer until the output of the last layer of upper sampling layer is obtained as the output of the complex convolution module;

in the complex convolution module, a complex two-dimensional convolution network is adopted by a downsampling layer and an upsampling layer, and a calculation formula is as follows:

wherein R (W.times.h) represents the real part of the complex two-dimensional convolution network output, I (W.times.h) represents the imaginary part of the complex two-dimensional convolution network output, W is a complex weight matrix, A is the real part in complex weight, B is the imaginary part in complex weight, h represents the input of the complex two-dimensional convolution network, x and y represent the real part and the imaginary part of h respectively, and x represents convolution operation;

3) Training branch one:

inputting the non-phase total field data as a real part into a branch I of a complex domain Unet network, firstly generating an imaginary part corresponding to the real part through an imaginary part generating module, and combining the real part and the generated imaginary part as the input of a first complex network to obtain predicted scattered field data; calculating loss according to the predicted scattered field data and the ideal scattered field data, and training the branch I;

4) Training the second branch:

inputting ideal scattered field data into a branch II of a complex domain Unet network, processing by a second complex network, outputting complex data, and then taking a model to obtain a predicted image; calculating loss according to the predicted image and the real sample image, and training the second branch;

5) Performing fine tuning training on a complex domain Unet network:

the trained branch I and the branch II are cascaded, the predicted scattered field data which is predicted and output after the phase-free total field data passes through the branch I is used as the input of the branch II, and the predicted image data is output; calculating loss according to the predicted image and the real sample image, and performing fine adjustment on the first branch and the second branch which are cascaded in the complex domain Unet network to obtain a trained complex domain Unet network;

6) Aiming at the non-phase data to be imaged, taking the non-phase data as the input of a first branch in a trained complex domain Unet network, obtaining predicted scattered field data, taking the predicted scattered field data as the input of a second branch, and taking the modulus of the complex data output by the second branch to obtain a generated image.

2. The method of phase-free data imaging based on a complex neural network of claim 1, wherein the real sample image of step 1) has a dielectric constant in the range of 1.0-2.0.

3. The method of claim 1, wherein the complex activation function module applies the activation function in real and imaginary parts, respectively.

4. The method of claim 1, wherein the loss function uses a mean square error loss when training the pair of branches.

5. The method of claim 1, wherein the sum of the structural similarity loss and the mean square error loss is taken as the total loss when training the two branches.

6. The method for phase-free data imaging based on a complex neural network according to claim 5, wherein when the structural similarity loss is calculated, respectively performing normalization processing on the predicted image and the real sample image to enable the pixel value to be in a range of 0-1.

7. The complex neural network-based non-phase data imaging method of claim 1, wherein the dimensions of the real sample images in the training set are uniform, and the non-phase total field data and the fringe field data generated from the real sample images are uniform.

Images