CN110136062A - A Super-resolution Reconstruction Method for Joint Semantic Segmentation - Google Patents

Abstract

The present invention proposes an image super-resolution reconstruction method combined with semantic segmentation, which uses the intermediate results and final results of low-quality images generated during semantic segmentation to perform super-resolution reconstruction, and performs large-scale super-resolution reconstruction time can have a more realistic effect. Since the high-level semantic information of the image is the inherent information of the image, it contains a large number of category priors at the pixel level, so it can be used as the constraint information in the super-resolution reconstruction process to improve the quality of its reconstruction results. The present invention combines image super-resolution reconstruction, a low-level problem of computer vision, with image semantic segmentation as a high-level problem, and uses various information generated after the image is semantically segmented to constrain and coordinate the process of super-resolution reconstruction. Enhancement solves the problem of lack of authenticity in the reconstruction of low-resolution images under the condition of large zoom factors, and has a higher improvement in subjective quality evaluation.

Description

A Super-resolution Reconstruction Method for Joint Semantic Segmentation

technical field

The invention relates to the technical field of image processing, in particular to a method for image super-resolution reconstruction using semantic segmentation.

Background technique

Image super-resolution reconstruction refers to the use of various technical means to convert low-resolution images into high-resolution images, recover more high-frequency information, and make the image have clearer texture and details. The image super-resolution reconstruction method has been developed for half a century since it was first proposed. Many image super-resolution reconstruction methods can be roughly divided into three categories according to their principles: interpolation-based methods, reconstruction-based methods, and learning-based methods. Methods.

Interpolation-based methods link the super-resolution reconstruction problem with the image interpolation problem and are the most straightforward methods in super-resolution reconstruction. Common interpolation methods include nearest neighbor interpolation method, bilinear interpolation method, bicubic interpolation method, etc. The core idea is to search the points in the target image for their related points in the source image according to the scaling relationship, and then calculate the pixel value of the target point by interpolating the pixel values of the relevant points in the source image. Based on the interpolation method, the advantage is that it is very simple and intuitive, and the running speed is fast. The disadvantage is that the adaptability is relatively poor, it is not easy to add the prior information of the image, and it is easy to introduce additional noise, so that the reconstructed image lacks details and produces blur and jagged. And so on.

The reconstruction-based method has received the most extensive attention and research. This method assumes that the high-resolution image is obtained through appropriate motion transformation, blur and noise to obtain the low-resolution image, and transforms the super-resolution reconstruction problem into a constraint The optimization problem of the cost function under the condition. The core idea is to start from the degradation model of the image, use regularization and other methods to extract the key information in the low-resolution image, and combine the prior knowledge of the unknown super-resolution image to constrain the generation of the super-resolution image. This type of method only needs some local a priori assumptions during reconstruction, which alleviates the blurring or jagged effect produced by the interpolation method to a certain extent, but when the magnification is too large, the degradation model cannot well provide the reconstruction required. prior knowledge, resulting in the lack of high-frequency information in the reconstruction results.

Learning-based methods are a hot research direction of super-resolution algorithms in recent years. The basic idea is to learn a joint system model by training a set of training sets that include both high-resolution images and low-resolution images, and use the learned model to perform super-resolution reconstruction on similar low-resolution images. , to improve the image resolution. The learning-based method makes full use of the prior knowledge of the image, can recover more high-frequency information in the low-resolution image, and obtain better reconstruction results than the other two methods. Among all learning-based methods, the super-resolution reconstruction method based on deep learning has achieved excellent results in recent years.

Although today's single-image super-resolution reconstruction technology relies on deep learning to achieve great breakthroughs in accuracy and speed, its effect will decline when dealing with more complex low-resolution images. For example: when the processed low-resolution image contains many objects and there is a large part of overlap and occlusion between the objects, the existing methods cannot well divide the boundary between the overlapping and occlusion objects, resulting in the texture of the reconstruction result Insufficient detail, even multiple overlapping objects are reconstructed as one.

Contents of the invention

In order to solve the above-mentioned problems, the present invention proposes a new super-resolution reconstruction method combined with semantic segmentation. Semantic segmentation is one of the basic tasks in computer vision. Its purpose is to divide the visual input into different semantically interpretable categories. For an image, it is to divide the pixels in the image into different categories. Based on the characteristics of semantic segmentation to classify pixels, the super-resolution reconstruction method combined with semantic segmentation can better deal with low-resolution images with multiple overlapping and occluding objects.

Aiming at the deficiencies in the prior art, the present invention provides a method for super-resolution reconstruction of low-resolution images, comprising the following steps:

Step 1, constructing a low-resolution semantic segmentation data set, the low-resolution semantic segmentation data set includes a low-resolution image and a corresponding semantic layout;

Step 2, use the low-resolution semantic segmentation dataset to train the semantic segmentation network;

Step 3, constructing a data set for training a super-resolution reconstruction network, the data set for training a super-resolution reconstruction network includes a semantic layout map of a low-resolution image, a semantic feature map and a corresponding high-resolution image, The semantic layout map and semantic feature map of the low-resolution image are obtained by inputting the low-resolution image into the semantic segmentation network trained in step 2;

Step 4, take the semantic layout map and semantic feature map as input, and the high-resolution image corresponding to the semantic layout map as the real value, train the super-resolution reconstruction network so that it can output the corresponding high-resolution reconstruction according to the input semantic layout map result;

Step 5, input a low-resolution image to be reconstructed into the semantic segmentation result obtained in step 2, obtain its semantic layout map and semantic feature map, and then input it into the super-resolution reconstruction network trained in step 4 , and finally get the reconstructed high-resolution image.

Further, the low-resolution semantic segmentation dataset described in step 1 is the low-resolution image and semantic layout diagram obtained by downsampling the high-resolution image and semantic layout diagram in the usual semantic segmentation dataset with the same scaling factor Constitutes a low-resolution semantic segmentation dataset.

Further, the semantic segmentation network in step 2 is a fully convolutional network, which is obtained by changing the fully connected layer in VGG16 to a convolutional layer. The specific network structure is: convolutional layer×2+pooling layer + convolutional layer × 2 + pooling layer + convolutional layer × 3 + pooling layer + convolutional layer × 3 + pooling layer + convolutional layer × 3 + pooling layer + convolutional layer × 2 + deconvolution The product layer, where the convolution kernel size of the convolutional layer is 3×3, and the pooling layer uses the maximum pooling.

Further, the weights of the fully convolutional network are initialized to the weights in the pre-trained VGG16; the loss function optimized in the training is the sum of the deviations of the pixel prediction values of the last layer of the network; the specific parameters of the training are: training The batch size of is 20, and the Adam algorithm with the momentum of 0.9 and the decay rate of 10 ^-4 is used for optimization, and the learning rate of the network is 10 ^-4 .

Further, the super-resolution reconstruction network described in step 4 is a cascaded reconstruction network composed of a series of cascaded reconstruction modules, and the cascaded reconstruction modules operate at increasing resolutions, wherein each reconstruction module consists of 3 network Layer composition: the first layer is a feature fusion layer, which is used to fuse the input semantic layout map and semantic feature map with the output result of the previous layer; the next two layers are with 3×3 convolution kernel, layer regularization and correction The convolutional layer of the linear unit is used to reconstruct the fused features.

Further, the specific operational relationship between the reconstruction modules in the super-resolution reconstruction network is as follows,

The first reconstruction module takes the semantic layout map and semantic feature map downsampled to the current resolution as input, and outputs a result of the current resolution, which is regarded as the feature map after merging and convolution, and the subsequent reconstruction module The result of the previous module and the downsampled semantic layout map and semantic feature map are used as input to output a new result. After several such processes, the final output of the reconstruction module is the result of super-resolution reconstruction. , the mathematical description of this process is as follows:

Among them, O _i represents the output of the i-th reconstruction module, f represents the convolution and other operations in the reconstruction module, L represents the semantic layout map, F represents the semantic feature map, Indicates feature fusion.

Further, the loss function used when training the super-resolution reconstruction network in step 4 is,

Among them, I is the high-resolution image representing the real value, f is the cascade reconstruction network to be trained, θ is the parameter set in f, L is the input semantic layout map, Φ is the trained visual perception network, the visual The perception network is a VGG network, Φ _l represents the convolutional layer in the visual perception network, and λ _l is a hyperparameter that controls the weight, and its value is adjusted during the training process.

Further, when training the super-resolution reconstruction network, the specific settings are: the overall number of iterations is 200 generations; the learning rate of the model is 10 ^-4 , and every 100 generations of training, the learning rate is reduced to half; the momentum is 0.9, The Adam algorithm with a decay rate of 10 ^-4 is optimized.

Compared with the prior art, the present invention has the following advantages and positive effects:

Since the high-level semantic information of the image is the inherent information of the image, it contains a large number of category priors at the pixel level, so it can be used as the constraint information in the super-resolution reconstruction process to improve the quality of its reconstruction results. The present invention combines image super-resolution reconstruction, a low-level problem of computer vision, with image semantic segmentation as a high-level problem, and uses various information generated after the image is semantically segmented to constrain and coordinate the process of super-resolution reconstruction. Enhancement solves the problem of lack of authenticity in the reconstruction of low-resolution images under the condition of large zoom factors, and has a higher improvement in subjective quality evaluation.

Description of drawings

FIG. 1 is a network structure diagram of a fully convolutional network in an embodiment of the present invention.

FIG. 2 is a block diagram of a cascaded reconstruction network in an embodiment of the present invention.

Fig. 3 is the overall flow chart of the present invention.

Fig. 4 is a comparison diagram of visual effects between the present invention and the comparison method, wherein (a) is Bicubic, (b) is SRCNN, (c) is SRDenNet, (d) is SRGAN, (e) is the present invention .

Detailed ways

In order to facilitate those of ordinary skill in the art to understand and implement the present invention, the present invention will be described in further detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described here are only used to illustrate and explain the present invention, and are not intended to limit this invention.

The present invention combines the characteristics of two computer vision tasks, image semantic segmentation and image super-resolution reconstruction, uses the features generated by image semantic segmentation as prior information for super-resolution reconstruction, and proposes a joint semantic segmentation image Super-resolution reconstruction methods. The overall flow process that this method explains is as shown in Figure 3, and this method can be realized with computer software technology, and embodiment is carried out a specific setting forth to the flow process of the present invention with the training of network as main content, as follows:

Step 1, constructing a low-resolution semantic segmentation dataset, the low-resolution semantic segmentation dataset includes low-resolution pictures and corresponding semantic layout maps. The usual semantic segmentation datasets contain high-resolution images and their corresponding semantic layouts, and the high-resolution images and corresponding semantic layouts are uniformly down-sampled to obtain low-resolution semantic segmentation datasets.

In the specific implementation, image processing software is used to read in all high-resolution pictures and their corresponding semantic layout diagrams, unify their sizes, and then use bicubic interpolation to down-sample all high-resolution pictures with a scaling factor of 4. Then use the same method to downsample the corresponding semantic layout map to the same resolution. In this way, a low-resolution semantic segmentation dataset consisting of low-resolution images and corresponding semantic layout maps is obtained.

Step 2, use the low-resolution semantic segmentation dataset to train the semantic segmentation network. The object processed by the usual semantic segmentation network is a high-resolution image, and the low-resolution data set obtained in step 1 is used to train the semantic segmentation network so that it can output the corresponding accurate semantic layout when inputting a low-resolution image;

In this embodiment, the semantic segmentation network is described by taking a fully convolutional network (FCN, Fully Convolutional Networks) as an example. The fully convolutional network is a convolutional neural network without a fully connected layer, which can predict and classify each pixel in the image to obtain semantic segmentation results. Its network structure is shown in Figure 1. In particular, the fully convolutional network in this embodiment is improved from the VGG16 classification network, which is obtained by changing the fully connected layer in VGG16 to a convolutional layer. In the fully convolutional network, record x _ij as the data vector at the position (i, j) of a certain layer of the network, y _ij is the data vector at the position (i, j) of the next network layer, and y _ij is determined by x _ij through the following formula It can be concluded that:

y _ij =f _ks ({xsi+δi,sj+δj}0≤δi,δj≤k)

Among them, k represents the size of the convolution kernel, s represents the step size or downsampling factor of the convolution kernel, si, sj represent the position coordinates of the data vector at the position of the original network layer (i, j) after convolution or pooling operations Changes related to s, δi, δj represent the spatial displacement generated during convolution or pooling, usually caused by zero padding operations. f _ks determines the type of network layer, which may be matrix multiplication for convolution or pooling, or space maximization for maximum pooling, or element-wise non-linear mapping of activation functions. For a fully convolutional network, the functions implemented by each network layer can be summarized by the above formula.

The specific implementation of training the full convolutional network is as follows:

1. Build your network. In this embodiment, the main body of the fully convolutional network is composed of VGG16, and its network structure is: convolutional layer×2+pooling layer+convolutional layer×2+pooling layer+convolutional layer×3+pooling layer+convolution Layer×3+pooling layer+convolution layer×3+pooling layer+convolution layer×2+deconvolution layer. The convolution kernel size of the convolutional layer is 3×3, and the pooling layer uses the maximum pooling. As the convolutional layer deepens, the size of the data becomes smaller and the number of channels increases.

2. Initialize the weights in the network. Different from random initialization in general, the weights in this embodiment are initialized to the weights in the pre-trained VGG16.

3. Train the network. The loss function optimized during training is the sum of the deviations of the pixel prediction values of the last layer of the network. In this embodiment, the specific parameters of the training are: the training batch size is 20, the Adam algorithm with the momentum of 0.9 and the decay rate of 10 ⁻⁴ is used for optimization, and the learning rate of the network is 10 ⁻⁴ .

Step 3, constructing a data set for training a super-resolution reconstruction network, the data set for training a super-resolution reconstruction network includes a semantic layout map of a low-resolution picture and a corresponding high-resolution picture. Input the low-resolution image into the semantic segmentation network obtained in step 2, and obtain its semantic segmentation result—semantic layout map. In addition, the intermediate result generated in the process of semantic segmentation, the semantic feature map, can also be obtained. The semantic layout map and the corresponding feature map and the corresponding high-resolution photos will form a new dataset to train the super-resolution reconstruction network. After the image is input into the semantic segmentation network, the final output of the network is the semantic layout map, and the semantic feature map needs to be extracted from different network layers of the semantic segmentation network. In this embodiment, the selected semantic feature map is the feature in the convolutional layer before the pooling layer in the full convolutional network.

Step 4, take the semantic layout map and semantic feature map as input, and the high-resolution image corresponding to the semantic layout map as the real value, train a super-resolution reconstruction network so that it can output the corresponding high-resolution image according to the input semantic layout map reconstruction results;

In this embodiment, what is selected as the super-resolution reconstruction network is a cascaded reconstruction network composed of a series of cascaded reconstruction modules, the structure of which is shown in FIG. 2 . Each reconstruction module operates at a different resolution. The resolution of the first module is set to 8×16, and the resolution of subsequent modules is doubled in turn. After 5 reconstruction modules, the final output resolution is 256×512. The first reconstruction module takes the semantic layout map and feature map downsampled to the current resolution as input, and outputs a result at the current resolution, which can be regarded as a merged and convoluted feature map. The subsequent reconstruction module will take the result of the previous module together with the downsampled semantic layout map and feature map as input, and output a new result. After several such processes, the final output of the reconstruction module is the result of super-resolution reconstruction. The mathematical description of this process is as follows:

Among them, O _i represents the output of the i-th reconstruction module, f represents the convolution and other operations in the reconstruction module, L represents the semantic layout map, F represents the semantic feature map, Indicates feature fusion.

Each reconstruction module consists of three network layers: the first layer is a feature fusion layer, which is used to fuse the input semantic layout map and semantic feature map with the output result of the previous layer; Convolutional layers with kernels, layer regularization, and rectified linear units are used to reconstruct the fused features. Except for the last reconstruction module, the structure of each reconstruction module is the same, but the focus of reconstruction of each module is different, because the input feature map contains different levels of information.

The cascade reconstruction network uses the semantic layout map as a framework, and uses various information contained in the feature map to reconstruct the details of the image. Therefore, the loss function used in training is also different from the general super-resolution reconstruction method, which is different from the conventional average The square error loss function directly compares the reconstruction result with the original high-definition image pixel by pixel. The cascaded reconstruction network uses a loss function called perceptual loss. The purpose is to compare the feature difference between the reconstruction result and the real value in the visual perception network. , which is defined as:

Among them, I is the high-resolution image representing the real value, f is the cascaded reconstruction network to be trained, θ is the parameter set in f, L is the input semantic layout map, λ _l is the hyperparameter to control the weight, in the training In the process, its value will be adjusted with the training process. Φ is the trained visual perception network, and Φ _l represents the convolutional layer in the visual perception network. The visual perception network is an image classification network that has been trained with a large amount of data. The ability to correctly classify objects in the input image, usually using the publicly released VGG series network, can be found on its official website. After training with a perceptual loss function, the cascaded reconstruction network can reconstruct more realistic reconstruction results.

The specific implementation scheme of training the super-resolution reconstruction network is as follows:

1. Build your network. The super-resolution reconstruction network in this embodiment is formed by cascading a series of reconstruction modules, and each cascade module has the same structure. The constructed reconstruction module consists of a three-layer network. The first layer of the network fuses the input features, and the last two layers of the network are convolutional layers with a convolution kernel size of 3×3 and layer regularization and LRELU activation. function.

2. Initialize the weights in the network and randomly initialize the weights in the network.

3. Train the network. The function that needs to be optimized in training is perceptual loss. The specific setting of training is that the overall number of iterations is 200 generations; the learning rate of the model is 10 ^-4 , and every 100 generations of training, the learning rate is reduced to half; the momentum is 0.9 , the Adam algorithm with a decay rate of 10 ^-4 is optimized.

Step 5, use the trained network for super-resolution reconstruction. The specific implementation plan is: input a low-resolution image to be reconstructed into the semantic segmentation network obtained in step 2, obtain its semantic layout map and semantic feature map, and then input it into the super-resolution image obtained in step 4 Reconstruct the network and finally get the reconstructed high-resolution image.

In order to verify the technical effect of the present invention, the Cityscapes urban landscape data set is used for verification. There are 2975 high-resolution images and corresponding fine semantic layout maps in the Cityscapes dataset. 1000 of the 2975 images are used to train the semantic segmentation network, and the remaining 1975 images are used to train the super-resolution reconstruction network. The methods used for comparison are bicubic interpolation (Bicubic), super-resolution convolutional neural network SRCNN (C.Dong, C.C.Loy, K.He, and X.Tang. Image super-resolution using deep convolutional networks.IEEE Transactions onPattern Analysis and Machine Intelligence, 38(2):295–307, 2016.), super-resolution dense neural network SRDenNet (T.Tong, G.Li, X.Liu, and Q.Gao, "Image super-resolution using dense skip connections , "in 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017, pp.4809–4817.) and super-resolution generative confrontation network SRGAN (C.Ledig, L.Theis, F.Huszar, J.Caballero, A . Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802, 2016.2).

Table 1 shows the objective and subjective evaluation indicators corresponding to each method when the scaling factor is 4, including PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) and MOS (Mean Subjective Opinion Score). It can be seen from Table 1 that the method of the present invention has a stable improvement in the subjective quality of the restored image.

Table 1 Objective and subjective scoring of each method

The comparison of visual effects is shown in Figure 4. From the comparison results, it can be known that the method of the present invention is more vivid and specific in reconstruction details than other methods, has a higher sense of reality and visual persuasion as a whole, and maintains the objective evaluation index Under the basically unchanged level, the subjective evaluation indicators have been greatly improved.

It should be understood that the parts not described in detail in this specification belong to the prior art.

It should be understood that the above descriptions for the embodiments are relatively detailed, and should not be considered as limiting the protection scope of the present invention. Under the enlightenment of the present invention, those skilled in the art can also make replacements or modifications without departing from the protection scope of the claims of the present invention, but still belong to the protection scope of the present invention. The scope of protection should be determined by the appended claims.

Claims (8)

1. a kind of super resolution ratio reconstruction method of combination semantic segmentation, which comprises the steps of:

Step 1, the semantic segmentation data set of low resolution is constructed, the semantic segmentation data set of the low resolution includes low resolution Rate image and corresponding semantic layout；

Step 2, semantic segmentation network is trained using the semantic segmentation data set of low resolution,；

Step 3, the data set for training Super-resolution reconstruction establishing network is constructed, it is described for training Super-resolution reconstruction establishing network Data set includes the semantic layout, semantic feature figure and corresponding high-definition picture of low-resolution image, wherein low resolution The semantic layout of rate image, semantic feature figure are by being input to trained semantic segmentation in step 2 for low-resolution image It is obtained in network；

Step 4, using semantic layout and semantic feature figure as input, the corresponding high-definition picture of semantic layout is as true Real value, training Super-resolution reconstruction establishing network can export corresponding super-resolution reconstruction knot according to the semantic layout of input Fruit；

Step 5, a low resolution picture to be reconstructed is input to semantic segmentation obtained in step 2 as a result, obtaining its language Then adopted layout and semantic feature figure are input to the Super-resolution reconstruction establishing network that training obtains in step 4, finally obtain High-definition picture after reconstruction.

2. a kind of super resolution ratio reconstruction method of combination semantic segmentation according to claim 1, it is characterised in that: step 1 The semantic segmentation data set of the low resolution is by the high-definition picture and semantic cloth in common semantic segmentation data set Office's figure carries out the down-sampling of the same zoom factor, and obtained low-resolution image and semantic layout constitute the language of low resolution Adopted partitioned data set.

3. a kind of super resolution ratio reconstruction method of combination semantic segmentation according to claim 1, it is characterised in that: step 2 In semantic segmentation network be full convolutional network, which is that the full articulamentum in VGG16 is changed to institute after convolutional layer , specific network structure are as follows: the convolutional layer × pond the 2+ layer+convolutional layer × pond the 2+ layer+convolutional layer × pond 3+ layer+convolutional layer × The pond the 3+ layer+convolutional layer × pond 3+ layer+convolutional layer × 2+ warp lamination, wherein the convolution kernel size of convolutional layer is 3 × 3, pond Change layer using maximum pond.

4. a kind of super resolution ratio reconstruction method of combination semantic segmentation according to claim 3, it is characterised in that: full convolution The weight initialization of network is by the weight in the VGG16 of pre-training；The loss function optimized in training is that network is last One layer of the sum of the deviation of pixel predictors；Trained design parameter are as follows: batch size of training is 20, uses momentum for 0.9, declines Variability is 10^-4Adam algorithm optimize, the learning rate of network is 10^-4。

5. a kind of super resolution ratio reconstruction method of combination semantic segmentation according to claim 1, it is characterised in that: step 4 The Super-resolution reconstruction establishing network is that network, cascade reconstruction mould are rebuild in a series of cascade as composed by cascade reconstruction modules Block is run with incremental resolution ratio, wherein each reconstruction module is made of 3 network layers: first layer is characterized fused layer, effect For the semantic layout of input and semantic feature figure are merged with the output result of preceding layer；Two layers next is with 3 × 3 convolution Core, layer regularization and the convolutional layer for correcting linear unit, as being rebuild to fused feature.

6. a kind of super resolution ratio reconstruction method of combination semantic segmentation according to claim 5, it is characterised in that: super-resolution The carrying out practically relationship that rate is rebuild between the reconstruction module in network is as follows,

First is rebuild module using the semantic layout and semantic feature figure that are down sampled to current resolution as input, output one A current resolution as a result, this result regard as through merging and convolution after characteristic pattern, subsequent reconstruction module will be previous The result of a module and the semantic layout after down-sampling and semantic feature figure export a new knot together as input Fruit, by multiple such process, the result of final reconstruction module output is super-resolution rebuilding as a result, this process Mathematical description is as follows:

Wherein, O_iIndicate the output of i-th of reconstruction module, f indicates the operation such as convolution rebuild in module, and L indicates semantic layout Figure, F indicate that semantic feature figure, ⊕ indicate Fusion Features.

7. a kind of super resolution ratio reconstruction method of combination semantic segmentation according to claim 5, it is characterised in that: step 4 The loss function used when middle trained Super-resolution reconstruction establishing network is,

Wherein, I is the high-definition picture for representing true value, and f is that network is rebuild in cascade to be trained, and θ is the parameter set in f It closes, L is the semantic layout of input, and Φ is trained visual perception network, which is VGG network, Φ_l Indicate the convolutional layer in visual perception network, λ_lFor the hyper parameter for controlling weight, its value is with training process in the training process It is adjusted.

8. a kind of super resolution ratio reconstruction method of combination semantic segmentation according to claim 5, it is characterised in that: training is super When resolution reconstruction network, be specifically configured to: whole the number of iterations was 200 generations；The learning rate of model is 10^-4, and every warp After the training of 100 generations, learning rate is reduced to half；Use momentum for the 0.9, rate of disintegration 10^-4Adam algorithm optimize.