CN109711413B - Image semantic segmentation method based on deep learning - Google Patents

Abstract

An image semantic segmentation method based on deep learning, which consists of four parts: data set processing, construction of deep semantic segmentation network, deep semantic segmentation network training and parameter learning, and semantic segmentation of test images. The present invention takes the RGB image and the grayscale image of the input image as the input of the network model, makes full use of the edge information of the grayscale image, and effectively increases the richness of the input features; Based on the local features of the image, capture more context dependencies and global feature information; add coordinate information to the feature map through the first coordinate channel module and the second coordinate channel module, enrich the coordinate features of the model, and improve the generalization ability of the model , producing semantic segmentation results with high resolution and precise boundaries.

Description

Image Semantic Segmentation Method Based on Deep Learning

technical field

The invention belongs to the technical field of computer vision and deep learning, and in particular relates to an image semantic segmentation method based on deep learning.

Background technique

Image semantic segmentation is to understand and recognize the content of pictures from the pixel level. Its purpose is to establish a one-to-one mapping relationship between each pixel and semantic categories, and perform segmentation based on semantic information. It is widely used in scene understanding and automatic driving. , medical image analysis, robot vision and other fields. Image semantic segmentation is the cornerstone of image understanding, and the quality of its segmentation results will directly affect the processing of subsequent image content. Therefore, the research on image semantic segmentation technology has very important practical significance.

Most of the traditional image semantic segmentation methods rely on manual feature extraction and probabilistic graphical models, such as: random forest, conditional random field (CRF), Markov random field (MRF), etc. These methods can only learn shallow representation information, Accurate and fine segmentation results cannot be produced. Since 2012, with the rapid development of deep learning, image semantic segmentation methods based on convolutional neural networks have become a research hotspot. In 2014, Hariharan et al. proposed the SDS (simultaneous detection and segmentation) method of collaborative target detection and semantic segmentation. This method first uses the MCG method to extract multiple candidate regions in each image, and then uses CNN to extract the bounding box in two ways ( bounding-box) features and foreground area features, and complete the information fusion of the two features, and finally, use the non-maximum constraint NMS (non-maximum suppression) method to generate semantic segmentation results. In addition to the SDS method, there are similar methods such as R-CNN and SPP, which are semantic segmentation methods based on candidate regions, but these methods rely on a large number of region proposals, resulting in very large memory consumption and long training time. The accuracy of semantic segmentation results is low.

In order to further reduce memory overhead and improve semantic segmentation accuracy. In 2015, Long et al. proposed the fully convolutional network model FCN (fully convolutional networks), which converts the last fully connected layer of the deep convolutional neural network into a convolutional layer to form an end-to-end, pixel-to-pixel The fully convolutional network framework has brought image semantic segmentation into a new era. Kendall et al. proposed a deep convolutional encoding-decoding architecture SegNet, which consists of a convolutional encoding network and a deconvolutional decoding network. Each encoder layer corresponds to a decoder layer, and the output of the final encoder is are fed into a soft-max classifier for pixel-by-pixel classification. On the basis of FCN, Chen et al. proposed a more mature semantic segmentation model Deeplab-CRF, which uses an optimized DCNN (deep convolutional neural network) to obtain a rough score map and upsample to the original image through bilinear interpolation. The image size is then iteratively optimized using a fully connected conditional random field (CRF) to obtain fine semantic segmentation results.

Disadvantages of the above semantic segmentation methods: first, the model input is generally an RGB image, and the input is too single, which may lead to the loss of local features; second, these methods are based on convolutional neural networks for feature extraction, and do not make full use of images. The local feature information and global context dependence of the image lead to very rough segmentation edges of the image and very low segmentation accuracy.

Contents of the invention

The technical problem to be solved by the present invention is to overcome the defects of the existing methods and provide an image semantic segmentation method based on deep learning with high segmentation accuracy and strong generalization ability.

The technical solution adopted to solve the above technical problems comprises the following steps:

S1. Data set processing

Divide the image data set into a training image set and a test image set, and perform data enhancement operations on the training image set to increase the number of training images to tens of thousands of units;

S2. Build a deep semantic segmentation network

The deep semantic segmentation network consists of a parallel deep neural network module, a feature fusion module, and a Softmax classification layer. The parallel deep neural network module is used to extract features from the input image, and the feature fusion module combines the output features of the parallel deep neural network. Graph carries out weighted fusion and obtains new feature map, and described Softmax classification layer converts pixel category label prediction score into pixel category label prediction probability distribution map;

Described parallel deep neural network module is made up of the first deep neural network module and the second deep neural network module, and the first deep neural network module and the second deep neural network module network structure are identical, the input of the first deep neural network module Be the RGB image of input image, the input of the second depth neural network module is the grayscale image of input image;

The first deep neural network module is composed of a full convolutional network module, a first coordinate channel module, a first circulation layer module, a second coordinate channel module, a second circulation layer module, and a spatial pyramid pooling module. The structure of the channel module is the same as that of the second coordinate channel module, the structure of the first recurrent layer module is the same as that of the second recurrent layer module, the full convolution network module performs local feature extraction on the input image, and the first recurrent layer module uses In order to capture the context dependencies and global feature information of the image, the first coordinate channel module connects the i, j, r coordinate channels to the feature map output by the full convolutional network module to form a new feature map to learn more coordinate features information and improve the generalization capability of the model, the spatial pyramid pooling module performs convolution operations on the feature map output by the second loop layer module on multiple sampling rates, and extracts feature information of different scale regions;

S3, deep semantic segmentation network training and parameter learning

S31. Network model parameter initialization: use the pre-training model of ResNet101 on the ImageNet dataset to initialize the parameters of the full convolutional network module, use the standard uniform distribution to initialize the parameters of the first cycle layer module and the second cycle layer module, and use the standard The Gaussian distribution initializes the parameters of the convolution layer of the spatial pyramid pooling module;

S32. Use the data-enhanced training image set to train the deep semantic segmentation network, generate a pixel category prediction label probability distribution map, and use the predicted label probability and the original label probability to calculate the prediction loss. Specifically, the mixed loss function L(θ) is used as the objective function.

L(θ)＝L ₁ (θ)+L ₂ (θ)

where L ₁ (θ) is the cross-entropy loss function, L ₂ (θ) is the L2 regularization term, and θ is the parameter of the deep semantic segmentation network;

S33, using the stochastic gradient descent algorithm to optimize the objective function, using the backpropagation algorithm to update the network model parameters, and ending the training until the value of the objective function no longer decreases;

S4. Perform semantic segmentation on the test image

S41. Input the test image set into the deep semantic segmentation network trained in step S3;

S42, the parallel deep neural network module performs feature extraction on the input test image set

The RGB image of the test image is used as the input of the first deep neural network module, and the grayscale image of the test image is used as the input of the second deep neural network module;

The feature extraction process of the first deep neural network module is as follows: the full convolutional network module extracts local features from the RGB image of the test image through atrous convolution, maximum pooling, and convolution operations; the feature map output by the full convolutional network module is passed through The new feature map obtained by the first coordinate channel module is sent to the first circulation layer module for horizontal and vertical scanning, and the global feature information of the image is learned; the feature map output by the first circulation layer module is passed through the second coordinate channel module to obtain new features The image is then sent to the second circulation layer module for horizontal and vertical scanning to capture the global feature information of the image; the feature map output by the second circulation layer module is input into the spatial pyramid pooling module, and convolution operations are performed at multiple sampling rates to extract Feature information of regions of different scales;

The second deep neural network module feature extraction process is the same as the first deep neural network module feature extraction process;

S43. Perform weighted fusion of the feature map output by the first deep neural network module and the feature map output by the second deep neural network module to obtain a new feature map;

S44. Send the result of step S43 to the Softmax classification layer for pixel category label prediction, obtain the object category to which each pixel in the image belongs, and perform bilinear interpolation to upsample to the original image size to obtain a fine semantic segmentation map.

As a preferred technical solution, the first recurrent layer module is composed of two bidirectional threshold recurrent units, and the number of neurons in the bidirectional threshold recurrent unit is 150.

As a preferred technical solution, the spatial pyramid pooling module is composed of four atrous convolutions with different sampling rates, the convolution kernel size of the atrous convolution is 3×3, and the dilation rates are 4, 6, and 8 respectively. , 12.

m为i坐标通道中的任意元素，n为j坐标通道中与m坐标位置相同的元素，将i坐标通道和j坐标通道中的元素线性缩放到[-1,1]范围内。As a preferred technical solution, the i, j, and r coordinate channels in the step S2 are composed of i coordinate channel, j coordinate channel, and r coordinate channel, and the i coordinate channel, j coordinate channel and r coordinate channel are all e× The coordinate matrix of f, the elements in the first row to the eth row of the i coordinate channel are 0, 1, ..., e-1 in sequence, and the elements in the first column to the f column of the j coordinate channel are 0, 1, . .., f-1, e, f take positive integers, and the r coordinate channel is

m is any element in the i coordinate channel, n is the element in the j coordinate channel at the same position as the m coordinate, and the elements in the i coordinate channel and the j coordinate channel are linearly scaled to the range [-1,1].

As a preferred technical solution, the learning rate of the parameter learning in the step S3 is attenuated according to the following formula:

In the formula, t is the number of iterations, l ₀ is the initial learning rate, l _t is the learning rate of the t-th iteration, and power is the momentum of 0.9.

The beneficial effects of the present invention are as follows:

The present invention uses the RGB image and the grayscale image of the input image as the input of the network model, makes full use of the edge information of the grayscale image, and effectively increases the richness of the input features; Based on the local features of the image, capture more context dependencies and global feature information; add coordinate information to the feature map through the first coordinate channel module and the second coordinate channel module, enrich the coordinate features of the model, and improve the generalization ability of the model , producing semantic segmentation results with high resolution and precise boundaries.

Description of drawings

Figure 1 is a flow chart of an image semantic segmentation method based on deep learning.

Fig. 2 is the first deep neural network module.

Figure 3 is a semantic segmentation map of some test images in the WeizmannHorse dataset.

Figure 4 is a semantic segmentation diagram of some test images in the StanfordBackground dataset.

Detailed ways

The present invention will be described in further detail below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to these embodiments.

Example 1

The WeizmannHorse dataset is an image segmentation dataset consisting of 328 images. Some images in the dataset are shown in Figure 3. The network model is trained using the Pytorch platform, and the code is written on python. This example is based on image semantics of deep learning The segmentation method, as shown in Figure 1, has the following steps:

S1. Data set processing

Randomly select 200 images from the WeizmannHorse dataset as a training image set, and the remaining 128 images as a test image set, and perform data enhancement operations on the training image set to increase the number of training images to 11,000;

S2. Build a deep semantic segmentation network

The deep semantic segmentation network consists of a parallel deep neural network module, a feature fusion module, and a Softmax classification layer. The parallel deep neural network module is used to extract features from the input image, and the feature fusion module weights the output feature maps of the two parallel deep neural networks. A new feature map is obtained by fusion, and the Softmax classification layer converts the pixel category label prediction score into a pixel category label prediction probability distribution map;

The parallel deep neural network module is made up of the first deep neural network module and the second deep neural network module, and the first deep neural network module and the second deep neural network module have the same structure, and the input of the first deep neural network module is the input image RGB image, the input of the second depth neural network module is the gray scale image of input image;

In Figure 2, the first deep neural network module is composed of a full convolutional network module, a first coordinate channel module, a first circulation layer module, a second coordinate channel module, a second circulation layer module, and a spatial pyramid pooling module. The first coordinate channel module has the same structure as the second coordinate channel module, and the first circulation layer module has the same structure as the second circulation layer module;

The full convolutional network module performs local feature extraction on the input image. The full convolutional network module is composed of the first convolution group to the fifth convolution group of the Resnet101 network in the Deeplab_largeFOV model. Among them, the first convolution group to the third convolution group The convolution group uses convolution operation and maximum pooling operation, and the fourth convolution group and the fifth convolution group use convolution operation and hole convolution operation;

同样地，使用另外一个双向门限递归单元沿着复合特征图

的每行进行水平扫描，一个自左向右扫描、一个自右向左扫描，每次读取一个区域块，并将输出预测按坐标索引连接起来得到一个新的复合特征图

新的复合特征图

具有来自整个图像的上下文信息；The first loop layer module is composed of two bidirectional threshold recurrent units. The number of neurons in the bidirectional threshold recurrent unit is 150, which is used to capture the context dependence and global feature information of the image; The graph X is divided into G×K non-overlapping area blocks, where G and K are equal to the length and width of the feature map X respectively; then a bidirectional threshold recursive unit is used to scan vertically along each column of the feature map X, and a Downscan, a bottom-up scan, reads one area block at a time, and connects the output predictions obtained by scanning according to the coordinate index to obtain a composite feature map

Similarly, use another bidirectional threshold recurrent unit along the composite feature map

Each line of each row is scanned horizontally, one scans from left to right, and the other scans from right to left, each time a region block is read, and the output predictions are connected by coordinate index to obtain a new composite feature map

New Composite Feature Map

have contextual information from the entire image;

m为i坐标通道中的任意元素，n为j坐标通道中与m坐标位置相同的元素，将i坐标通道和j坐标通道中的元素线性缩放到[-1,1]范围内；The first coordinate channel module connects the feature map output by the full convolutional network module to the i, j, r coordinate channels to form a new feature map to learn more coordinate feature information and improve the generalization ability of the model; i, j, r The coordinate channel is composed of i coordinate channel, j coordinate channel and r coordinate channel. The i coordinate channel, j coordinate channel and r coordinate channel are all coordinate matrices of e×f. The elements in the first row to the eth row of the i coordinate channel are 0, 1, ..., e-1, the elements in the first column to the fth column of the j coordinate channel are 0, 1, ..., f-1 in turn, e, f take positive integers, and the r coordinate channel is

m is any element in the i coordinate channel, n is the element in the j coordinate channel with the same position as the m coordinate, and the elements in the i coordinate channel and the j coordinate channel are linearly scaled to the range of [-1,1];

The spatial pyramid pooling module performs convolution operations on the feature maps output by the second loop layer module at multiple sampling rates to extract feature information of regions of different scales. This module consists of four hole convolutions with different sampling rates. The hole convolution The convolution kernel size of the product is 3×3, and the expansion rates are 4, 6, 8, and 12 respectively;

S3, deep semantic segmentation network training and parameter learning

S31. Network model parameter initialization: use the pre-training model of ResNet101 on the ImageNet dataset to initialize the parameters of the full convolutional network module, use the standard uniform distribution to initialize the parameters of the first cycle layer module and the second cycle layer module, and use the standard The Gaussian distribution initializes the parameters of the convolution layer of the spatial pyramid pooling module;

S32. Crop the size of the image in the training image set after data enhancement to 330×330, use the cropped training image to train the deep semantic segmentation network, generate a pixel category prediction label probability distribution map, and use the predicted label probability and the original label probability to calculate the prediction Loss, specifically using the hybrid loss function L(θ) as the objective function,

L(θ)＝L ₁ (θ)+L ₂ (θ)

where L ₁ (θ) is the cross-entropy loss function, L ₂ (θ) is the L2 regularization term, and θ is the parameter of the deep semantic segmentation network;

The cross-entropy loss function L ₁ (θ) of this embodiment is:

是原始标签概率向量，N是每张图片的像素个数，N为330×330＝108900，B是批大小，B为10，C是像素类别数，C为2，ln(.)是求自然对数；where _ypq is the predicted label probability vector,

is the original label probability vector, N is the number of pixels in each picture, N is 330×330=108900, B is the batch size, B is 10, C is the number of pixel categories, C is 2, ln(.) is seeking nature logarithm;

The L2 regularization term L ₂ (θ) of this embodiment is:

In the formula, λ is a regularization coefficient and is a positive number, N is the number of pixels of each image, N is 330×330=108900, B is the batch size, B is 10, S is the number of parameters of w and S is positive Integer, w is the weight parameter;

S33. Use the stochastic gradient descent algorithm to optimize the objective function, use the backpropagation algorithm to update the network model parameters, and end the training until the value of the objective function no longer decreases. In order to accelerate the model convergence, introduce a learning rate for parameter learning, and the learning rate is according to the following formula For attenuation:

where t is the number of iterations and t≤35000, l ₀ is the initial learning rate, l ₀ is 0.003, l _t is the learning rate of the t-th iteration, the gradient decay is 0.0001, and power is the momentum of 0.9;

S4. Perform semantic segmentation on the test image

S41. Input the test image set into the deep semantic segmentation network trained in step S3;

S42, the parallel deep neural network module performs feature extraction on the input test image set

The RGB image of the test image is used as the input of the first deep neural network module, and the grayscale image corresponding to the test image is used as the input of the second deep neural network module;

The feature extraction process of the first deep neural network module is as follows: the full convolutional network module extracts local features from the RGB image of the test image through atrous convolution, maximum pooling, and convolution operations; the feature map output by the full convolutional network module is passed through The new feature map obtained by the first coordinate channel module is sent to the first circulation layer module for horizontal and vertical scanning, and the global feature information of the image is learned; the feature map output by the first circulation layer module is passed through the second coordinate channel module to obtain new features The image is then sent to the second circulation layer module for horizontal and vertical scanning to capture the global feature information of the image; the feature map output by the second circulation layer module is input into the spatial pyramid pooling module, and convolution operations are performed at multiple sampling rates to extract Feature information of regions of different scales;

The second deep neural network module feature extraction process is the same as the first deep neural network module feature extraction process;

S43. Perform weighted fusion of the feature map output by the first deep neural network module and the feature map output by the second deep neural network module to obtain a new feature map;

S44. Send the result of step S43 to the Softmax classification layer for pixel category label prediction, obtain the object category to which each pixel in the image belongs, and perform bilinear interpolation to upsample to the original image size to obtain a fine semantic segmentation map.

Using the method of this embodiment to semantically segment 128 test images in the WeizmannHorse dataset, the semantic segmentation diagram of some test images is shown in Figure 3, where the first row is the input image, and the second row is the color label image corresponding to the input image , the third line is its corresponding semantic segmentation map.

Example 2

The StanfordBackground dataset is an image segmentation dataset consisting of 715 images. Some images in the dataset are shown in Figure 4. The network model is trained using the Pytorch platform, and the code is written in python.

This embodiment is based on the image semantic segmentation method of deep learning. In step S1, 573 images are randomly selected from the StanfordBackground data set as a training image set, and the remaining 142 images are used as a test image set, and the data enhancement operation is performed on the training image set. The number of training images is increased to 13,752; in step S32, the size of the image in the training image set after data enhancement is cut to 421×421, and the training image is used to train the deep semantic segmentation network to generate a pixel category prediction label probability distribution map, The prediction loss is calculated using the predicted label probability and the original label probability, specifically using the mixed loss function L(θ) as the objective function,

L(θ)＝L ₁ (θ)+L ₂ (θ)

where L ₁ (θ) is the cross-entropy loss function, L ₂ (θ) is the L2 regularization term, and θ is the parameter of the deep semantic segmentation network;

The cross-entropy loss function L ₁ (θ) of this embodiment is:

是原始标签概率向量，N是每张图片的像素个数，N为421×421＝177241，B是批大小，B为6，C是像素类别数，C为8，ln(.)是求自然对数；where _ypq is the predicted label probability vector,

is the original label probability vector, N is the number of pixels of each picture, N is 421×421=177241, B is the batch size, B is 6, C is the number of pixel categories, C is 8, ln(.) is seeking nature logarithm;

The L2 regularization term L ₂ (θ) of this embodiment is:

In the formula, λ is a regularization coefficient and is a positive number, N is the number of pixels in each image, N is 421×421=177241, B is the batch size, B is 6, S is the number of parameters of w and S is positive Integer, w is a weight parameter; in step S33, adopt the stochastic gradient descent algorithm to optimize the objective function, use the backpropagation algorithm to update the network model parameters, and end the training until the value of the objective function no longer decreases. In order to accelerate the model convergence, the parameter learning method is introduced Learning rate, the learning rate decays according to the following formula:

In the formula, t is the number of iterations and t≤35000, l ₀ is the initial learning rate, l ₀ is 0.001, l _t is the learning rate of the t-th iteration, the gradient decay is 0.0001, and power is the momentum of 0.9;

Other operating steps and parameters are the same as in Example 1.

Using the method of this embodiment to semantically segment 142 test images in the StanfordBackground dataset, the semantic segmentation diagram of some test images is shown in Figure 4, where the first row is the input image, and the second row is the color label image corresponding to the input image , the third line is its corresponding semantic segmentation map.

Claims (5)

1. An image semantic segmentation method based on deep learning is characterized by comprising the following steps:

s1, processing data sets

Dividing an image data set into a training image set and a test image set, performing data enhancement operation on the training image set, and increasing the number of training images to ten thousand-level units;

s2, constructing a deep semantic segmentation network

The deep semantic segmentation network is composed of a parallel deep neural network module, a feature fusion module and a Softmax classification layer, wherein the parallel deep neural network module is used for carrying out feature extraction on an input image, the feature fusion module carries out weighting fusion on an output feature map of the parallel deep neural network to obtain a new feature map, and the Softmax classification layer converts a pixel class label prediction score into a pixel class label prediction probability distribution map;

the parallel deep neural network module consists of a first deep neural network module and a second deep neural network module, the network structures of the first deep neural network module and the second deep neural network module are the same, the input of the first deep neural network module is an RGB image of an input image, and the input of the second deep neural network module is a gray image of the input image;

the first deep neural network module consists of a full convolution network module, a first coordinate channel module, a first circulation layer module, a second coordinate channel module, a second circulation layer module and a space pyramid pooling module, wherein the first coordinate channel module and the second coordinate channel module have the same structure, the first circulation layer module and the second circulation layer module have the same structure, the full convolution network module is used for extracting local features of an input image, the first circulation layer module is used for capturing context dependency and global feature information of the image, the first coordinate channel module is used for connecting i, j and r coordinate channels to a feature map output by the full convolution network module to form a new feature map so as to learn more coordinate feature information and improve the generalization capability of the model, and the space pyramid pooling module is used for performing convolution operation on the feature map output by the second circulation layer module at a plurality of sampling rates to extract feature information of different scale areas;

s3, deep semantic segmentation network training and parameter learning

S31, initializing network model parameters: performing parameter initialization on a full convolution network module by using a pre-training model of ResNet101 on an ImageNet data set, performing parameter initialization on a first circulation layer module and a second circulation layer module by using standard uniform distribution, and performing parameter initialization on a convolution layer of a space pyramid pooling module by using standard Gaussian distribution;

s32, training a deep semantic segmentation network by using the training image set after data enhancement to generate a pixel class prediction label probability distribution graph, calculating prediction loss by using the prediction label probability and the original label probability, specifically adopting a mixed loss function L (theta) as a target function,

L(θ)＝L ₁ (θ)+L ₂ (θ)

in the formula L ₁ (θ) is a cross entropy loss function, L ₂ (theta) is an L2 regularization term, theta is a parameter of the deep semantic segmentation network;

the cross entropy loss function L ₁ (θ) is:

in the formula y _pq Is a predicted label probability vector that is,

the method comprises the following steps of (1) obtaining an original label probability vector, wherein N is the number of pixels of each picture, B is the batch size, C is the number of pixel categories, and ln (.) is the natural logarithm;

the L2 regularization term L ₂ (θ) is:

in the formula, lambda is a regularization coefficient and is a positive number, N is the number of pixels of each image, B is the batch size, S is the parameter number of w, S is a positive integer, and w is a weight parameter;

s33, optimizing the target function by adopting a random gradient descent algorithm, and updating network model parameters by adopting a back propagation algorithm until the value of the target function does not descend any more, so as to finish training;

s4, performing semantic segmentation on the test image

S41, inputting the test image set into the deep semantic segmentation network trained in the step S3;

s42, the parallel deep neural network module performs feature extraction on the input test image set

The RGB image of the test image is used as the input of the first deep neural network module, and the gray image of the test image is used as the input of the second deep neural network module;

the first deep neural network module feature extraction process comprises the following steps: the full convolution network module performs local feature extraction on the RGB image of the test image through hole convolution, maximum pooling and convolution operations; the feature map output by the full convolution network module is sent to a first circulation layer module through a first coordinate channel module to obtain a new feature map, the new feature map is sent to the first circulation layer module to be scanned horizontally and vertically, and the global feature information of the image is learned; obtaining a new feature map from the feature map output by the first circulating layer module through the second coordinate channel module, sending the new feature map into the second circulating layer module for horizontal and vertical scanning, and capturing global feature information of the image; inputting the feature map output by the second circulation layer module into a space pyramid pooling module, performing convolution operation on a plurality of sampling rates, and extracting feature information of different scale areas;

the second deep neural network module feature extraction process is the same as the first deep neural network module feature extraction process;

s43, carrying out weighted fusion on the feature map output by the first deep neural network module and the feature map output by the second deep neural network module to obtain a new feature map;

and S44, sending the result of the step S43 into a Softmax classification layer to perform pixel class label prediction to obtain the object class of each pixel in the image, performing bilinear interpolation operation to up-sample the size of the original image, and obtaining a fine semantic segmentation image.

2. The deep learning based image semantic segmentation method according to claim 1, characterized in that: the first circulation layer module is composed of two bidirectional threshold recursion units, and the number of neurons of each bidirectional threshold recursion unit is 150.

3. The deep learning based image semantic segmentation method according to claim 1, characterized in that: the spatial pyramid pooling module is formed by convolution of 4 holes with different sampling rates, the convolution kernel size of the hole convolution is 3 multiplied by 3, and the expansion rates are 4, 6, 8 and 12 respectively.

4. The deep learning based image semantic segmentation method according to claim 1, characterized in that: in the step S2, the i, j and r coordinate channels are composed of an i coordinate channel, a j coordinate channel and an r coordinate channel, the i coordinate channel, the j coordinate channel and the r coordinate channel are all coordinate matrixes of e multiplied by f, the elements of the 1 st row to the e th row of the i coordinate channel are sequentially 0, 1,. The elements of the 1 st column to the f th column of the i coordinate channel are sequentially 0, 1,. The.f, f-1, e and f take positive integers, and the r coordinate channel is

m is any element in the i coordinate channel, n is the element in the j coordinate channel with the same position as the m coordinate, and the elements in the i coordinate channel and the j coordinate channel are linearly scaled to [ -1,1]Within the range.

5. The deep learning based image semantic segmentation method according to claim 1, characterized in that: the learning rate of the parameter learning in the step S3 is attenuated according to the following formula:

where t is the number of iterations, l ₀ Is the initial learning rate,/ _t Is the learning rate for the t-th iteration, power is the momentum of 0.9.

Images