CN118154984A - Unsupervised neighborhood classification superpixel generation method and system based on fusion guided filtering - Google Patents

Abstract

The invention relates to the technical field of image processing, in particular to an unsupervised neighborhood classification superpixel generation method and system integrating guide filtering, comprising the following steps: downsampling an input image to obtain a low resolution image; extracting features from the low resolution image using a lightweight network model, the lightweight network model comprising a multi-scale pyramid attention mechanism and a depth separable U-Net module; performing pixel neighborhood classification based on the characteristics to obtain the association mapping of the pixels and nine neighborhood super-pixel pairs; and inputting the low-resolution image and the association mapping to a guide filtering module, and carrying out joint up-sampling by taking the original input image as a guide image to obtain a super-pixel segmentation result. The method for quickly and accurately generating the super pixels of the unsupervised image reduces the parameter number of the model through the super pixels quick generation framework and accelerates the reasoning speed of the model.

Description

Unsupervised neighborhood classification superpixel generation method and system based on fusion guided filtering

Technical Field

The present invention belongs to the technical field of image processing, and in particular relates to an unsupervised neighborhood classification superpixel generation method and system using fusion guided filtering.

Background technique

Superpixel segmentation is to cluster pixels based on their features and divide the image into several connected regions, each of which is a superpixel. Compared with pixels, superpixels can effectively represent images with fewer elements. Therefore, superpixel segmentation is often used as a preprocessing step to reduce the computational complexity of subsequent computer vision tasks such as image segmentation, object tracking, and saliency detection. Therefore, image superpixel segmentation methods require high accuracy and efficiency.

In 2018, Yang et al. first applied the fully-connected convolutional network (FCN) to image superpixel segmentation. First, the image was initialized into a rectangular grid according to the number of target superpixels. Then, an encoder-decoder structure was used to convert the superpixel generation problem into a 9-classification problem by assigning each pixel to 9 adjacent grids. FCN is the first true end-to-end superpixel generation model. Based on FCN, Wang et al. designed and introduced the association implantation (AI) module between pixels and superpixels, which enables the network to explicitly capture the relationship between pixels and their surrounding superpixels. At the same time, a boundary-aware loss was proposed to make the network pay more attention to boundary information. However, neither SSN nor FCN constrains the connectivity of superpixels, so separated superpixels may be generated. In order to ensure the connectivity of superpixels, they both require a post-processing step to merge trivial superpixels, and this step is often an independent, non-neural network structure. In order to save the post-processing steps, Yuan et al. proposed a superpixel interpolation network (SIN) based on the idea of interpolation. It generates superpixel results of the initial resolution according to the number of target superpixels, and then generates segmentation results with the same resolution as the input image by alternating interpolation in the horizontal and vertical directions, ensuring the connectivity of superpixels from beginning to end. The above methods are all supervised learning methods, which require a large amount of labeled data for training to achieve good performance.

Summary of the invention

In view of the above-mentioned deficiencies in the prior art, the present invention provides a method and system for generating unsupervised neighborhood classification superpixels by fusion-guided filtering to solve the above-mentioned technical problems.

In a first aspect, the present invention provides an unsupervised neighborhood classification superpixel generation method using fusion guided filtering, comprising:

Downsample the input image to obtain a low-resolution image;

Extracting features from the low-resolution image using a lightweight network model, wherein the lightweight network model includes a multi-scale pyramid attention mechanism and a deep separable U-Net module;

Perform pixel neighborhood classification based on the features to obtain association mapping between pixels and nine-neighborhood superpixel pairs;

The low-resolution image and the association map are input into a guided filtering module, and the original input image is used as a guided map for joint upsampling to obtain a superpixel segmentation result.

In an optional embodiment, the method further comprises:

Use the gradient rescaling module to sequentially train the lightweight network model;

Construct a target loss function, and use the target loss function to optimize the lightweight network model. The target loss function includes:

;

in, represents the clustering loss,/> represents the reconstruction loss,/> represents boundary smoothing loss, L _edge represents edge loss, /> 、/> and/> Represents the weight coefficient.

In an optional embodiment, extracting features from the low-resolution image using a lightweight network model includes:

The multi-scale pyramid attention mechanism uses convolutions with various dilation rates to extract features from the initial feature vector of the low-resolution image at multiple scales.

The ECA channel attention mechanism is used to weight features of multiple scales to capture feature information of different scales of low-resolution images;

The channel attention features of multiple scales are fused to obtain multi-scale attention features;

The deep separable U-Net module performs deep convolution and point-wise convolution on the multi-scale attention features to obtain color-preserving and spatial features;

The gradient rescaling module is used to reconstruct features and obtain the features of the low-resolution image.

In an optional implementation, pixel neighborhood classification is performed based on the features to obtain an association map between pixels and nine-neighborhood superpixel pairs, including:

Calculating the similarity between the pixel and the neighboring superpixel based on the feature, and converting the similarity into a probability of assignment;

The pixel is assigned to the neighboring superpixel with the highest probability.

In an optional embodiment, the method further comprises:

Clustering loss ;

in, Represents the average probability of a pixel and the Sth superpixel among the surrounding 9 superpixels; the first term represents the entropy of each pixel/> , which ensures the accuracy of superpixel generation; the second item is the negative entropy of the average vector of each pixel, which makes the formation of superpixels more uniform.

In an optional embodiment, the method further comprises:

Reconstruction loss ;

L _c represents the reconstruction error of color features, and the formula is:

;

Represents the reconstruction error of spatial features, and the formula is:

;

Indicates/> and/> The weight parameter between and/> Respectively represent the height and width of the image, /> Indicates the image feature channel number, /> Indicates the number of image color feature channels, /> Indicates the number of image spatial feature channels.

In an optional embodiment, the method further comprises:

Boundary smoothing loss ;

in, Indicates soft allocation in the segmentation result/> or the original input image/> and/> Directional gradient operation, /> and Indicates the height and width of the original input image, /> and/> Respectively represent/> and input image/> The first/> The soft assigned vectors and features of pixels.

In an optional embodiment, the method further comprises:

Loss of original resolution output ;

The edge map is obtained by using a 3×3 Laplacian kernel to calculate the edge of the image and then converting it to the range of [0, 1] through a Softmax operation. 、/> and/> ; Then, by calculating /> 、/> and/> The KL divergence between them is used to measure the difference between the two probability distributions. The lower the edge loss value, the more similar the two probability distributions are. By minimizing the edge loss, the edge distribution of the reconstructed image is closest to the edge distribution of the original image.

In an optional embodiment, the low-resolution image and the association map are input into a guided filtering module, and the original input image is used as a guided map for joint upsampling to obtain a superpixel segmentation result, including:

The linear model is used to model the relationship between the superpixel result and the input image, which can be expressed as , where /> Represents the superpixel segmentation result, /> represents the input image, /> and/> represents the coefficient;

The local window Low-resolution images within /> And segmentation results/> The mapping relationship is expressed as:

;

in, Represents the pixel index, /> and/> Respectively represent/> and/> The index in is /> pixels, /> Represents a rectangular window/> The index of and/> Displayed in the window /> The coefficients of the linear function in ;

By minimizing the difference between the result image and the original image, we can get:

;

;

in, Indicates/> The number of pixels in, /> Indicates that the window is Middle/> The average value of the pixels, /> and Respectively represent/> Middle/> The variance and mean of the pixels, /> represents the regularization parameter; /> Slide on the image in steps of 1, using /> The operation is performed with windows of different sizes. Each pixel is covered by 9 windows. The mapping coefficients corresponding to the low-resolution image are obtained by taking the average of the 9 groups of coefficients./> and/> ;

right and/> Upsampling generation/> and/> , and finally get high-resolution output/> , the formula is:

;

The superpixel fast generation network is constructed by transforming the original input image into As a guide image, it compensates for the loss of color and spatial edge information caused by image downsampling.

In a second aspect, the present invention provides an unsupervised neighborhood classification superpixel generation system fused with guided filtering, comprising:

A downsampling module is used to downsample the input image to obtain a low-resolution image;

A feature extraction module, configured to extract features from the low-resolution image using a lightweight network model, wherein the lightweight network model includes a multi-scale pyramid attention mechanism and a deep separable U-Net module;

A classification processing module, used for performing pixel neighborhood classification based on the features to obtain an association map between pixels and nine-neighborhood superpixel pairs;

The upsampling module is used to input the low-resolution image and the association map into the guided filtering module, and perform joint upsampling using the original input image as the guided map to obtain a superpixel segmentation result.

The beneficial effect of the present invention is that the unsupervised neighborhood classification superpixel generation method and system provided by the present invention provide an unsupervised image superpixel fast and accurate generation segmentation method, which can be used for the segmentation of digital images such as natural images, indoor images and medical images. A multi-scale attention mechanism, a guided image and an edge loss function are used to focus on the edge information of the image, thereby improving the edge fit and smoothness. At the same time, a superpixel fast generation framework is designed to reduce the number of model parameters and speed up the model reasoning speed.

The unsupervised neighborhood classification superpixel generation method and system of fusion-guided filtering provided by the present invention specifically include the following advantages:

1) The fast superpixel generation framework uses a fast guided filtering module and combines it with the original resolution image for joint upsampling to better compensate for the loss of image information caused by downsampling while retaining the edge detail information of the image.

2) The multi-scale pyramid attention mechanism is combined with the deep separable U-Net module to form a lightweight feature extraction network. In the multi-scale pyramid attention mechanism, the ECA channel attention mechanism is used to weight features of different scales, so that feature information of different scales in the image can be captured; then the channel attention features of different scales are fused to obtain multi-scale attention features to further enhance the feature expression ability of the network.

3) In the deep convolution stage, a filter is applied to each input channel separately, thereby reducing the number of parameters while retaining spatial information. The point-by-point convolution stage uses a 1x1 convolution kernel to map the output of the deep convolution to the required output channel to obtain the final feature representation. In this module, the inputs of different dimensions in the module are directly connected to the outputs of the same dimension through jump connections, so that more low-level feature information can be passed to the high-level feature layer, which can enable the model to learn feature representations at different levels, so that the features extracted by the model can better retain color and spatial detail information.

4) A robust loss function is used to fully consider the edge information of the image. By comparing the distribution differences between the edge of the original image and the reconstructed image and superpixels, the model is promoted to generate superpixel segmentation results that are closer to the original image, ensuring the accuracy of superpixel segmentation.

In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a schematic flow chart of a method according to an embodiment of the present invention.

FIG. 2 is another schematic flow chart of a method according to an embodiment of the present invention.

FIG3 is a schematic diagram of a method according to an embodiment of the present invention.

FIG. 4 is a schematic diagram of a multi-scale pyramid attention module of a method according to an embodiment of the present invention.

FIG. 5 is a schematic diagram of an ECA channel attention module of a method according to an embodiment of the present invention.

FIG. 6 is a diagram showing the segmentation effect of a method according to an embodiment of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

The unsupervised neighborhood classification superpixel generation method of fusion guided filtering provided in an embodiment of the present invention is executed by a computer device, and correspondingly, the unsupervised neighborhood classification superpixel generation system of fusion guided filtering runs in the computer device.

FIG1 is a schematic flow chart of a method according to an embodiment of the present invention. The execution subject of FIG1 may be an unsupervised neighborhood classification superpixel generation system with fusion guided filtering. According to different requirements, the order of the steps in the flow chart may be changed, and some may be omitted.

As shown in FIG1 , the method includes:

Step 110, down-sampling the input image to obtain a low-resolution image;

Step 120, extracting features from the low-resolution image using a lightweight network model, wherein the lightweight network model includes a multi-scale pyramid attention mechanism and a depth-separable U-Net module;

Step 130, performing pixel neighborhood classification based on the features to obtain an association map between pixels and nine-neighborhood superpixel pairs;

Step 140: Input the low-resolution image and the association map into a guided filtering module, and perform joint upsampling using the original input image as a guided map to obtain a superpixel segmentation result.

To facilitate understanding of the present invention, the following is a further description of the unsupervised neighborhood classification superpixel generation method using fusion guided filtering provided by the present invention based on the principle of the unsupervised neighborhood classification superpixel generation method using fusion guided filtering, in combination with the embodiments and drawings.

Specifically, please refer to FIG. 2 and FIG. 3 , the unsupervised neighborhood classification superpixel generation method of fusion guided filtering includes:

S1. Convert the input image to a low-resolution image.

The input image is a Natural images of , first of all, the image /> Downsample to get a low-resolution input image/> , and add its spatial coordinate information to the feature vector of the image, defined as an extensible mapping , the low resolution input image/> Embedded into 5-dimensional feature vector/> And normalize it to get the initial eigenvector.

S2. Feature extraction.

The multi-scale pyramid attention mechanism module (as shown in Figure 4) combines the dilated spatial convolutional pooling pyramid and the ECA channel attention mechanism to extract and weight features more carefully at different scales, thereby obtaining a more representative feature representation. Specifically, first, convolutions with different dilation rates r = 0, 1, and 2 are used to filter the image input features. Feature extraction is performed at different scales; then, the ECA channel attention mechanism is used to weight features of different scales, so that feature information of different scales in the image can be captured. Finally, the channel attention features of different scales are fused to obtain multi-scale attention features to further enhance the feature expression ability of the network.

Among them, * is the convolution operation, is a dilated convolution with dilation rates of 0, 1, and 2,/> It is a nonlinear function implemented by ReLU.

Compared with the traditional two-dimensional convolution operation, the ECA channel attention module (as shown in Figure 5) uses a one-dimensional convolution operation to extract the relationship between channels, which not only reduces the number of parameters but also reduces the computational complexity. Therefore, while improving the performance of the model, the ECA module does not significantly increase the number of model parameters and the consumption of computing resources. Through the one-dimensional convolution operation, the ECA module can effectively extract the correlation and dependency between channels in the feature map. When calculating the channel attention weight, a local convolution operation is used, that is, only the relationship between channels in the local area of the feature map is considered. This localized operation helps to reduce the confusion of local information, allowing the module to pay more attention to important features in the image. At the same time, the ECA module can globally affect the entire feature map, ensuring the module's perception of the global information of the image.

The output feature vectors of the dilated convolution are Perform a global average pooling operation, and the result is recorded as/> :

in, and/> are the height and width of the input feature map respectively. Then, /> Through the 1D convolution layer, the result is recorded as/> :

in, is a 1D convolution kernel, /> is the size of the convolution kernel. By/> Activation function, get the weight result/> :

Finally, the input feature map and/> Multiply element by element to get the output feature vector of the ECA module:

.

The depth-separable U-Net module (DSUM) used is an improvement and optimization based on U-Net. As a component of the lightweight U-Net module, it adopts a more efficient convolution method to improve the inference speed and computational efficiency of the model. The depth-separable convolution decomposes the standard convolution operation into two steps: depth convolution and point-by-point convolution. In the depth convolution stage, a filter is applied to each input channel separately, thereby reducing the number of parameters while retaining spatial information. The point-by-point convolution stage uses a 1x1 convolution kernel to map the output of the depth convolution to the required output channel to obtain the final feature representation. In this module, the inputs of different dimensions in the module are directly connected to the outputs of the same dimension through jump connections, so that more low-level feature information can be passed to the high-level feature layer, which can enable the model to learn feature representations at different levels, so that the features extracted by the model can better retain color and spatial detail information.

The gradient rescaling module is used to reconstruct features and obtain the features of the low-resolution image.

S3. Pixel classification obtains pixel-nine-neighborhood superpixel association mapping.

The similarity between the pixel and the neighboring superpixel is calculated based on the feature, and the similarity is converted into a probability of assignment; and the pixel is assigned to the neighboring superpixel with the maximum probability.

The goal of superpixel segmentation is to assign pixels to sets of superpixels. In a superpixel in , each superpixel represents a set of pixels, and the parameter /> is an upper bound on the number of superpixels. However, computing the association map for all pixel-superpixel pairs is unnecessary and computationally expensive. Instead, for a given pixel p, the bound is set to the set of 9 grid cells in the neighborhood For each pixel p, only the nine neighboring grids around it are considered for assignment. Therefore, the association map of pixel-superpixel pairs can be represented as a tensor/> , where/> .

Consider superpixel segmentation as a nine-neighborhood classification task, [1] As a probabilistic representation of superpixels. The pixel at / > Assigned to the neighborhood superpixel, obtained by optimizing the defined objective loss function/> .

S4, fast guided filtering module for joint upsampling.

The difficulty of the superpixel fast generation network lies in how to obtain the high-resolution image segmentation result through the low-resolution image segmentation result. The key is to explore the relationship between the input image and its segmentation result. The goal of the image superpixel generation task is to accurately detect and maintain the image edge. Therefore, a linear model can be used to model the relationship between the superpixel result and the input image, which can be expressed as , where /> Represents the superpixel segmentation result, /> represents the input image, /> and/> In the same image, inputs of different resolutions should have the same linear relationship with superpixel segmentation results. Based on this, it is known that the low-resolution input And superpixel results/> By learning the linear relationship between the two and applying it to the original input resolution image, the image processing result of the original resolution can be generated. Local window/> Low-resolution images within /> And segmentation results/> The mapping relationship can be expressed as:

;

in, Represents the pixel index, /> and/> Respectively represent/> and/> The index in is /> pixels, /> Represents a rectangular window/> The index of and/> Displayed in the window /> By minimizing the difference between the result image and the original image, we can get:

;

;

in, Indicates/> The number of pixels in, /> Indicates that the window is Middle/> The average value of the pixels, /> and Respectively represent/> Middle/> The variance and mean of the pixels, /> represents the regularization parameter. /> Slide on the image with a step size of 1, each pixel will be covered by multiple windows, and multiple sets of coefficients will be obtained accordingly. This chapter uses /> Each pixel will be covered by 9 windows. The mapping coefficients corresponding to the low-resolution image are obtained by taking the average of the 9 groups of coefficients. and/> .right and/> Upsampling generation/> and/> , and finally get high-resolution output/> , the formula is:

;

The superpixel fast generation network is constructed by transforming the original input image into As a guide image, it compensates for the loss of color and spatial edge information caused by image downsampling, ensuring the accuracy of superpixels generated by upsampling.

S5. Calculate the target loss function and update the model parameters by gradient descent.

The objective loss function is as follows:

;

in, represents the clustering loss,/> represents the reconstruction loss,/> represents the boundary smoothing loss, and L _edge represents the loss of the original resolution output. /> 、/> and/> Represents the weight coefficient, which is set to 1, 1, and 2.5 respectively in the experiment.

(1) Clustering loss

To ensure the accuracy and uniformity of superpixel generation, we use the clustering loss proposed in [34] to consider the assignment probability of each pixel and the uniformity of superpixels:

.

in, Represents the average probability of the pixel and the Sth superpixel in the surrounding 9 superpixels. The first term represents the entropy of each pixel/> , which ensures the accuracy of superpixel generation. The second term is the negative entropy of the average vector of each pixel, which makes the formation of superpixels more uniform. By weighing accuracy and uniformity, a more spatially consistent and uniform segmentation result is obtained.

(2) Reconstruction loss

In order to ensure the validity and reliability of the learned image features, FRL is used in GRM to reconstruct the learned clustering features to obtain the reconstructed image features , the reconstruction loss is calculated using mean squared error (MSE), and the formula is:

.

Among them, represents the reconstruction error of color features, and the formula is:

.

Represents the reconstruction error of spatial features, and the formula is:

;

Indicates/> and/> The weight parameter between and/> Respectively represent the height and width of the image, /> Indicates the image feature channel number, /> Indicates the number of image color feature channels, /> Indicates the number of image spatial feature channels, /> ,/> .

(3) Boundary smoothing loss

In order to make the image superpixel consistent with the input image at the boundary, the Gaussian kernel function is used to calculate the input image Soft allocation of pixels with prediction/> The relationship between

;

in, Indicates soft allocation in the segmentation result/> or the original input image/> and/> Directional gradient operation, /> and Indicates the height and width of the original input image, /> and/> Respectively represent/> and input image/> The first/> The soft allocation vector and features of pixels are used in the experiment. Set to 8.

(4) Edge loss

In order to further improve the boundary fit and segmentation accuracy and guide the model to better learn edge information, inspired by the literature [37], this loss function is used to focus on the input image , pixel-superpixel association map/> and reconstruct the image/> The edge information in , and use KL divergence to measure the similarity of the edge distribution between them, the formula is

.

The edge map is obtained by using a 3×3 Laplacian kernel to calculate the edge of the image and then converting it to the range of [0, 1] through a Softmax operation. 、/> and/> . Then, by calculating/> 、/> and/> The KL divergence between is used to measure the difference between the two probability distributions. The lower the edge loss value, the more similar the two probability distributions are. Therefore, by minimizing the edge loss, we can make the edge distribution of the reconstructed image as close as possible to the edge distribution of the original image, thereby retaining important edge information.

The superpixel fast generation method for unsupervised image pixel neighborhood classification has achieved good performance in superpixel segmentation. It has been verified using the same experimental data as other unsupervised superpixel segmentation methods. The experiments were conducted on the test set in the BSDS500 dataset, which includes a pixel resolution of 481 321 and 321/> 200 images of 481. The estimates are measured using three common metrics in superpixel image segmentation:

(1) Achievable Segmentation Accuracy (ASA)

Used to evaluate the overall accuracy of superpixel segmentation, it takes into account the spatial adjacency between superpixels and is a connectivity-based metric. The calculation formula of ASA is:

,

Where N is the number of superpixels in the image, It is the algorithm segmentation of the superpixels, /> is the corresponding true label /> The value of ASA ranges from 0 to 1. The closer the value is to 1, the closer the segmentation result is to the true label.

(2) Boundary Recall BR

It is used to evaluate the boundary detection ability of superpixel segmentation. It measures the ratio between the number of boundaries extracted by the algorithm and the number of true boundaries. The calculation formula of BR is:

,

in, represents the boundary set of the true label,/> Represents the set of superpixel result boundaries generated by the superpixel algorithm. The value range of BR is between 0 and 1. The closer the value is to 1, the closer the boundary extracted by the algorithm is to the real boundary.

(3) UnderSegmentation Error USE

It is used to evaluate the under-segmentation in superpixel segmentation, that is, the situation where the same object is segmented into multiple superpixels. The calculation formula of USE is:

,

in, It is the algorithm segmentation of the superpixels, /> is the corresponding true label /> The value range of USE is between 0 and 1. The closer the value is to 0, the higher the overlap between the segmentation result and the true label.

In some embodiments, the unsupervised neighborhood classification superpixel generation system of fusion guided filtering may include multiple functional modules composed of computer program segments. The computer program of each program segment in the unsupervised neighborhood classification superpixel generation system of fusion guided filtering may be stored in the memory of a computer device and executed by at least one processor to perform (see FIG. 1 for details) the function of unsupervised neighborhood classification superpixel generation of fusion guided filtering.

In this embodiment, the unsupervised neighborhood classification superpixel generation system of fusion guided filtering can be divided into multiple functional modules according to the functions performed. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

A downsampling module is used to downsample the input image to obtain a low-resolution image;

A feature extraction module, configured to extract features from the low-resolution image using a lightweight network model, wherein the lightweight network model includes a multi-scale pyramid attention mechanism and a deep separable U-Net module;

A classification processing module, used for performing pixel neighborhood classification based on the features to obtain an association map between pixels and nine-neighborhood superpixel pairs;

The upsampling module is used to input the low-resolution image and the association map into the guided filtering module, and perform joint upsampling using the original input image as the guided map to obtain a superpixel segmentation result.

Optionally, as an embodiment of the present invention, it further includes:

Use the gradient rescaling module to sequentially train the lightweight network model;

Construct a target loss function, and use the target loss function to optimize the lightweight network model. The target loss function includes:

;

in, represents the clustering loss,/> represents the reconstruction loss,/> represents boundary smoothing loss, L _edge represents edge loss, /> 、/> and/> Represents the weight coefficient.

Optionally, as an embodiment of the present invention, extracting features from the low-resolution image using a lightweight network model includes:

The multi-scale pyramid attention mechanism uses convolutions with various dilation rates to extract features from the initial feature vector of the low-resolution image at multiple scales.

The ECA channel attention mechanism is used to weight features of multiple scales to capture feature information of different scales of low-resolution images;

The channel attention features of multiple scales are fused to obtain multi-scale attention features;

The deep separable U-Net module performs deep convolution and point-wise convolution on the multi-scale attention features to obtain color-preserving and spatial features;

The gradient rescaling module is used to reconstruct features and obtain the features of the low-resolution image.

Optionally, as an embodiment of the present invention, pixel neighborhood classification is performed based on the features to obtain an association mapping between pixels and nine-neighborhood superpixel pairs, including:

Calculating the similarity between the pixel and the neighboring superpixel based on the feature, and converting the similarity into a probability of assignment;

The pixel is assigned to the neighboring superpixel with the highest probability.

Optionally, as an embodiment of the present invention, it further includes:

Clustering loss ;

in, Represents the average probability of a pixel and the Sth superpixel among the surrounding 9 superpixels; the first term represents the entropy of each pixel/> , which ensures the accuracy of superpixel generation; the second item is the negative entropy of the average vector of each pixel, which makes the formation of superpixels more uniform.

Optionally, as an embodiment of the present invention, it further includes:

Reconstruction loss ;

L _c represents the reconstruction error of color features, and the formula is:

;

Represents the reconstruction error of spatial features, and the formula is:

;

Indicates/> and/> The weight parameter between and/> Respectively represent the height and width of the image, /> Indicates the image feature channel number, /> Indicates the number of image color feature channels, /> Indicates the number of image spatial feature channels.

Optionally, as an embodiment of the present invention, it further includes:

Boundary smoothing loss ;

in, Indicates soft allocation in the segmentation result/> or the original input image/> and/> Directional gradient operation, /> and Indicates the height and width of the original input image, /> and/> Respectively represent/> and input image/> The first/> The soft assigned vectors and features of pixels.

Optionally, as an embodiment of the present invention, it further includes:

Loss of original resolution output ;

The edge map is obtained by using a 3×3 Laplacian kernel to calculate the edge of the image and then converting it to the range of [0, 1] through a Softmax operation. 、/> and/> ; Then, by calculating /> 、/> and/> The KL divergence between them is used to measure the difference between the two probability distributions. The lower the edge loss value, the more similar the two probability distributions are. By minimizing the edge loss, the edge distribution of the reconstructed image is closest to the edge distribution of the original image.

Optionally, as an embodiment of the present invention, the low-resolution image and the association map are input into a guided filtering module, and the original input image is used as a guided map for joint upsampling to obtain a superpixel segmentation result, including:

The linear model is used to model the relationship between the superpixel result and the input image, which can be expressed as , where /> Represents the superpixel segmentation result, /> represents the input image, /> and/> represents the coefficient;

The local window Low-resolution images within /> And segmentation results/> The mapping relationship is expressed as:

;

in, Represents the pixel index, /> and/> Respectively represent/> and/> The index in is /> pixels, /> Represents a rectangular window/> The index of and/> Displayed in the window /> The coefficients of the linear function in ;

By minimizing the difference between the result image and the original image, we can get:

;

;

in, Indicates/> The number of pixels in, /> Indicates that the window is Middle/> The average value of the pixels, /> and Respectively represent/> Middle/> The variance and mean of the pixels, /> represents the regularization parameter; /> Slide on the image in steps of 1, using /> The operation is performed with windows of different sizes. Each pixel is covered by 9 windows. The mapping coefficients corresponding to the low-resolution image are obtained by taking the average of the 9 groups of coefficients./> and/> ;

right and/> Upsampling generation/> and/> , and finally get high-resolution output/> , the formula is:

;

The superpixel fast generation network is constructed by transforming the original input image into As a guide image, it compensates for the loss of color and spatial edge information caused by image downsampling.

Although the present invention has been described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all of these shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims (10)

1. A method for generating superpixels by unsupervised neighborhood classification by fusion guided filtering, characterized by comprising: Downsample the input image to obtain a low-resolution image; Extracting features from the low-resolution image using a lightweight network model, wherein the lightweight network model includes a multi-scale pyramid attention mechanism and a deep separable U-Net module; Perform pixel neighborhood classification based on the features to obtain association mapping between pixels and nine-neighborhood superpixel pairs; The low-resolution image and the association map are input into a guided filtering module, and the original input image is used as a guided map for joint upsampling to obtain a superpixel segmentation result. 2. The method according to claim 1, characterized in that the method further comprises: Use the gradient rescaling module to sequentially train the lightweight network model; Construct a target loss function, and use the target loss function to optimize the lightweight network model. The target loss function includes: ; in, represents the clustering loss,/> represents the reconstruction loss,/> represents boundary smoothing loss, L _edge represents edge loss, /> 、/> and/> Represents the weight coefficient. 3. The method according to claim 1, characterized in that extracting features from the low-resolution image using a lightweight network model comprises: The multi-scale pyramid attention mechanism uses convolutions with various dilation rates to extract features from the initial feature vector of the low-resolution image at multiple scales. The ECA channel attention mechanism is used to weight features of multiple scales to capture feature information of different scales of low-resolution images; The channel attention features of multiple scales are fused to obtain multi-scale attention features; The deep separable U-Net module performs deep convolution and point-wise convolution on the multi-scale attention features to obtain color-preserving and spatial features; The gradient rescaling module is used to reconstruct features and obtain the features of the low-resolution image. 4. The method according to claim 1, characterized in that pixel neighborhood classification is performed based on the features to obtain an association map between pixels and nine-neighborhood superpixel pairs, comprising: Calculating the similarity between the pixel and the neighboring superpixel based on the feature, and converting the similarity into a probability of assignment; The pixel is assigned to the neighboring superpixel with the highest probability. 5. The method according to claim 2, characterized in that the method further comprises: Clustering loss ; in, Represents the average probability of a pixel and the Sth superpixel among the surrounding 9 superpixels; the first term represents the entropy of each pixel/> , which ensures the accuracy of superpixel generation; the second item is the negative entropy of the average vector of each pixel, which makes the formation of superpixels more uniform. 6. The method according to claim 2, characterized in that the method further comprises: Reconstruction loss ; L _c represents the reconstruction error of color features, and the formula is: ; Represents the reconstruction error of spatial features, and the formula is: ; Indicates/> and/> The weight parameter between and/> Respectively represent the height and width of the image, /> Indicates the image feature channel number, /> Indicates the number of image color feature channels, /> Indicates the number of image spatial feature channels. 7. The method according to claim 2, characterized in that the method further comprises: Boundary smoothing loss ; in, Indicates soft allocation in the segmentation result/> or the original input image/> and/> Directional gradient operation, /> and/> Indicates the height and width of the original input image, /> and/> Respectively represent/> and input image/> The first/> The soft assigned vectors and features of pixels. 8. The method according to claim 2, characterized in that the method further comprises: Loss of original resolution output ; The edge map is obtained by using a 3×3 Laplacian kernel to calculate the edge of the image and then converting it to the range of [0, 1] through a Softmax operation. 、/> and/> ; Then, by calculating /> 、/> and/> The KL divergence between them is used to measure the difference between the two probability distributions. The lower the edge loss value, the more similar the two probability distributions are. By minimizing the edge loss, the edge distribution of the reconstructed image is closest to the edge distribution of the original image. 9. The method according to claim 1, characterized in that the low-resolution image and the association map are input into a guided filtering module, and the original input image is used as a guided map for joint upsampling to obtain a superpixel segmentation result, comprising: The relationship between the superpixel result and the input image is modeled using a linear model, which can be expressed as ,in, Represents the superpixel segmentation result, /> represents the input image, /> and/> represents the coefficient; The local window Low-resolution images within /> And segmentation results/> The mapping relationship is expressed as: ; in, Represents the pixel index, /> and/> Respectively represent/> and/> The index in is /> pixels, /> Represents a rectangular window/> The index of and/> Displayed in the window /> The coefficients of the linear function in ; By minimizing the difference between the result image and the original image, we can get: ; ; in, Indicates/> The number of pixels in, /> Indicates that the window is Middle/> The average value of the pixels, /> and/> Respectively represent/> Middle/> The variance and mean of the pixels, /> represents the regularization parameter; /> Slide on the image with a step size of 1, using The operation is performed with windows of different sizes. Each pixel is covered by 9 windows. The mapping coefficients corresponding to the low-resolution image are obtained by taking the average of the 9 groups of coefficients./> and/> ; right and/> Upsampling generation/> and/> , and finally get high-resolution output/> , the formula is: ; The superpixel fast generation network is constructed by transforming the original input image into As a guide image, it compensates for the loss of color and spatial edge information caused by image downsampling. 10. An unsupervised neighborhood classification superpixel generation system fused with guided filtering, characterized by comprising: A downsampling module is used to downsample the input image to obtain a low-resolution image; A feature extraction module, configured to extract features from the low-resolution image using a lightweight network model, wherein the lightweight network model includes a multi-scale pyramid attention mechanism and a deep separable U-Net module; A classification processing module, used for performing pixel neighborhood classification based on the features to obtain an association map between pixels and nine-neighborhood superpixel pairs; The upsampling module is used to input the low-resolution image and the association map into the guided filtering module, and perform joint upsampling using the original input image as the guided map to obtain a superpixel segmentation result.