CN114820666B

CN114820666B - Method and device for increasing matting accuracy, computer equipment and storage medium

Info

Publication number: CN114820666B
Application number: CN202210472640.7A
Authority: CN
Inventors: 杨松
Original assignee: Shenzhen Wondershare Software Co Ltd
Current assignee: Shenzhen Wondershare Software Co Ltd
Priority date: 2022-04-29
Filing date: 2022-04-29
Publication date: 2024-07-23
Anticipated expiration: 2042-04-29
Also published as: CN114820666A

Abstract

The invention discloses a method, a device, computer equipment and a storage medium for increasing matting accuracy. The method comprises the steps of obtaining an RGB image and an initial Trimap image of an image to be scratched; coding the RGB image and the initial Trimap image to obtain a coded image; performing foreground edge prediction processing on the coded image to obtain a foreground edge predicted image; performing background elimination processing on the coded image to obtain an accurate Trimap prediction graph; performing target foreground prediction processing on the coded image to obtain a foreground predicted image; and decoding the coded image, and separating the target by utilizing the decoded coded image, the foreground edge predicted image and the accurate Trimap predicted image to obtain a target image. The invention effectively suppresses background information by improving the network structure, and has the advantage of being capable of obtaining relatively high matting accuracy.

Description

Method and device for increasing matting accuracy, computer equipment and storage medium

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a computer device, and a storage medium for increasing matting accuracy.

Background

Natural image matting is a fundamental study in the field of image processing. The method can realize high-precision foreground extraction under natural images, avoids the constraint of arranging solid color backgrounds by traditional color key matting, and has higher application value in the tasks of image and video synthesis, augmented reality, target tracking, target classification and the like.

In the prior art, an end-to-end depth neural network is adopted, and the network mainly adopts a transparency mask extraction technology to extract a foreground object for matting processing, but the matting is extremely complex due to factors such as the shape of the foreground object is unstable, the background texture is complex and changeable, the transparency mask is accurately estimated, and the fuzzy limit between the foreground and the background.

Disclosure of Invention

The invention aims to provide a method, a device, computer equipment and a storage medium for increasing the matting accuracy, which aim to solve the problem that the conventional matting accuracy is still to be improved.

In order to solve the technical problems, the aim of the invention is realized by the following technical scheme: a method of increasing matting accuracy is provided, comprising:

acquiring an RGB image and an initial Trimap image of an image to be scratched;

Coding the RGB image and the initial Trimap image to obtain a coded image;

Performing foreground edge prediction processing on the coded image to obtain a foreground edge predicted image;

performing background elimination processing on the coded image to obtain an accurate Trimap prediction graph;

Performing target foreground prediction processing on the coded image to obtain a foreground predicted image;

And decoding the coded image, and separating a target by using the decoded coded image, the foreground edge predicted image and the accurate Trimap predicted image to obtain a target image.

In addition, the technical problem to be solved by the invention is to provide a device for increasing the matting accuracy, which comprises:

The acquisition unit is used for acquiring an RGB image and an initial Trimap image of the image to be scratched;

The coding unit is used for coding the RGB image and the initial Trimap image to obtain a coded image;

The foreground edge prediction unit is used for carrying out foreground edge prediction processing on the coded image to obtain a foreground edge predicted image;

the background eliminating unit is used for carrying out background eliminating processing on the coded image to obtain an accurate Trimap predictive graph;

the target foreground prediction unit is used for carrying out target foreground prediction processing on the coded image to obtain a foreground predicted image;

And the target separation unit is used for decoding the coded image, and separating a target by utilizing the decoded coded image, the foreground edge predicted image and the accurate Trimap predicted image to obtain a target image.

In addition, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the method for increasing the matting accuracy according to the first aspect when executing the computer program.

In addition, an embodiment of the present invention further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, where the computer program when executed by a processor causes the processor to perform the method for increasing matting accuracy described in the first aspect.

The embodiment of the invention discloses a method, a device, computer equipment and a storage medium for increasing matting accuracy. The method comprises the steps of obtaining an RGB image and an initial Trimap image of an image to be scratched; coding the RGB image and the initial Trimap image to obtain a coded image; performing foreground edge prediction processing on the coded image to obtain a foreground edge predicted image; performing background elimination processing on the coded image to obtain an accurate Trimap prediction graph; performing target foreground prediction processing on the coded image to obtain a foreground predicted image; and decoding the coded image, and separating the target by utilizing the decoded coded image, the foreground edge predicted image and the accurate Trimap predicted image to obtain a target image. The embodiment of the invention effectively suppresses background information by improving the network structure, and has the advantage of being capable of obtaining relatively high matting accuracy.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of a method for increasing matting accuracy according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of step S102 according to an embodiment of the present invention;

fig. 3 is a schematic flow chart of step S103 according to an embodiment of the present invention;

Fig. 4 is a schematic flow chart of step S104 according to an embodiment of the present invention;

Fig. 5 is a schematic view of a sub-flowchart of step S105 according to an embodiment of the present invention;

Fig. 6 is a schematic view of a sub-flow of step S106 according to an embodiment of the present invention;

FIG. 7 is a block diagram of a network model for increasing matting accuracy provided by an embodiment of the present invention;

fig. 8 is a schematic block diagram of an apparatus for increasing matting accuracy provided by an embodiment of the invention;

Fig. 9 is a schematic block diagram of a computer device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

Referring to fig. 1, fig. 1 is a flowchart of a method for increasing matting accuracy according to an embodiment of the present invention;

As shown in fig. 1, the method includes steps S101 to S106.

S101, acquiring an RGB image and an initial Trimap image of an image to be scratched;

s102, carrying out coding processing on the RGB image and the initial Trimap image to obtain a coded image;

S103, carrying out foreground edge prediction processing on the coded image to obtain a foreground edge predicted image;

S104, performing background elimination processing on the coded image to obtain an accurate Trimap prediction graph;

S105, carrying out target foreground prediction processing on the coded image to obtain a foreground predicted image;

S106, decoding the coded image, and separating the target by using the decoded coded image, the foreground edge predicted image and the accurate Trimap predicted image to obtain a target image.

In this embodiment, as shown in fig. 7, the input of the network is an RGB image of the to-be-scratched image and a corresponding initial Trimap image, color feature extraction may be performed by Matlab software or by using a color histogram method to obtain a corresponding RGB image, rough segmentation may be performed on the to-be-scratched image by using a semantic segmentation network, and a foreground area, a background area and an unknown area may be confirmed to obtain a corresponding initial Trimap image, and then feature encoding processing is performed on the RGB image and the initial Trimap image to obtain an encoded image.

Then, carrying out foreground edge prediction processing on the coded image so as to decode a foreground edge position;

then, carrying out background elimination processing on the coded image so as to decode inhibition background information;

Then, carrying out target foreground prediction processing on the coded image so as to decode a target foreground position;

Based on the above, attention mechanisms can be inserted in the decoding process, so that the deep learning training effect is improved; finally, separating the target by using the decoded coded image and the foreground edge predicted image and the accurate Trimap predicted image to obtain a target image; the invention effectively suppresses background information by improving the network structure, and has the advantage of being capable of obtaining relatively high matting accuracy.

In one embodiment, as shown in fig. 2, step S102 includes:

s201, inputting an RGB image and an initial Trimap image into an encoder network;

s202, performing color feature coding on the RGB image through an encoder network;

S203, encoding a foreground region, a background region and an unknown region in the initial Trimap through an encoder network;

S204, performing feature fusion on the coded RGB image and the initial Trimap image, and outputting a coded image.

In this embodiment, color coding features are obtained by coding an RGB map, an initial Trimap map is coded to obtain pixel features of a foreground region, a background region and an unknown region, and then the color coding features and the pixel features are fused, so that information retention in feature fusion is facilitated, the result obtained by the fusion features is superior to the matting effect of using only the initial Trimap map, and robustness of precision estimation of a mask with variable shape of a foreground object, complex and variable background texture and transparency is very high.

In one embodiment, as shown in fig. 3, step S103 includes:

S301, inputting the coded image into a first decoding network to perform target edge decoding to obtain edge pixel information of a foreground;

s302, carrying out foreground edge extraction on the coded image according to the edge pixel information, and outputting a foreground edge predicted image.

In this embodiment, the first decoding network may be an edge decoding network, which is used to identify edge pixel information of a foreground object in the encoded image; in general, the boundary between the foreground region and the background region may be used as the edge position of the foreground, but the pixels in the unknown region affect the accuracy of the edge position of the foreground, so that the pixels in the unknown region need to be classified, and in this embodiment, the transparency mask of the foreground region is obtained by measuring the transparency estimated values of the adjacent pixels in the foreground region, then the transparency estimated value between each pixel in the unknown region and the adjacent pixels in the foreground region is measured, screening is performed by a preset transparency threshold, and the pixels corresponding to the transparency threshold are classified into the foreground region to update the transparency mask of the foreground region, and then the foreground edge in the encoded image is extracted according to the updated transparency mask, so as to output the foreground edge predicted image.

In one embodiment, as shown in fig. 4, step S104 includes:

s401, inputting the coded image into a second decoding network to perform background pixel decoding to obtain background pixel information;

s402, confirming a background area according to background pixel information;

s403, eliminating the background area and outputting the accurate Trimap prediction graph.

In this embodiment, the region pixel score in the encoded image is obtained by decoding the encoded image, the region pixel score is screened according to a preset background pixel threshold, a position corresponding to the screened region pixel score is confirmed as a background pixel position, and then the confirmed background pixel position is removed and an accurate Trimap predictive map is output.

In one embodiment, as shown in fig. 5, step S105 includes:

S501, inputting the coded image into a fourth decoder network for decoding, and extracting the integral relation between a foreground region and an unknown region;

s502, performing pixel optimization processing on the unknown region according to the overall relation between the foreground region and the unknown region, and outputting a foreground predicted image.

In this embodiment, the fourth decoder network is a target foreground prediction network, and is configured to predict a target object in the encoded image, extract an overall relationship between the foreground region and the unknown region by decoding the encoded image, further improve the accuracy of the foreground region by performing optimization training on boundary pixels between the foreground region and the unknown region, and output a foreground predicted image.

In one embodiment, as shown in fig. 6, step S106 includes:

s601, inputting the coded image into a third decoding network for decoding;

s602, performing full connection processing on the decoded coded image, the foreground edge predicted image, the accurate Trimap predicted image and the foreground predicted image, and then performing convolution processing to obtain a target image.

In this embodiment, the images respectively output by the first decoding network, the second decoding network, the third decoding network and the fourth decoding network are subjected to feature full-connection processing to obtain a fused image, and the fused image is subjected to convolution operation by a plurality of convolution layers, so that a high-precision target image is output.

The embodiment of the invention also provides a device for increasing the matting accuracy, which is used for executing any embodiment of the method for increasing the matting accuracy. In particular, referring to fig. 8, fig. 8 is a schematic block diagram of an apparatus for increasing matting accuracy according to an embodiment of the present invention.

As shown in fig. 8, an apparatus 800 for increasing matting accuracy includes: an acquisition unit 801, an encoding unit 802, a foreground edge prediction unit 803, a background rejection unit 804, a target foreground prediction unit 805, and a target separation unit 806.

An acquiring unit 801, configured to acquire an RGB image and an initial Trimap image of an image to be scratched;

an encoding unit 802, configured to encode the RGB map and the initial Trimap map to obtain an encoded image;

A foreground edge prediction unit 803, configured to perform foreground edge prediction processing on the encoded image, to obtain a foreground edge predicted image;

The background rejection unit 804 is configured to perform background rejection processing on the encoded image to obtain an accurate Trimap predicted map;

the target foreground prediction unit 805 is configured to perform target foreground prediction processing on the encoded image to obtain a foreground predicted image;

The target separation unit 806 is configured to decode the encoded image, and separate the target by using the decoded encoded image, the foreground edge prediction image, and the accurate Trimap prediction image, to obtain a target image.

The device effectively inhibits background information through improving the network structure, and has the advantage of being capable of obtaining relatively high matting accuracy.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus and units described above may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again.

The means for increasing matting accuracy described above may be implemented in the form of a computer program which is executable on a computer device as shown in fig. 9.

Referring to fig. 9, fig. 9 is a schematic block diagram of a computer device according to an embodiment of the present invention. The computer device 900 is a server, and the server may be a stand-alone server or a server cluster formed by a plurality of servers.

With reference to fig. 9, the computer device 900 includes a processor 902, a memory, and a network interface 905, which are connected by a system bus 901, wherein the memory may include a non-volatile storage medium 903 and an internal memory 904.

The non-volatile storage medium 903 may store an operating system 9031 and a computer program 9032. The computer program 9032, when executed, may cause the processor 902 to perform a method of increasing matting accuracy.

The processor 902 is operative to provide computing and control capabilities supporting the operation of the entire computer device 900.

The internal memory 904 provides an environment for the execution of a computer program 9032 in a non-volatile storage medium 903, which computer program 9032, when executed by the processor 902, causes the processor 902 to perform a method of increasing matting accuracy.

The network interface 905 is used for network communication such as providing transmission of data information, etc. It will be appreciated by those skilled in the art that the architecture shown in fig. 9 is merely a block diagram of some of the architecture relevant to the present inventive arrangements and is not limiting of the computer device 900 to which the present inventive arrangements may be implemented, and that a particular computer device 900 may include more or less components than those shown, or may combine some components, or have a different arrangement of components.

Those skilled in the art will appreciate that the embodiment of the computer device shown in fig. 9 is not limiting of the specific construction of the computer device, and in other embodiments, the computer device may include more or less components than those shown, or certain components may be combined, or a different arrangement of components. For example, in some embodiments, the computer device may include only a memory and a processor, and in such embodiments, the structure and function of the memory and the processor are consistent with the embodiment shown in fig. 9, and will not be described again.

It should be appreciated that in an embodiment of the invention, the Processor 902 may be a central processing unit (Central Processing Unit, CPU), the Processor 902 may also be other general purpose processors, digital signal processors (DIGITAL SIGNAL processors, DSPs), application SPECIFIC INTEGRATED Circuits (ASICs), off-the-shelf Programmable gate arrays (Field-Programmable GATEARRAY, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

In another embodiment of the invention, a computer-readable storage medium is provided. The computer readable storage medium may be a non-volatile computer readable storage medium. The computer readable storage medium stores a computer program, wherein the computer program when executed by a processor implements a method of increasing matting accuracy of an embodiment of the invention.

The storage medium is a physical, non-transitory storage medium, and may be, for example, a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk.

It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus, device and unit described above may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.

Claims

1.A method of increasing matting accuracy, comprising:

acquiring an RGB image and an initial Trimap image of an image to be scratched;

Coding the RGB image and the initial Trimap image to obtain a coded image;

Decoding the coded image, and separating a target by using the decoded coded image, the foreground edge predicted image and the accurate Trimap predicted image to obtain a target image;

The process of encoding the RGB map and the initial Trimap map to obtain an encoded image includes: inputting the RGB map and the initial Trimap into an encoder network; color feature encoding in the RGB map through the encoder network; encoding a foreground region, a background region and an unknown region in the initial Trimap image through the encoder network; performing feature fusion on the RGB image and the initial Trimap image after encoding, and outputting an encoded image;

The method for decoding the coded image, separating a target by using the decoded coded image, the foreground edge predicted image and the accurate Trimap predicted image to obtain a target image comprises the following steps: inputting the coded image into a third decoding network for decoding; and performing full connection processing on the decoded coded image, the foreground edge predicted image, the accurate Trimap predicted image and the foreground predicted image, and then performing convolution processing to obtain a target image.

2. A method of increasing matting accuracy as defined in claim 1, wherein obtaining an RGB image and an initial Trimap image of the image to be matting comprises:

extracting color features in the image to be scratched to obtain a corresponding RGB image;

And carrying out semantic segmentation processing on the image to be scratched to obtain a corresponding initial Trimap.

3. A method for increasing matting accuracy according to claim 1, characterized in that said performing foreground edge prediction processing on the coded image to obtain a foreground edge predicted image comprises:

inputting the coded image into a first decoding network to perform target edge decoding to obtain edge pixel information of a foreground;

And extracting the foreground edge of the coded image according to the edge pixel information, and outputting a foreground edge predicted image.

4. The method for increasing matting accuracy according to claim 1, wherein the performing a background rejection process on the encoded image to obtain an accurate Trimap prediction map includes:

inputting the encoded image into a second decoding network to perform background pixel decoding to obtain background pixel information;

Confirming a background area according to the background pixel information;

and eliminating the background area and outputting an accurate Trimap prediction graph.

5. A method for increasing matting accuracy according to claim 1, characterized in that said performing a target foreground prediction process on the coded image to obtain a foreground predicted image comprises:

Inputting the coded image into a fourth decoder network for decoding, and extracting the overall relation between a foreground region and an unknown region;

and carrying out pixel optimization processing on the unknown region according to the overall relation between the foreground region and the unknown region, and outputting a foreground predicted image.

6. An apparatus for increasing matting accuracy, comprising:

The target separation unit is used for decoding the coded image, and separating a target by utilizing the decoded coded image, the foreground edge predicted image and the accurate Trimap predicted image to obtain a target image;

The encoding unit is used for inputting the RGB map and the initial Trimap into an encoder network; color feature encoding in the RGB map through the encoder network; encoding a foreground region, a background region and an unknown region in the initial Trimap image through the encoder network; performing feature fusion on the RGB image and the initial Trimap image after encoding, and outputting an encoded image;

The target separation unit is used for inputting the coded image into a third decoding network for decoding; and performing full connection processing on the decoded coded image, the foreground edge predicted image, the accurate Trimap predicted image and the foreground predicted image, and then performing convolution processing to obtain a target image.

7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements a method of increasing matting accuracy as claimed in any one of claims 1 to 5 when the computer program is executed.

8. A computer readable storage medium storing a computer program which when executed by a processor causes the processor to perform a method of increasing matting accuracy as claimed in any one of claims 1 to 5.