CN118229724A - Method and device for acquiring opacity graph in portrait matting process - Google Patents

Abstract

The embodiment of the application provides a method and a device for acquiring an opacity chart in a portrait matting process, belonging to the technical field of image processing, wherein the method comprises the following steps: acquiring an original image to be scratched; performing portrait and hair segmentation and binarization processing on the original image to obtain a first binary mask and a second binary mask; performing self-adaptive morphological processing on the binary mask to obtain an initial trisection diagram of the portrait prospect; performing an iterative process, the iterative process comprising: taking the initial trisection image or the trisection image obtained in the last iteration process as prior information, inputting the prior information and the original image into a trained portrait matting network model together, and obtaining an output opacity image; dividing, binarizing and self-adapting morphology processing are carried out on the opacity graph, and a trisection graph of the current iteration process is obtained; and outputting the opacity graph of the last iteration process when the opacity graph output by the portrait matting network model meets the standard of the reference real data or reaches the maximum iteration times.

Description

Method and device for obtaining opacity map during portrait cutout process

Technical Field

The present application belongs to the field of image processing technology, and more specifically, relates to a method and device for obtaining an opacity map during a portrait cutout process.

Background technique

Portrait cutout, which is to separate the foreground of a portrait from the background image in an image and obtain its corresponding opacity map, is widely used in film and television production, photography post-processing, advertising design, education and training, etc. Commonly used portrait cutout models are mainly deep learning models, whose learning and generalization capabilities strongly depend on the quality and scale of the dataset. The design of annotation tools directly affects the quality of the dataset and the efficiency of the annotation staff. Therefore, it is crucial to design an efficient and user-friendly annotation tool.

The annotation tool can be an opacity map. There are currently two commonly used methods for obtaining portrait cutout benchmark real data (i.e., opacity map): one is to manually separate the portrait foreground and background images through a variety of tools and techniques provided by Adobe Photoshop software; the other is to switch between a variety of monochrome backgrounds, keep the foreground and camera position fixed for multiple shots to accurately control the shooting conditions, and then obtain the benchmark real data through triangulation technology.

The operations of the above two methods are relatively complicated and require a high level of professionalism from the labelers, which makes it very difficult to collect real data for portrait cutout benchmarks.

Summary of the invention

In view of the defects of the related art, the purpose of the present application is to provide a method and device for obtaining an opacity map in the process of portrait cutout, aiming to solve the problem of high difficulty in collecting real data for portrait cutout benchmarks.

In a first aspect, an embodiment of the present application provides a method for obtaining an opacity map during a portrait cutout process, comprising:

Get the original image to be cut out;

Performing portrait segmentation and binarization processing on the original image to obtain a first binary mask; performing hair segmentation and binarization processing on the original image to obtain a second binary mask;

Performing adaptive morphological processing on the first binary mask and the second binary mask to obtain an initial trisection image of the portrait foreground;

An iterative process is performed, and the iterative process includes: using the initial trisection map or the trisection map obtained in the previous iterative process as prior information, and inputting it together with the original image into the trained portrait cutout network model to obtain an opacity map output by the portrait cutout network model; segmenting and binarizing the opacity map to obtain a third binary mask; performing adaptive morphological processing on the third binary mask to obtain a trisection map of the current iterative process;

When the opacity map output by the portrait cutout network model meets the benchmark real data standard or reaches the maximum number of iterations, the opacity map of the last iteration process is output.

In some embodiments, the portrait matting network model includes: an encoder module, a decoder block module, a bridge block embedded between the encoder module and the decoder module, and a propagation refinement module;

Get the opacity map output by the portrait cutout network model, including:

Inputting the prior information and the original image into the encoder for feature extraction to obtain a first output, where the first output includes the extracted low-level detail features and high-level semantic features;

Input the first output to the bridge block to perform a jump link between the encoder module and the decoder module to obtain the second output;

The first output and the second output are connected and input to the decoder for upsampling, and an initial opacity map is output;

The original image and the initial opacity map are input into the propagation refinement module for iterative refinement, and the refined opacity map is output.

In some embodiments, the propagation refinement module includes at least two propagation units, each propagation unit includes two ResBlock sub-units and one convolutional LSTM sub-unit.

In some embodiments, the method further comprises:

receiving a first input from a user, where the first input is used to fine-tune the tripartite map as prior information; and/or,

A second input from the user is received, where the second input is used to fine-tune the opacity map output by the portrait cutout network model.

In some embodiments, the joint training loss of the portrait matting network model includes:

Prediction loss, which is used to characterize the absolute difference between the true opacity map and the predicted opacity in the uncertain region;

Laplacian loss, which is used to characterize the _L1 distance between the Laplacian pyramid of the true opacity and the predicted opacity in the uncertain region;

The synthesis loss is used to characterize the absolute difference between the original image and the synthesized image, which is generated based on the predicted opacity, foreground image and background image.

In some embodiments, adaptive morphological processing includes:

Performing distance transformation on the target binary mask to obtain a distance map, and taking the maximum value of the distance map as the portrait size parameter; the target binary mask is the first binary mask, the second binary mask or the third binary mask;

Acquire adaptive parameters based on the portrait size parameters, where the adaptive parameters include a regional expansion parameter and an erosion parameter;

Based on the adaptive parameters, the target binary mask is respectively expanded and eroded to obtain a fourth binary mask after the expansion and erosion processing;

A foreground area of the tripartite map is determined based on the target binary mask, an uncertain area of the tripartite map is determined based on the target binary mask and the fourth binary mask, and an area other than the foreground area and the uncertain area is used as a background area of the tripartite map.

In a second aspect, an embodiment of the present application further provides a device for obtaining an opacity map during a portrait cutout process, comprising:

An original image acquisition unit, used to acquire the original image to be cut out;

The segmentation and binarization processing unit is used to perform portrait segmentation and binarization processing on the original image to obtain a first binary mask; perform hair segmentation and binarization processing on the original image to obtain a second binary mask;

A morphological processing unit, configured to perform adaptive morphological processing on the first binary mask and the second binary mask to obtain an initial trisection image of the portrait foreground;

An iteration unit is used to execute an iteration process, wherein the iteration process includes: using an initial trisection image or a trisection image obtained in a previous iteration process as prior information, and inputting the trisection image together with the original image into a trained portrait cutout network model to obtain an opacity map output by the portrait cutout network model; segmenting and binarizing the opacity map to obtain a third binary mask; performing adaptive morphological processing on the third binary mask to obtain a trisection image of the current iteration process;

The opacity map output unit is used to output the opacity map of the last iteration process when the opacity map output by the portrait cutout network model meets the benchmark real data standard or reaches the maximum number of iterations.

In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory, wherein when the programs stored in the memory are executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation manner of the first aspect.

The method and device for obtaining an opacity map in a portrait cutout process provided in an embodiment of the present application perform portrait segmentation and hair segmentation and binarization processing on an original image to obtain a first binary mask and a second binary mask, and then perform adaptive morphological processing on the binary mask to obtain an initial tri-map, thereby obtaining more accurate prior information; then, a pre-trained portrait cutout network model is used to obtain the opacity map and perform an iterative cycle until the iterative condition is met, and then the opacity map is output as the reference real data, thereby achieving automatic, accurate and rapid acquisition of the reference real data.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the present application or related technologies, the following is a brief introduction to the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a schematic diagram of a flow chart of a method for obtaining an opacity map in a portrait cutout process provided in an embodiment of the present application;

FIG2 is a second flow chart of a method for obtaining an opacity map in a portrait cutout process provided in an embodiment of the present application;

FIG3 is a third flow chart of a method for obtaining an opacity map in a portrait cutout process provided in an embodiment of the present application;

FIG4 is a schematic diagram of the structure of a device for obtaining an opacity map during a portrait cutout process provided by an embodiment of the present application;

FIG5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

The term "and/or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol "/" in this article indicates that the associated objects are in an or relationship, for example, A/B means A or B.

The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

FIG. 1 is a flow chart of a method for obtaining an opacity map in a portrait cutout process provided by an embodiment of the present application. As shown in FIG. 1 , the method includes at least the following steps:

S101, obtaining an original image to be cut out.

S102, performing portrait segmentation and binarization processing on the original image to obtain a first binary mask; performing hair segmentation and binarization processing on the original image to obtain a second binary mask.

Specifically, the binarization of the image is to set the grayscale values of the pixels on the image to 0 and 255, that is, the entire image presents an obvious visual effect of only black and white. The purpose of the binarization is to separate the target of interest from the background. The target of interest in the embodiment of the present application is the portrait area and the hair area in the original image. Portrait segmentation and binarization and hair segmentation and binarization can be achieved by any general network learning model. In actual application, the portrait segmentation model and the hair segmentation model can also be continuously updated to obtain a more accurate initial tri-map result.

S103: Perform adaptive morphological processing on the first binary mask and the second binary mask to obtain an initial trisection image of the portrait foreground.

Specifically, by performing adaptive morphological processing on the first binary mask and the second binary mask, each pixel in the original image can be divided into a foreground area, an uncertain area, and a background area, so as to obtain an initial tripartite map of the portrait foreground.

S104, executing an iterative process, the iterative process comprising: using the initial tri-map or the tri-map obtained in the previous iterative process as prior information, and inputting it together with the original image into the trained portrait cutout network model to obtain an opacity map output by the portrait cutout network model; segmenting and binarizing the opacity map to obtain a target binary mask; and performing adaptive morphological processing on the target binary mask to obtain a tri-map of the current iterative process.

Specifically, for the first iteration, the initial trisection image is used as prior information and the original image is input into the trained portrait cutout network model; for the non-first iteration, the trisection image obtained in the previous iteration process is used as prior information and the original image is input into the trained portrait cutout network model to obtain the opacity map output by the model in the current iteration process. The opacity map output by the model is segmented and binarized to obtain the target binary mask, and then adaptive morphological processing is performed to finally output the trisection image, completing a complete iteration process.

S105. When the opacity map output by the portrait cutout network model meets the benchmark real data standard or reaches the maximum number of iterations, the opacity map of the last iteration process is output.

Specifically, when the opacity map output by the model during the iteration process meets the benchmark real data annotation, or when the maximum number of iterations is reached, the opacity map output by the model during the last iteration is output, which is both the cutout result and the benchmark real data annotation result.

The method for obtaining an opacity map in a portrait cutout process provided in an embodiment of the present application performs portrait segmentation and hair segmentation and binarization on an original image to obtain a first binary mask and a second binary mask, and then performs adaptive morphological processing on the binary mask to obtain an initial tri-map, thereby obtaining more accurate prior information; then, a pre-trained portrait cutout network model is used to obtain the opacity map and perform an iterative cycle until the iterative condition is met, and then the opacity map is output as the reference real data, thereby achieving automated, accurate and rapid acquisition of the reference real data.

In some embodiments, the method for obtaining the opacity map during the portrait cutout process further includes:

receiving a first input from a user, where the first input is used to fine-tune the tripartite map as prior information; and/or,

A second input from the user is received, where the second input is used to fine-tune the opacity map output by the portrait cutout network model.

Specifically, before the triplicated image, which is used as prior information, is input into the portrait cutout network model, the category labels of the pixels in the local area of the triplicated image can be adjusted by the annotator through fine-tuning operations on the triplicated image to obtain more accurate prior information.

It is conceivable that before the opacity map output by the portrait cutout network model is segmented and binarized, the semantic information of the pixels in the local area can also be changed by fine-tuning the opacity map by the annotator.

The fine-tuning operation can specifically be to add polygons, polygons, rectangles, circles, polylines, line segments, and point annotations to the pixels in the local area, change the semantic category of the pixels, such as correcting the foreground pixels misclassified as background to foreground pixels, or modifying the pixel range of the uncertain area.

The method for obtaining the opacity map during the portrait cutout process provided in the embodiment of the present application combines a deep learning algorithm and prediction results with simple manual fine-tuning to quickly obtain benchmark real data, thereby reducing the difficulty of labeling work during the cutout process and improving labeling efficiency.

FIG. 2 is a second flow chart of a method for obtaining an opacity map in a portrait cutout process provided by an embodiment of the present application. As shown in FIG. 2 , the portrait cutout network model specifically includes: an encoder, a decoder, a bridge block embedded between the encoder and the decoder, and a propagation refinement module; obtaining the opacity map output by the portrait cutout network model in S104 specifically includes:

Inputting the prior information and the original image into the encoder for feature extraction to obtain a first output, where the first output includes the extracted low-level detail features and high-level semantic features;

Input the first output to the bridge block to perform a jump link between the encoder module and the decoder module to obtain the second output;

The first output and the second output are connected and input to the decoder for upsampling, and an initial opacity map is output;

The original image and the initial opacity map are input into the propagation refinement module for iterative refinement, and the refined opacity map is output.

Specifically, the portrait cutout network is an encoder-decoder network containing jump connections, which predicts a binary mask of the portrait foreground based on the original image. The portrait cutout network model includes an encoder module, a decoder block module, a bridge block embedded between the encoder module and the decoder module, and a propagation refinement module.

The encoder module is responsible for extracting rich semantic features and detail features from portrait image samples and prior information (i.e., the triplicate image) to obtain the first output; the decoder module upsamples the first output of the encoder module, retains the necessary local detail information, and predicts a preliminary opacity map; the propagation refinement module consists of three propagation units, which further refines the output of the decoder module to obtain a refined prediction.

The input of the encoder module is converted into a downsampled feature map through subsequent convolutional layers and maximum pooling layers. Specifically, the encoder module has 14 convolutional layers and 5 maximum pooling layers.

The decoder module uses unpooling layers, inverse max pooling operations, and convolutional layers to upsample feature maps and output a rough opacity map. The decoder module uses a smaller structure than the encoder network to reduce the number of parameters and speed up the training process. Specifically, the decoder module has 6 convolutional layers, 5 unpooling layers, and a final alpha prediction layer.

A bridge block is inserted between the encoder module and the decoder module to utilize local context in different receptive fields. As shown in Figure 2, the bridge block consists of three dilated convolutional layers. The features output by the encoder module and the bridge block are connected and input to the decoder module. In the embodiment of the present application, the style of U-net is followed, and a jump link is made between the encoder module and the decoder module to preserve fine details.

The propagation refinement module contains at least two propagation units, each of which consists of two ResBlocks and a convolutional LSTM subunit, and its input is the original image and the output of the decoder module. As shown in Figure 2, the propagation refinement module contains three propagation units, which take the input image, fused features, and previous opacity propagation results as input in each loop iteration. ResBlocks extract features from the input, while the convolutional LSTM retains memory between propagation steps. The propagation unit gradually refines the predicted opacity map, producing a final result with more accurate edge details and fewer undesirable artifacts.

In some embodiments, the joint training loss of the portrait cutout IoT model includes:

Prediction loss, which is used to calculate the absolute difference between the true opacity and the predicted opacity of the uncertain region in the input triplicate map;

Laplacian loss, which is used to calculate the L1 distance between the true opacity of the uncertain region in the input triplicate map and the Laplacian pyramid of the predicted opacity;

The synthesis loss is used to calculate the absolute difference between the original image and the synthesized image, which is generated based on the predicted opacity, foreground image and background image.

Specifically, the joint loss function used in the training process of the portrait cutout network model is the sum of the prediction loss _LT , the Laplace loss _Llap and the synthesis loss _Lcom :

L＝ _LT + _Llap + _Lcom

The prediction loss _LT is used to characterize the absolute difference between the true opacity _αp and the predicted opacity _αp in the uncertain region, specifically:

Where i represents the pixel index, and W _i ^T ∈ {0, 1) represents whether the pixel i belongs to the uncertain region. In order to calculate the stability, let ε = 10 ^-6 .

The Laplace loss L _lap is used to characterize the L ₁ distance between the Laplace pyramid of the true opacity α _p and the predicted opacity α _p in the uncertain region, specifically:

Wherein, Lap ^k represents the kth layer of the Laplacian pyramid.

The synthesis loss L _com is used to characterize the absolute difference between the input image I and the synthesized image generated according to the predicted opacity α _p , the foreground image F and the background B, specifically:

L _com =||α _p F+(1-α _p )BI||

In some embodiments, the adaptive morphological processing specifically includes:

Performing distance transformation on the hair binary mask to obtain a distance map, and taking the maximum value of the distance map as the portrait size parameter; the target binary mask is the first binary mask, the second binary mask or the third binary mask;

Acquire adaptive parameters based on the portrait size parameters, where the adaptive parameters include a regional expansion parameter and an erosion parameter;

Based on the adaptive parameters, the target binary mask is respectively expanded and eroded to obtain a fourth binary mask after the expansion and erosion processing;

A foreground area of the tripartite map is determined based on the target binary mask, an uncertain area of the tripartite map is determined based on the target binary mask and the fourth binary mask, and an area other than the foreground area and the uncertain area is used as a background area of the tripartite map.

Specifically, performing adaptive morphological processing on the binary mask to obtain a triplicate map of the portrait foreground refers to generating a triplicate map by adaptively dilating and corroding the binary mask, that is, calculating adaptive morphological parameters according to the foreground size of the image, thereby performing partition dilation and corrosion on the portrait or hair.

First, we perform distance transformation on the target binary mask to obtain a distance map, and use the maximum value of the distance map as the portrait size parameter. Distance transformation describes the distance between a pixel in the image and a certain area block. The pixel value in the area block is 0, and the pixel value of the adjacent area block is smaller. The farther away from the area block, the larger the pixel value.

Based on the portrait size parameters, adaptive parameters are obtained. The adaptive parameters specifically include regional expansion parameters and erosion parameters. The head region expansion parameters satisfy:

head_dilate_intersize = D * head_parameter / 100

Among them, head_dilate_intersize is an adjustable head area expansion parameter, which is generally set to 3.5.

The body area expansion parameters satisfy:

body_dilate_intersize = D * body_parameter / 100

Among them, body_dilate_intersize is an adjustable body area expansion parameter, which is generally set to 1.5.

The corrosion parameters meet the following requirements:

erode _intersize = D*body_parameter/100

Furthermore, based on the aforementioned adaptive parameters, the binary masks of the hair and the portrait are respectively dilated and eroded to obtain a fourth binary mask after dilation and erosion.

The original binary mask is used as the foreground area of the tripartite map, the result of subtracting the original binary mask from the fourth binary mask after dilation and erosion processing is used as the uncertain area of the tripartite map, and the remaining pixels are used as the background area of the tripartite map.

Optionally, the binarization processing rules during the iteration process are as follows:

Wherein, m is the binarized portrait mask, and α is the (fine-tuned) opacity map. The portrait mask is subjected to dilation and erosion operations to generate a trisection map, and the adaptive parameters use the body region dilation parameters and erosion parameters obtained in S102.

The technical solution provided in the embodiment of the present application is further illustrated below through a specific example.

FIG. 3 is a third flow chart of a method for obtaining an opacity map during a portrait cutout process provided in an embodiment of the present application. As shown in FIG. 3 , the method at least includes:

Step a, obtaining the image to be labeled;

Step b, performing portrait segmentation and hair segmentation processing on the image respectively to obtain binary masks of the portrait and hair;

Step c: Perform morphological processing on the two segmentation outputs to generate a three-part image of the portrait foreground;

Step d: The annotator makes appropriate fine-tuning on the three-dimensional image (this step can be omitted);

Step e: input the three-part image and the original image into the portrait cutout network to obtain the opacity map predicted by the algorithm;

Step f, the labeler fine-tunes the opacity map (this step can be omitted);

Step g: Binarize the fine-tuned opacity map to obtain the corresponding binary mask, and then generate the corresponding trisection map. Repeat steps e to g until the opacity map meets the benchmark real data accuracy, and the labeling is completed.

In this example, the training data set used to train the portrait cutout network model is a sample of portrait images. Each original portrait image may contain one or more portrait foregrounds. After obtaining the original image, portrait segmentation and hair segmentation are performed on it to obtain a portrait binary mask and a hair binary mask; next, the two binary masks are subjected to adaptive morphological processing, as well as adaptive dilation and erosion, to obtain a triplicate map of the portrait foreground; the annotator can make appropriate fine-tuning to the triplicate map, change the semantic information of the pixels, and input it into the portrait cutout network to predict the opacity map; the annotator can also fine-tune the opacity map, or directly re-input the predicted opacity map as prior information into the portrait cutout network for prediction, and repeat steps e to g until the accuracy requirements of the benchmark real data are met.

FIG4 is a schematic diagram of the structure of a device for obtaining an opacity map in a portrait cutout process provided by an embodiment of the present application. As shown in FIG4 , the device at least includes:

The original image acquisition unit 401 is used to acquire the original image to be cut out;

The segmentation and binarization processing unit 402 is used to perform portrait segmentation and binarization processing on the original image to obtain a first binary mask; perform hair segmentation and binarization processing on the original image to obtain a second binary mask;

A morphological processing unit 403 is used to perform adaptive morphological processing on the first binary mask and the second binary mask to obtain an initial trisection image of the portrait foreground;

The iteration unit 404 is used to perform an iteration process, which includes: using the initial trisection image or the trisection image obtained in the previous iteration process as prior information, and inputting it together with the original image into the trained portrait cutout network model to obtain an opacity map output by the portrait cutout network model; segmenting and binarizing the opacity map to obtain a third binary mask; performing adaptive morphological processing on the third binary mask to obtain a trisection image of the current iteration process;

The opacity map output unit 405 is used to output the opacity map of the last iteration process when the opacity map output by the portrait cutout network model meets the benchmark real data standard or reaches the maximum number of iterations.

In some embodiments, the portrait cutout network model includes: an encoder module, a decoder block module, a bridge block embedded between the encoder module and the decoder module, and a propagation refinement module; the iteration unit 404 is specifically used to:

Inputting the prior information and the original image into the encoder for feature extraction to obtain a first output, where the first output includes the extracted low-level detail features and high-level semantic features;

Input the first output to the bridge block to perform a jump link between the encoder module and the decoder module to obtain the second output;

The first output and the second output are connected and input to the decoder for upsampling, and an initial opacity map is output;

The original image and the initial opacity map are input into the propagation refinement module for iterative refinement, and the refined opacity map is output.

In some embodiments, the propagation refinement module includes at least two propagation units, each propagation unit includes two ResBlock sub-units and one convolutional LSTM sub-unit.

In some embodiments, the device further comprises a user input receiving unit, configured to:

receiving a first input from a user, where the first input is used to fine-tune the tripartite map as prior information; and/or,

A second input from the user is received, where the second input is used to fine-tune the opacity map output by the portrait cutout network model.

In some embodiments, the joint training loss of the portrait matting network model includes:

Prediction loss, which is used to characterize the absolute difference between the true opacity map and the predicted opacity in the uncertain region;

Laplacian loss, which is used to characterize the _L1 distance between the Laplacian pyramid of the true opacity and the predicted opacity in the uncertain region;

The synthesis loss is used to characterize the absolute difference between the original image and the synthesized image, which is generated based on the predicted opacity, foreground image and background image.

In some embodiments, adaptive morphological processing includes:

Performing distance transformation on the target binary mask to obtain a distance map, and taking the maximum value of the distance map as the portrait size parameter; the target binary mask is the first binary mask, the second binary mask or the third binary mask;

Acquire adaptive parameters based on the portrait size parameters, where the adaptive parameters include a regional expansion parameter and an erosion parameter;

Based on the adaptive parameters, the target binary mask is respectively expanded and eroded to obtain a fourth binary mask after the expansion and erosion processing;

A foreground area of the tripartite map is determined based on the target binary mask, an uncertain area of the tripartite map is determined based on the target binary mask and the fourth binary mask, and an area other than the foreground area and the uncertain area is used as a background area of the tripartite map.

It is understandable that the detailed functional implementation of each of the above-mentioned units/modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

Based on the method in the above embodiment, an embodiment of the present application provides an electronic device. The device may include: at least one memory for storing programs and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the above embodiment.

FIG5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. As shown in FIG5 , the electronic device may include: a processor 501, a communication interface 520, a memory 503, and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504. The processor 501 may call the software instructions in the memory 503 to execute the method described in the above embodiment.

In addition, the logic instructions in the above-mentioned memory 503 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present application.

Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

It is understandable that the processor in the embodiment of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by a computer-readable storage medium. The computer instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims (10)

1. A method for obtaining an opacity map in a portrait cutout process, comprising: Get the original image to be cut out; Performing portrait segmentation and binarization processing on the original image to obtain a first binary mask; performing hair segmentation and binarization processing on the original image to obtain a second binary mask; Performing adaptive morphological processing on the first binary mask and the second binary mask to obtain an initial trisection image of the portrait foreground; An iterative process is performed, wherein the iterative process includes: using the initial trisection image or the trisection image obtained in the previous iterative process as prior information, and inputting it together with the original image into a trained portrait cutout network model to obtain an opacity map output by the portrait cutout network model; segmenting and binarizing the opacity map to obtain a third binary mask; performing adaptive morphological processing on the third binary mask to obtain a trisection image of the current iterative process; When the opacity map output by the portrait cutout network model meets the benchmark real data standard or reaches the maximum number of iterations, the opacity map of the last iteration process is output. 2. The method for obtaining an opacity map in a portrait cutout process according to claim 1, wherein the portrait cutout network model comprises: an encoder module, a decoder block module, a bridge block embedded between the encoder module and the decoder module, and a propagation refinement module; The obtaining of the opacity map output by the portrait cutout network model comprises: Inputting the prior information and the original image into the encoder for feature extraction to obtain a first output, wherein the first output includes the extracted low-level detail features and high-level semantic features; Inputting the first output to the bridge block to perform a jump link between the encoder module and the decoder module to obtain a second output; Connecting the first output and the second output and inputting them into the decoder for upsampling, and outputting an initial opacity map; The original image and the initial opacity map are input into the propagation refinement module for iterative refinement, and a refined opacity map is output. 3. According to the method for obtaining the opacity map in the portrait cutout process described in claim 2, it is characterized in that the propagation refinement module includes at least two propagation units, each propagation unit includes two ResBlock sub-units and one convolutional LSTM sub-unit. 4. The method for obtaining an opacity map during a portrait cutout process according to claim 1, characterized in that the method further comprises: receiving a first input from a user, wherein the first input is used to fine-tune the tripartite map as prior information; and/or, A second input from a user is received, where the second input is used to fine-tune the opacity map output by the portrait cutout network model. 5. The method for obtaining an opacity map in a portrait cutout process according to claim 1, wherein the joint training loss of the portrait cutout network model comprises: Prediction loss, which is used to characterize the absolute difference between the true opacity map and the predicted opacity in the uncertain region; Laplacian loss, which is used to characterize the _L1 distance between the Laplacian pyramid of the true opacity and the predicted opacity in the uncertain region; The synthesis loss is used to characterize the absolute difference between the original image and the synthesized image generated based on the predicted opacity, foreground image and background image. 6. The method for obtaining an opacity map during a portrait cutout process according to claim 1, wherein the adaptive morphological processing comprises: Performing distance transformation on the target binary mask to obtain a distance map, and taking the maximum value of the distance map as the portrait size parameter; the target binary mask is the first binary mask, the second binary mask or the third binary mask; Acquire adaptive parameters based on the portrait size parameters, where the adaptive parameters include a region expansion parameter and an erosion parameter; Based on the adaptive parameters, the target binary mask is respectively expanded and eroded to obtain a fourth binary mask after the expansion and erosion processing; A foreground area of the tri-map is determined based on the target binary mask, an uncertain area of the tri-map is determined based on the target binary mask and the fourth binary mask, and an area other than the foreground area and the uncertain area is used as a background area of the tri-map. 7. A device for obtaining an opacity map during a portrait cutout process, comprising: An original image acquisition unit, used to acquire the original image to be cut out; A segmentation and binarization processing unit, configured to perform portrait segmentation and binarization processing on the original image to obtain a first binary mask; and perform hair segmentation and binarization processing on the original image to obtain a second binary mask; A morphological processing unit, configured to perform adaptive morphological processing on the first binary mask and the second binary mask to obtain an initial trisection image of the portrait foreground; An iterative unit is used to perform an iterative process, wherein the iterative process includes: using the initial trisection map or the trisection map obtained in the previous iterative process as prior information, and inputting it together with the original image into a trained portrait cutout network model to obtain an opacity map output by the portrait cutout network model; segmenting and binarizing the opacity map to obtain a third binary mask; and performing adaptive morphological processing on the third binary mask to obtain a trisection map of the current iterative process; The opacity map output unit is used to output the opacity map of the last iteration process when the opacity map output by the portrait cutout network model meets the benchmark real data standard or reaches the maximum number of iterations. 8. An electronic device, comprising: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 6. 9. A computer-readable storage medium storing a computer program, wherein when the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 6. 10. A computer program product, characterized in that when the computer program product is run on a processor, the processor is caused to execute the method according to any one of claims 1 to 6.