CN113709365B - Image processing method, device, electronic equipment and storage medium - Google Patents

Abstract

An image processing method, an image processing device, an electronic device and a storage medium, wherein the method comprises the following steps: acquiring aperture shape parameters, and acquiring an blurring dynamics map and a weight map corresponding to a first image, wherein the blurring dynamics map at least comprises blurring radiuses respectively corresponding to all first pixel points in a background area of the first image, and the weight map at least comprises diffusion weights respectively corresponding to all first pixel points in the background area of the first image; performing point diffusion processing on each first pixel point in a background area of the first image according to the aperture shape parameter, the blurring dynamics map and the weight map to obtain a diffusion result corresponding to each first pixel point; and superposing diffusion results corresponding to the first pixel points in the background area, and blurring the background area according to the superposition results. By implementing the embodiment of the application, the natural and real image blurring effect can be provided.

Description

Image processing method, device, electronic device and storage medium

technical field

The present application relates to the field of image technology, and in particular to an image processing method, device, electronic equipment, and storage medium.

Background technique

Currently, most image blur effects are based on Gaussian blur and spot maps. Among them, the spot map can mainly blur the light source with higher brightness in the image. However, in practice, it is found that although both Gaussian blur and spot map can achieve a blur effect, the two are separated from each other, resulting in an unnatural and unreal overall blur effect of the image.

Contents of the invention

The embodiment of the present application discloses an image processing method, device, electronic equipment, and storage medium, which can provide a natural and real image blurring effect.

The embodiment of the present application discloses an image processing method. The method includes: obtaining the shape parameter of the aperture, and obtaining a blurring intensity map and a weighting map corresponding to the first image, and the blurring intensity map includes at least the blurring intensity map of the first image. Blur radii corresponding to each first pixel in the background area, the weight map at least includes diffusion weights corresponding to each first pixel in the background area of the first image; according to the aperture shape parameter, the Blur intensity map and weight map, performing point diffusion processing on each first pixel in the background area of the first image to obtain a diffusion result corresponding to each first pixel; for each first pixel in the background area The diffusion results corresponding to the points are superimposed, and the background area is blurred according to the superposition results.

The embodiment of the present application discloses an image processing device, including: an acquisition module for obtaining aperture shape parameters, and obtaining a blurring strength map and a weight map corresponding to the first image, and the blurring strength map includes at least the first Blur radii corresponding to each first pixel in the background area of the image, the weight map at least includes diffusion weights corresponding to each first pixel in the background area of the first image; the diffusion module is configured to The shape parameters of the aperture, the blurring intensity map and the weight map are used to perform point diffusion processing on each first pixel point in the background area of the first image to obtain a diffusion result corresponding to each first pixel point; superimposing blurring A module, configured to superimpose the diffusion results corresponding to each first pixel in the background area, and blur the background area according to the superposition results.

The embodiment of the present application discloses an electronic device, including a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes any of the methods disclosed in the embodiment of the present application. An image processing method.

The embodiment of the present application discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any image processing method disclosed in the embodiment of the present application is implemented.

Compared with related technologies, the embodiments of the present application have the following beneficial effects:

The electronic device can obtain the shape parameter of the aperture, and obtain the blur strength map and the weight map corresponding to the first image. The blur strength map at least includes blur radii corresponding to the respective first pixels in the background area of the first image, and the weight map includes at least the blur radii corresponding to the respective first pixels in the background area of the first image. Diffusion weights.

The electronic device may further perform point diffusion processing on each of the first pixels in the background area of the first image according to the aperture shape parameter, the blurring intensity map, and the weight map, to obtain a pixel corresponding to each first pixel. Diffusion results, and superimpose the diffusion results to blur the background area according to the superposition results, so that the real lens blur can be simulated through point diffusion processing, which solves the problem of Gaussian blur and fragmentation of the spot map, making the image as a whole The bokeh effect is harmonious and natural, and it can simulate the natural bokeh and bright and transparent effect of SLR camera photography, which greatly improves the texture of the bokeh effect. In addition, the electronic device can perform point diffusion processing on each first pixel point in the background area without identifying the position of the light source point, thereby avoiding the problem of inaccurate recognition of the position of the light source point and resulting in a decrease in the blur effect, which is conducive to providing more Realistic and natural bokeh effect.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings that need to be used in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without making creative efforts.

Fig. 1 is a schematic structural diagram of an image processing circuit disclosed in an embodiment;

Fig. 2 is a method flow diagram of an image processing method disclosed in an embodiment;

Fig. 3 is an example diagram of superimposing diffusion results corresponding to two adjacent first pixel points disclosed by an embodiment;

Fig. 4 is a schematic flow chart of another image processing method disclosed in an embodiment;

Fig. 5 is a schematic flowchart of a method for generating a background mask disclosed by an embodiment;

Fig. 6 is a schematic flowchart of a method for darkening the brightness of the background area disclosed by an embodiment;

Fig. 7 is a schematic flowchart of a method for color enhancement of a highlight area disclosed by an embodiment;

Fig. 8 is a schematic flowchart of a method for obtaining a brightness weight map disclosed by an embodiment;

Fig. 9 is a schematic flowchart of a method for obtaining a chromaticity weight map disclosed by an embodiment;

Fig. 10 is an example diagram of a first curve, a second curve and a weight mapping curve disclosed by an embodiment;

Fig. 11 is a schematic flowchart of an image processing method disclosed in an embodiment;

Fig. 12 is a schematic structural diagram of an image processing device disclosed in an embodiment;

FIG. 13 is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some, not all, embodiments of the application. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of the present application and the drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally further includes For other steps or units inherent in these processes, methods, products or apparatuses.

In related technologies, the image blurring effect can be realized based on Gaussian blur and light spot map. Among them, Gaussian blur is generally based on the convolution of the gather method, while the spot map is based on the pixel accumulation of the spread method. Therefore, the two are separated from each other, making it difficult to integrate the virtual effect achieved by Gaussian blur and spot map respectively. Also, Gaussian Blur is simulated lens blur, there is actually a gap between it and real lens blur. The spot map relies on the recognition of the light source point. It is necessary to identify the position of the light source point in the image first, and then blur the position of the light source point. However, if the location of the light source point is not identified accurately, the effect of blurring the light spots in the image will be greatly reduced, and there will be a large gap between it and the real lens blur. To sum up, the image blurring effect in the related art has the problem of being unnatural and unreal.

The embodiment of the present application discloses an image processing method, device, electronic equipment, and storage medium, which can provide a natural and real image blurring effect. Each will be described in detail below.

Please refer to FIG. 1 . FIG. 1 is a schematic structural diagram of an image processing circuit disclosed by an embodiment. The image processing circuit can be applied to electronic devices such as smart phones, smart tablets, and smart watches, but is not limited thereto. As shown in FIG. 1 , the image processing circuit may include an imaging device (camera) 110, an attitude sensor 120, an image memory 130, an image signal processing (Image Signal Processing, ISP) processor 140, a logic controller 150 and a display 160.

The image processing circuit includes an ISP processor 140 and a control logic 150 . Image data captured by imaging device 110 is first processed by ISP processor 140 , which analyzes the image data to capture image statistics that can be used to determine one or more control parameters of imaging device 110 . Imaging device 110 may include one or more lenses 112 and image sensor 114 . Image sensor 114 may include a color filter array (such as a Bayer filter), image sensor 114 may obtain light intensity and wavelength information captured by each imaging pixel, and provide a set of raw image data (RAW image) that may be processed by ISP processor 140. data). The attitude sensor 120 (such as a three-axis gyroscope, Hall sensor, accelerometer, etc.) can provide the collected image processing parameters (such as anti-shake parameters) to the ISP processor 140 based on the interface type of the attitude sensor 120 . The attitude sensor 120 interface may adopt a SMIA (Standard Mobile Imaging Architecture, Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.

In addition, the image sensor 114 can also send the original image data to the attitude sensor 120, and the attitude sensor 120 can provide the original image data to the ISP processor 140 based on the attitude sensor 120 interface type, or the attitude sensor 120 can store the original image data in the image memory 130 in.

The ISP processor 140 processes raw image data on a pixel-by-pixel basis in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 140 may perform one or more image processing operations on the raw image data, gather statistical information about the image data. Among other things, image processing operations can be performed with the same or different bit depth precision.

ISP processor 140 may also receive image data from image memory 130 . For example, the attitude sensor 120 interface sends raw image data to the image storage 130, and the raw image data in the image storage 130 is provided to the ISP processor 140 for processing. The image memory 130 may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include a DMA (Direct Memory Access) feature.

When receiving raw image data from the image sensor 114 interface or from the attitude sensor 120 interface or from the image memory 130 , the ISP processor 140 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 130 for additional processing before being displayed. The ISP processor 140 receives the processed data from the image memory 130, and performs image data processing in the original domain and one or more color spaces such as YUV, RGB, YCbCr, etc. on the processed data. The image data processed by the ISP processor 140 may be output to the display 160 for viewing by the user and/or for further processing by a graphics engine or a GPU (Graphics Processing Unit, graphics processor). In addition, the output of the ISP processor 140 can also be sent to the image memory 130 , and the display 160 can read image data from the image memory 130 . In one embodiment, image memory 130 may be configured to implement one or more frame buffers.

Statistics determined by ISP processor 140 may be sent to control logic 150 . For example, the statistical data may include the vibration frequency of the gyroscope, automatic exposure, automatic white balance, automatic focus, flicker detection, black level compensation, lens 112 shading correction and other image sensor 114 statistical information. Control logic 150 may include a processor and/or a microcontroller that executes one or more routines (e.g., firmware) that determine control parameters of imaging device 110 and ISP processing based on received statistical data. The control parameters of the device 140. For example, the control parameters of the imaging device 110 may include attitude sensor 120 control parameters (such as gain, integration time of exposure control, anti-shake parameters, etc.), camera flash control parameters, camera anti-shake displacement parameters, lens 112 control parameters (such as focus or focal length for zooming) or a combination of these parameters. ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during YUV processing), as well as lens 112 shading correction parameters.

In one embodiment, the imaging device 110 can capture the first image, and send the first image to the ISP processor 140 , or store the first image in the image memory 130 . The ISP processor can acquire the first image, the shape of the aperture, and the blurring intensity map and weight map corresponding to the first image, and the blurring intensity map can at least include blurring radii corresponding to each first pixel in the background area of the first image. , the weight map may at least include diffusion weights corresponding to respective first pixel points in the background area of the first image.

The ISP processor 140 may perform point diffusion processing on each of the first pixels in the background area of the first image according to the obtained aperture shape parameters, blurring intensity map, and weight map to obtain a diffusion result corresponding to each first pixel. . And, the ISP processor 140 may also superimpose the diffusion results corresponding to each first pixel in the background area, and blur the background area of the first image according to the superimposition results.

In some embodiments, the ISP processor 140 may also send the second image obtained by blurring the background area of the first image to the display 160 so as to display the second image through the display 160 .

Please refer to FIG. 2 . FIG. 2 is a schematic flowchart of an image processing method disclosed in an embodiment, and the method can be applied to the aforementioned electronic device. As shown in Figure 2, the method may include the following steps:

210. Acquire the shape parameters of the aperture, and acquire the blur strength map and weight map corresponding to the first image.

The aperture shape parameter can be input by the user and can be used to simulate the aperture shape of different lenses. For example, the shape of the aperture simulated by the aperture shape parameter may include: circle, ring, heart, star, irregular figure, etc., but not limited thereto.

The first image may be an image captured by an imaging device of the electronic device, or an image obtained by the electronic device from another terminal or service device, which is not specifically limited. The first image may include a foreground area and a background area. The foreground area may refer to an area occupied by a subject such as a portrait in the image, and the background area may refer to other image areas in the first image except the foreground area. Taking the portrait area as an example in the foreground area, the electronic device may distinguish the portrait area and the background area of the first image through image recognition methods such as portrait cutout or portrait segmentation.

The blur strength map corresponding to the first image may at least include blur radii corresponding to respective first pixels in the background area of the first image. Wherein, the first pixel point may be any pixel point in the background area of the first image. The blur radius can be used to indicate the degree of blur. The larger the blur radius, the blurrier the blurred image. In the blur strength map corresponding to the first image, the blur radii corresponding to different first pixels may be the same or different.

The blur intensity map corresponding to the first image may be generated according to the depth information of each first pixel in the background area of the first image, and the depth information may be used to indicate the physical distance between the object in the background area and the imaging device. For example, the depth information of the first pixel may be positively correlated with the blur radius corresponding to the first pixel. The electronic device can estimate the depth of the first image through methods such as structured light, Time of Flight (TOF), binocular stereo imaging, monocular phase detection, and monocular depth estimation based on deep learning or machine learning. , so as to at least obtain the depth of field information of each first pixel in the background area of the first image, but not limited thereto.

The weight map corresponding to the first image may at least include diffusion weights respectively corresponding to each first pixel in the background area of the first image. The weight map corresponding to the first image can be used to guide the point diffusion processing of the first pixel. In the weight map corresponding to the first image, the diffusion weights corresponding to different first pixel points may be the same or different. The weight map corresponding to the first image may be generated according to the pixel values of each first pixel in the background area of the first image. Wherein, the pixel value may include a luma component value and a chrominance component value.

Exemplarily, the electronic device may perform image data processing on the first image in the YUV color space through the ISP processor. The luminance component value of the first pixel can be the Y component value of the first pixel on the Y channel, and the chrominance component value of the first pixel can include the U component value of the first pixel on the U channel and the first pixel The V component value of the point on the V channel.

According to the aperture shape parameter, the blurring strength map and the weight map corresponding to the first image, perform point diffusion processing on each first pixel point in the background area of the first image, and obtain a diffusion result corresponding to each first pixel point.

The electronic device may traverse each first pixel in the background area of the first image. For the currently accessed first pixel point, the electronic device may obtain the blur radius corresponding to the currently accessed first pixel point from the blur strength map corresponding to the first image, and obtain the blur radius corresponding to the first pixel point from the weight map corresponding to the first image. The diffusion weight corresponding to the first pixel currently accessed, and according to the aperture shape parameter, the blur radius and the diffusion weight corresponding to the first pixel currently accessed, perform point diffusion processing on the first pixel currently accessed to obtain Diffusion result corresponding to the first pixel currently accessed.

Point spread processing can refer to simulating real lens blur through Point Spread Function (PSF), the pixel value of the first pixel in the first image, according to the diffusion weight corresponding to the first pixel and blurring The radius spreads out to a specific aperture shape.

That is, the diffusion result corresponding to the first pixel may include the diffusion range corresponding to the first pixel, the target pixel value of each second pixel within the diffusion range corresponding to the first pixel, and the target pixel value of each second pixel corresponding to the first pixel. Diffusion times of each second pixel within the diffusion range of . Wherein, the diffusion range corresponding to the first pixel point is centered on the first pixel point, the radius of the circumscribed circle of the diffusion range is the blur radius corresponding to the first pixel point, and the outer contour of the diffusion range can be the aperture shape indicated by the aperture shape parameter . The above-mentioned second pixel point can be any pixel point within the diffusion range corresponding to the first pixel point, therefore, each second pixel point within the diffusion range corresponding to the first pixel point includes the first pixel point located in the center of the diffusion range .

The diffusion results corresponding to the first pixel points in the background area are superimposed, and the background area is blurred according to the superposition results.

Superimposing the diffusion results corresponding to each first pixel in the background area by the electronic device may include: superimposing the target pixel values of each second pixel within the diffusion range corresponding to each first pixel in the background area, and The diffusion times of each second pixel point within the diffusion range corresponding to each first pixel point are superimposed.

Therefore, the superimposition result may include the superimposed pixel value of each first pixel point in the background area, and the total number of diffusions of each first pixel point in the background area after superimposition.

For example, please refer to FIG. 3 . FIG. 3 is an example diagram of superimposing diffusion results corresponding to two adjacent first pixel points disclosed by an embodiment. As shown in FIG. 3 , the diffusion range corresponding to the first pixel point 310 may include 9 second pixel points, and the target pixel values of the 9 second pixel points may be A1-A9 respectively. The diffusion range corresponding to the first pixel point 320 may include 9 second pixel points, and the target pixel values of the 9 second pixel points may be respectively B1-B9.

The superimposed pixel value of the first pixel point 310 may be A1+B6; the superimposed pixel value of the first pixel point 320 may be B1+A2.

The manner in which the electronic device superimposes the diffusion times of each second pixel within the diffusion range corresponding to each first pixel is similar to the manner in which the target pixel value of each second pixel is superimposed, and will not be described in detail below.

After the electronic device superimposes the diffusion results corresponding to each first pixel point in the background area and obtains the superposition results, the background area may be blurred according to the superposition results. Wherein, the electronic device blurring the background area according to the superposition result and blurring the background area may include: for each first pixel in the background area, according to the superimposed pixel value of the first pixel and the superimposed diffusion total number Calculate the blurred pixel value of the first pixel. For example, the superimposed pixel value of the first pixel may be divided by the total number of superimposed diffusions to obtain the blurred pixel value of the first pixel as a result of blurred rendering.

The electronic device performs point diffusion processing on each first pixel in the background area with a corresponding weight. Under mutual overlap, the bright spots in the first image form bright spots, while the brightness in the first image The lower scotoma forms a hazy, diffuse circle.

It can be seen that, in the above-mentioned embodiments, the electronic device can simulate the real lens blur through point diffusion processing, which solves the problem of Gaussian blur and the fragmentation of the light spot map, so that the overall blur effect of the image is harmonious and natural, integrated, and more capable It simulates the natural bokeh and bright and transparent effect of SLR camera photography, which greatly improves the texture of the bokeh effect. In addition, the electronic device can perform point diffusion processing on each first pixel point in the background area without identifying the position of the light source point, thereby avoiding the problem of inaccurate recognition of the position of the light source point and resulting in a decrease in the blur effect, which is conducive to providing more Realistic and natural bokeh effect.

In order to illustrate the point diffusion processing more clearly, please refer to FIG. 4 . FIG. 4 is a schematic flowchart of another image processing method disclosed in an embodiment, and the method can be applied to any of the aforementioned electronic devices. As shown in Figure 4, the method may include the following steps:

Obtain the shape parameters of the aperture, and obtain the blur strength map and weight map corresponding to the first image.

In some embodiments, the blur strength map corresponding to the first image may include not only the blur radii corresponding to each first pixel in the background area of the first image, but also each pixel in the foreground area of the first image. The blur radius corresponding to the third pixel. That is, the blur strength map corresponding to the first image may include a blur radius corresponding to each pixel in the full image of the first image. When the electronic device generates the blur strength map corresponding to the first image, it may generate the blur radius corresponding to each pixel according to the field depth information of each pixel included in the first image.

The weight map corresponding to the first image, in addition to the diffusion weights corresponding to each first pixel in the background area of the first image, may also include the diffusion weights corresponding to each third pixel in the foreground area of the first image . That is, the weight map corresponding to the first image may include a diffusion weight corresponding to each pixel in the full image of the first image. When generating the weight map corresponding to the first image, the electronic device may generate the diffusion weight corresponding to each pixel according to the pixel value of each pixel included in the first image.

For each first pixel in the background area of the first image, the blur radius corresponding to the first pixel is obtained from the blur strength map, and the diffusion weight corresponding to the first pixel is obtained from the weight map.

The electronic device may generate a background mask used to indicate the background area of the first image, so as to identify each first pixel contained in the background area of the first image.

The electronic device may traverse each first pixel in the background mask, obtain the blur radius corresponding to the currently accessed first pixel from the blur strength map, and obtain the corresponding value from the weight map. Diffusion weight corresponding to the first pixel currently accessed.

A first aperture kernel corresponding to the first pixel is determined according to the blur radius of the first pixel and the aperture shape parameter.

The electronic device may pre-generate, store, or obtain multiple aperture kernels from other devices such as a service device, and different aperture kernels may be used to indicate different diffusion ranges. Different aperture cores may correspond to different blurring radii, each aperture core may correspond to an aperture shape, and the same aperture shape may correspond to multiple aperture cores with different blurring radii.

When the electronic device obtains the aperture shape parameter and the blur radius of the first pixel point, it can determine from the plurality of aperture kernels which corresponds to the blur radius of the first pixel point and corresponds to the aperture shape indicated by the aperture shape parameter. The first aperture kernel, the first aperture kernel may be used to indicate the diffusion range of the first pixel.

Optionally, the electronic device may pre-generate multiple aperture kernels according to the aperture image. Wherein, the electronic device can acquire multiple aperture images with different aperture shapes, and the above aperture shapes can be personalized graphics, heart shapes, star shapes, circles or rings, etc., but are not limited thereto.

The electronic device can zoom each aperture image in the plurality of aperture images multiple times, and each zoom can obtain an aperture kernel corresponding to a blur radius. Therefore, after the electronic device scales each aperture image multiple times, multiple aperture kernels corresponding to each aperture shape can be obtained, and each aperture kernel can correspond to a blurring radius. For example, the electronic device can sequentially scale the aperture image to a size of 3×3, 4×4...N×N.

440. Calculate the target pixel value of each second pixel within the diffusion range corresponding to the first pixel according to the pixel value of the first pixel in the first image, the diffusion weight corresponding to the first pixel, and the first aperture kernel.

The electronic device can multiply the pixel value of the first pixel in the first image by the diffusion weight corresponding to the first pixel, and then multiply it by the first aperture kernel, so as to obtain each second pixel within the diffusion range corresponding to the first pixel. The target pixel value of the pixel. That is, the pixel value of the first pixel is diffused into a diffusion range centered on the first pixel.

In some embodiments, the pixel value of the first pixel point may include a luma component value and a chrominance component value. Correspondingly, the weight map may also include a luminance weight map and a chromaticity weight map, the diffusion weight may include a luminance weight and a chromaticity weight, the luminance weight map may at least include the luminance weights of each first pixel in the background area, and the chromaticity weight map may include It may at least include the chromaticity weights of each first pixel in the background area.

When performing point diffusion processing on the first pixel, the electronic device may perform point diffusion processing on the luminance component value and the chrominance component value of the first pixel according to the corresponding diffusion weight, and fuse them again during superimposition.

Exemplarily, the luminance component value of the first pixel point may be the Y component value of the Y channel, and the chrominance component value may include a U component value of the U channel and a V component value of the V channel.

For each first pixel point, the electronic device can multiply the Y component value of the first pixel point by the brightness weight of the first pixel point, and then multiply by the first aperture kernel to obtain the first pixel point corresponding to the first pixel point in the diffusion range. The target Y component value of the two pixels in the Y channel.

The electronic device can multiply the U component value of the first pixel by the chromaticity weight of the first pixel, and then multiply by the first aperture kernel to obtain the U channel value of each second pixel in the diffusion range corresponding to the first pixel. Target U component value.

The electronic device can multiply the V component value of the first pixel by the chromaticity weight of the first pixel, and then multiply it by the first aperture kernel to obtain the V channel value of each second pixel in the diffusion range corresponding to the first pixel. Target V component value.

According to the diffusion weight corresponding to the first pixel and the first aperture kernel, the number of diffusions of each second pixel within the diffusion range corresponding to the first pixel is calculated.

The electronic device may multiply the diffusion weight corresponding to the first pixel by the first aperture kernel, so as to obtain the diffusion times of each second pixel within the diffusion range corresponding to the first pixel.

After executing steps 440 to 450, the electronic device can obtain the diffusion results corresponding to the first pixels, and superimpose the diffusion results corresponding to the first pixels in the background area to blur the background area.

In some embodiments, the electronic device may perform the following steps to superimpose the diffusion results corresponding to each first pixel in the background area.

460. Superimpose the target pixel values of each second pixel contained in the diffusion range corresponding to each first pixel in the background area to the pixel value overlay map, and add each target pixel value contained in the diffusion range corresponding to each first pixel in the background area The number of diffusion times of the second pixel is superimposed on the number statistics graph.

The pixel value superposition map can be used to record the point diffusion superposition result of pixel values, the size of the pixel value superposition map can be greater than or equal to the first image, and the initial value of the pixel value superposition map can be 0. The number of color channels of the pixel overlay image may be consistent with the number of color channels of the first image, and each channel may be used to record a point diffusion overlay result of component values of the same channel.

The number statistics map can be used to record the point diffusion superposition result of the diffusion times, the size of the times statistics map can be greater than or equal to the first image, and the initial value of the times statistics map can be 0.

Optionally, in order to preserve the point diffusion effect of the edge pixels of the first image, the edge range of the pixel value overlay map and the count statistics map can be expanded according to the maximum blur radius in the blur intensity map, so that the image size of the pixel value overlay map larger than the first image.

After each calculation of the diffusion result corresponding to the first pixel, the electronic device can superimpose the target pixel value of each second pixel contained in the diffusion result to the pixel value overlay map, and the target pixel value of each second pixel contained in the diffusion result The number of spreads of points is superimposed on the count graph. By analogy, until the diffusion results corresponding to each first pixel contained in the background area are superimposed on the pixel overlay map and the count statistics map.

According to the pixel value of each first pixel point in the background area in the superimposed pixel value superimposition map, and the total number of diffusions of each first pixel point in the background area in the superimposed number of times statistics map, for each first pixel point in the background area Pixels are blurred.

The pixel value of each first pixel included in the background area in the superimposed pixel value superposition map may be obtained by diffusion and superposition of the pixel values of one or more first pixels, and each first pixel is in The total number of diffusions in the statistical graph of the number of times after superposition may also be obtained by superimposing the number of diffusions of one or more first pixel points.

Therefore, for each first pixel, the electronic device may divide the pixel value obtained by superimposing the first pixel by the total number of diffusions of the first pixel to obtain a blurred rendering result of blurring the first pixel.

Exemplarily, blurring the first pixel can be calculated by the following formula:

Among them, Bokeh can represent the blurred pixel value of the first pixel, kernel can represent the first aperture kernel, weight can represent the diffusion weight corresponding to the first pixel, and input can represent the pixel of the first pixel in the first image Value, that is, the pixel value before blurring.

It can be seen that, in the foregoing embodiments, the background mask can be used to indicate the background area of the first image, so that the electronic device can perform point diffusion processing on each first pixel included in the background area, so that the bright spots in the background area form light spots, and the dark spots The dots form a blurring circle, which more realistically and naturally simulates the blur effect of a real lens. In addition, different aperture shapes can be simulated through different aperture kernels, so that blurred light spots can take on different shapes.

In the foregoing embodiments, the background mask can be used to indicate the background area of the first image, and can be used to indicate the range of pixels that need to be subjected to dot diffusion processing, so as to avoid blurring the foreground area of the first image by mistake.

In some embodiments, the electronic device may use the defocusing intensity map of the first image to generate the background mask.

Please refer to FIG. 5 . FIG. 5 is a schematic flowchart of a method for generating a background mask disclosed by an embodiment, and the method can be applied to the aforementioned electronic device. As shown in Figure 5, the method may include the following steps:

510. Remove the portrait region in the first image according to the blur strength map of the first image.

The foreground area of the first image may include a portrait area, and the portrait area generally does not undergo blurring processing, so the area where the pixels with a blurring radius of 0 in the blurring strength map are located corresponds to the portrait area. The electronic device removes pixels with a blur radius of 0 in the first image, so that the portrait area can be removed in the first image.

The blur strength map may be generated according to the depth information of each pixel in the full image of the first image. Pixels with smaller depths can be considered to belong to the portrait area, so the blur radius corresponding to pixels with smaller depths can be set to 0. Therefore, the blur intensity map of the first image can help identify the portrait area to a certain extent, and the electronic device can remove pixels with a blur radius of 0 in the first image to remove the portrait area.

Optionally, before removing the portrait area in the first image, the electronic device may first expand the portrait area in the blur intensity map of the first image, including: eroding the blur intensity map to expand the range of the portrait area. Wherein, the erosion radius corresponding to each pixel in the blur strength map may be positively correlated with the blur radius corresponding to each pixel. That is, if the blur radius corresponding to a pixel in the blur intensity map is larger, the erosion radius corresponding to the pixel is larger.

Compared with the blur intensity map before erosion, the number of pixels with a blur radius of 0 may increase in the blur intensity map after erosion. The electronic device may remove the portrait region in the first image according to the expanded blur strength map of the portrait region. The portrait area is removed from the first image.

520. Remove the hair region in the first image from which the portrait region has been removed according to the hair mask, so as to obtain a background mask of the first image.

The foreground region of the first image may also include a hair region. The electronic device may perform hair recognition on the first image, for example, by using a deep learning method to identify the confidence level that each pixel in the first image belongs to hair, so as to generate a hair mask.

The hair mask can be used to indicate the hair region in the first image, the hair mask can include the confidence that each pixel in the first image is recognized as hair, and the pixels with the confidence higher than the confidence threshold can be used to indicate the first image in the hair area. The electronic device may subtract the hair region indicated by the hair mask from the first image from which the portrait region has been removed, so as to remove the hair region and obtain the background mask of the first image.

Optionally, although methods such as deep learning can identify the hair region in the first image, certain errors may exist. In order to reduce the impact of hair recognition errors, the electronic device may first expand the hair mask of the first image before removing the hair region. The electronic device can first count the area of the outer contour of the hair in the hair mask, calculate the expansion radius according to the area ratio of the outer contour of the hair, and then increase the expansion radius according to the blurring radius of the pixel points of the outer hair outline. Wherein, the area of the outer contour of the hair region may refer to the number of pixels included in the outer contour of the hair region, and the increase of the blur radius and the expansion radius may be positively correlated. That is, if the virtualization radius corresponding to a pixel in the hair mask is larger, the expansion radius corresponding to the pixel is increased more.

The electronic device can expand the hair mask according to the expansion radius of the pixels to expand the hair area. In addition, the electronic device can further increase the confidence of the pixels in the hair mask, so as to increase the number of pixels identified as hair, which can further expand the hair area.

The electronic device can subtract the hair region expanded by the hair mask from the first image from which the portrait region has been removed, so as to obtain the background mask of the first image, so as to reduce the false blur caused by the hair strands. The problem of color bleeding (Color Bleeding) caused by halos forming color bands.

In some embodiments, before generating the deblurring strength map of the first image, the electronic device may also perform blemish removal (Inpaint) filling on the portrait area and hair outline area of the first image. That is, the pixels of the background area adjacent to the portrait area and the hair outline area can be used to fill the portrait area and the hair outline area, so as to delete the light spots on the edges of the portrait and hair. After filling the first image with Inpaint, generate a blur intensity map, and generate a background mask according to the blur intensity map, which can not only optimize the problem of color leakage, but also make the processing of the first image conform to the gradient direction of the original image , texture features, color features, etc., which are beneficial to make the blurring effect on the first image more realistic and natural.

It can be seen that, in the foregoing embodiments, the electronic device can process the portrait area and the hair area of the first image to more accurately identify the background area of the first image, thereby reducing false positives of pixels on the edge of the portrait and the hair edge. The problem of color leakage caused by blurring makes the blurring effect of the first image more realistic and natural.

In addition, in the description of the foregoing embodiments, the dot diffusion processing can diffuse and blur brighter bright spots in the first image into light spots. If the overall brightness of the background area of the first image is high, it may cause the light spots in the blur effect to spread too much, and instead cause the blur effect to be unnatural and unreal.

Therefore, in some embodiments, the electronic device may perform preprocessing on the first image, including: the electronic device may reduce brightness component values of each first pixel in the background area of the first image. Wherein, the reduction amount of the brightness component value of the first pixel point may have a positive correlation with the brightness of the shooting scene of the first image. That is, the brighter the shooting scene of the first image is, the more the brightness component value of the first pixel is depressed.

For example, please refer to FIG. 6 . FIG. 6 is a schematic flowchart of a method for darkening the brightness of a background area disclosed by an embodiment, and the method can be applied to the aforementioned electronic device. As shown in Figure 6, the following steps may be included:

610. Calculate a gamma (Gamma) parameter used for brightness suppression.

The Gamma parameter can be used for Gamma correction, and the Gamma parameter can be calculated by the following formula:

Wherein, Gamma_1 may be a Gamma parameter for brightness suppression, Contrast may be an input contrast parameter, and iso may be a sensitivity parameter of an imaging device that captures the first image.

620. Perform curve mapping and stretching on the luminance component values of each first pixel point based on the calculated Gamma parameter, so as to reduce the luminance component values of the first pixel points. Wherein, the stretched luminance component values of each first pixel in the background area of the first image can be calculated with reference to the following formula:

Wherein, y1 may represent the brightness component value after stretching, x1 may represent the brightness component value before stretching, and the brightness component value before stretching may refer to the brightness component value of the first pixel in the first image.

Through the aforementioned formula (2) and formula (3), the electronic device can reduce the brightness component value of each first pixel in the background area of the first image, and the brighter the shooting scene is, the more the brightness component value is reduced. In this way, the background can be suppressed and the light source can be highlighted, which can improve the contrast.

In some embodiments, the electronic device preprocessing the first image may further include: the electronic device may increase the chrominance component values of each first pixel contained in the highlight area in the background area of the first image, so as to process the highlight area Color Enhancement, which enhances the color in the center of the light source. Wherein, the luminance component values of each first pixel contained in the highlight area are all higher than a luminance threshold, and the luminance threshold can be set according to actual service requirements, and is not specifically limited. That is, the highlight area may be a relatively bright image area in the background area. In addition, the increase amount of the chrominance component of the first pixel contained in the highlight area may have a negative correlation with the brightness of the shooting scene of the first image. That is, the darker the shooting scene of the first image is, the more the chrominance component value of the first pixel contained in the highlight area increases.

For example, please refer to FIG. 7 . FIG. 7 is a schematic flowchart of a method for color enhancement of a highlight area disclosed by an embodiment, and the method can be applied to the aforementioned electronic device. As shown in Figure 7, the following steps may be included:

710. Blur the chrominance component values of the first image.

The chrominance component value of the first image may include a U component value and a V component value, and the electronic device may respectively perform image blurring on the U component value of the entire first image, and perform image blurring on the V component value of the entire first image . Wherein, the way of image blurring may include but not limited to: N×N Block mean blurring, median blurring, etc.

Generally speaking, because the energy of the center of the light source is relatively large, it may exceed the upper limit of the load capacity of the sensor of the imaging device, resulting in the center of the light source having a weaker color in the captured first image. For blurring the chrominance component values of the first image, the chrominance component values in the neighborhood can be used to reinforce the chrominance component values in the weakly colored areas, so as to compensate for the problem caused by the sensor's carrying capacity.

720. Calculate a Gamma parameter for color enhancement according to the saturation parameter, and use the calculated Gamma parameter to perform color enhancement on the entire image of the first image to obtain a color-enhanced first image.

The electronic device may use the Gamma parameter for color enhancement to perform curve mapping and stretching on the chrominance component values of the full image of the first image, so as to perform color enhancement on the entire first image.

The Gamma parameter for color enhancement can be calculated by the following formula:

Wherein, Gamma_2 may be a Gamma parameter for color enhancement, Saturation may be an input saturation parameter, and iso may be a sensitivity parameter of an imaging device that captures the first image.

The curve mapping and stretching of the chrominance component values of the full image of the first image can be calculated by the following formula:

Wherein, i may be the chroma component value, the LUT may be the chroma component value after stretching, and the chroma component value before stretching may refer to the chroma component value of the first pixel in the first image.

In addition, after the electronic device performs curve mapping and stretching on the chrominance component values of the full image of the first image, it needs to backfill the uncolored area with the chroma component value=128 before stretching. That is, the chromaticity components of the colorless region with the chromaticity component before stretching = 128 can be kept consistent before and after stretching.

730. Calculate a highlight mask of the first image.

The highlight mask of the first image may be used to indicate highlight areas in the first image, which may include highlight areas in background areas. The electronic device may stretch and map the brightness component values of the full image of the first image to obtain a highlight mask of the first image, and the highlight mask may include stretched brightness components of each pixel in the first image. Stretch mapping of luma component values can be calculated by the following formula:

Wherein, y2 may represent the brightness component value after stretching, x2 may represent the brightness component value before stretching, and the brightness component value before stretching may refer to the brightness component value of the first pixel in the first image.

It should be noted that, from the above formula (6), it can be seen that if the value of the brightness component before stretching is low, the value of the brightness component after stretching may be 0, and the value of the brightness component before stretching is higher, Then the value of the brightness component after stretching is higher. Therefore, the highlight mask obtained by stretching the brightness component value can be used to indicate the highlight region where the brightness component value of the pixel point in the first image is higher than the brightness threshold.

740. Perform color enhancement on the highlight area of the first image according to the color-enhanced first image and the highlight mask of the first image.

The electronic device can fuse the highlight mask and the color-enhanced first image, at least to increase the chromaticity component value of each first pixel contained in the highlight area in the background area, so that the color of the highlight area can be enhanced, and the center of the light source can be improved. color.

It can be seen that, in the foregoing embodiments, the electronic device can perform preprocessing on the first image, correct the brightness component value of the first image to improve the overall contrast of the image, and correct the chrominance component value of the first image to improve the light source. Saturation at this point, while blurring the first image, it can avoid too much spread of the light spot as much as possible, and enhance the color of the center of the light source, making the blur effect more natural and real.

In the foregoing embodiments, the electronic device may perform point diffusion processing on the background area of the first image according to the shape of the aperture, and different diffusion weights and blur radii, so that the blur effect on the first image can simulate real lens blur. The diffusion weight is one of the important influencing factors of the point diffusion processing, and the following content introduces the implementation manner of obtaining the weight map by the electronic device.

In an embodiment, the electronic device may calculate brightness weights corresponding to each first pixel in the background area according to brightness component values of each first pixel in the background area of the first image, so as to obtain a brightness weight map.

The brightness weight map may include brightness weights corresponding to each first pixel in the background area, and the brightness weight corresponding to each first pixel may be positively correlated with the brightness component value of each first pixel. That is, the greater the brightness component value of the first pixel, the greater the brightness weight corresponding to the first pixel.

The electronic device may perform one or more types of processing on the brightness component values of each first pixel in the background area, and use the processed brightness component values as brightness weights corresponding to each first pixel.

In an embodiment, the electronic device may calculate the chromaticity weight corresponding to each first pixel in the background area according to the luminance component value and the chromaticity component value of each first pixel in the background area of the first image.

The chromaticity weight map may include chromaticity weights corresponding to each first pixel in the background area, the chromaticity weights corresponding to each first pixel may be positively correlated with the luminance component values of each first pixel, and each first pixel The chrominance weights corresponding to the points may also have a positive correlation with the chrominance component values of each first pixel point. That is, the larger the luminance component value and the chrominance component value of the first pixel point are, the larger the chrominance weight corresponding to the first pixel point is.

In some embodiments, the electronic device may directly superimpose the chroma component value and the luminance component value of each first pixel point in the background area, and use the superimposed component value as a chroma weight corresponding to each first pixel point. Alternatively, the electronic device firstly performs one or more processes on the chroma component values and luminance component values of each first pixel, and superimposes the processed chrominance component values and/or luminance component values to obtain The value of the superimposed component of is used as the chromaticity weight corresponding to each first pixel point.

Exemplarily, if the first image includes three color channels of Y, U, and V, the electronic device may calculate the brightness weight corresponding to each first pixel according to the Y component value of each first pixel in the Y channel in the background area; And, the electronic device can also calculate the color corresponding to each first pixel according to the Y component value of each first pixel in the background area, and the U component value of each first pixel in the U channel and the V component value in the V channel. degree weight.

Exemplarily, the electronic device may also acquire a second image whose shooting content is the same as that of the first image, and the exposure value corresponding to the second image is lower than that of the first image. That is, the second image may be a dark frame of the first image. The electronic device may acquire the pixel value of the first pixel in the second image as the brightness component value of the first pixel. The overall energy of the second image is low, which can directly reflect the energy distribution law in the shooting scene. The pixel value in the second image is used as the brightness component value to generate the diffusion weight, which is beneficial to optimize the diffusion result of point diffusion processing, making the virtual The effect is more natural and realistic.

The luminance weight map and the chrominance weight map are introduced separately below.

Please refer to FIG. 8 . FIG. 8 is a schematic flowchart of a method for obtaining a brightness weight map disclosed by an embodiment, and the method can be applied to the aforementioned electronic device. As shown in Figure 8, the method may include the following steps:

810. Set, in the background area of the first image, the brightness weight corresponding to the first pixel whose brightness component value is smaller than the brightness threshold to zero.

820. Calculate brightness weights corresponding to the first pixels included in the highlight area according to the area of the highlight area included in the background area of the first image.

Wherein, the luminance component values of the first pixels included in the highlight area are all greater than or equal to the luminance threshold, and the luminance threshold can be set according to actual service requirements, and is not specifically limited. The area of the highlight area may be represented by the number of first pixels included in the highlight area.

Optionally, if the area of the highlight area is smaller than the area threshold, the electronic device may set the brightness weights corresponding to each first pixel contained in the highlight area to zero.

Optionally, if the area of the highlight area is greater than or equal to the area threshold, the electronic device may reduce the brightness component value of each first pixel contained in the highlight area according to the area of the highlight area, and use the reduced brightness component value as the highlight Brightness weights corresponding to the respective first pixels included in the area.

Wherein, the area threshold can be set according to actual business requirements, and is not specifically limited. The reduction amount of the luminance component value of each first pixel contained in the highlight area is positively correlated with the area of the highlight area. That is, the larger the area of the highlight area, the more the luminance component value of the first pixel in the highlight area is deleted (reduced). By reducing the brightness component value of the large-area highlight area, the energy of the dense light source area in the first image can be deleted, so as to avoid the problem of overexposure caused by spot stacking after point diffusion superposition.

In some embodiments, the background area of the first image may include two or more highlight areas that are not connected to each other. Since the reduction of the luminance component value is associated with the area of the highlight area, the electronic device can first expand each highlight area before performing the aforementioned step 920, so as to connect two adjacent highlight areas, so that the two adjacent highlight areas Regions can be connected into a highlight region. After dilating each highlight area, the electronic device can count the area of each highlight area, and perform the aforementioned step 820 to calculate the brightness weight corresponding to each first pixel contained in the highlight area.

It can be seen that, in the above-mentioned embodiments, the electronic device can reduce the energy of the dense light source area to avoid stacking and overexposure of the light spots.

Please refer to FIG. 9 . FIG. 9 is a schematic flowchart of a method for obtaining a chromaticity weight map disclosed by an embodiment, and the method can be applied to the aforementioned electronic device. As shown in Figure 9, the method may include the following steps:

910. Calculate the chroma value of each first pixel point according to the chroma component value of each first pixel point in the background area.

If the chrominance component value includes the U component value of the U channel and the V component value of the V channel, when the electronic device superimposes the luminance component value and the chrominance component value of the first pixel, it can first calculate the The value computes the chroma value.

Exemplarily, the chromaticity value can be calculated by the following formula:

Wherein, chroma may be a chroma value, u may be a U component value, and v may be a V component value.

920. Superimpose the luminance component value of each first pixel point in the background area and the chrominance component value of each first pixel point in the background area to obtain component superposition values of each first pixel point in the background area.

The electronic device may superimpose the chrominance value and the brightness component value of each first pixel point in the background area, so as to obtain component superposition values of each first pixel point.

Optionally, before the electronic device superimposes the luminance component value of each first pixel in the background area and the chroma value of each first pixel in the background area, it may also first calculate the chroma value Do Chroma Stretch. After performing chromaticity stretching, the electronic device may superimpose the luminance component value of each first pixel point in the background area with the stretched chromaticity value, so as to obtain component superposition values of each first pixel point in the background area. Wherein, the chromaticity stretch variation amount of the chromaticity value of any first pixel may be positively correlated with the luminance component value of the first pixel. That is, the brighter the first pixel is, the more the chromaticity is stretched.

Exemplarily, the electronic device can superimpose the luminance component value of each first pixel point in the background area with the stretched chromaticity value, which can be calculated by the following formula:

可表示拉伸后的色度值。Among them, y3 can represent the component superposition value after superimposing the luminance component value and stretching the chroma value, and x3 can represent the luminance component value,

It can represent the chromaticity value after stretching.

930. Determine the chromaticity weight corresponding to each first pixel in the background area according to the component superposition value of each first pixel in the background area.

Optionally, the electronic device may directly determine the component superposition value of each first pixel point in the background area as the chromaticity weight corresponding to each first pixel point in the background area.

Optionally, the electronic device performing step 920 may also include the following steps:

The electronic device performs brightness stretching on the brightness component values of each first pixel point in the background area to obtain the stretched brightness component value of each first pixel point in the background area.

Wherein, the brightness stretching variation of each first pixel in the background area is positively correlated with the brightness component value of each first pixel in the background area before stretching. That is to say, the first pixel in the background area before stretching

Exemplarily, the electronic device may perform luminance stretching on the luminance component values of each first pixel in the background area by using the aforementioned formula (6). That is to say, after stretching the brightness of each first pixel, the electronic device can obtain a highlight mask of the first image, and the highlight mask can at least include the stretched brightness component value of each first pixel in the background area , which can be used to indicate highlight areas in background areas.

For each first pixel in the background area, the electronic device can compare the component superposition value of the first pixel with the stretched luminance component value of the first pixel, and determine the maximum value obtained from the comparison as the first pixel. The chroma weight corresponding to a pixel.

The electronic device determines the maximum value of the comparison as the chromaticity weight corresponding to the first pixel point, which can enhance the chromaticity weight corresponding to the first pixel point with higher chromaticity and/or higher brightness, and yes, the light spot can spread more. Multi-color, enhances the spot color in the bokeh effect.

It can be seen that in the foregoing embodiments, the first pixel with a higher chroma component value corresponds to a higher chroma weight, and the chroma weight corresponding to the first pixel in the highlight area is also higher, so that the brightness can be made higher The light spots diffuse more colors, which is beneficial to enhance the color of the light spots in the blur effect.

In some embodiments, after the electronic device obtains the diffusion weights corresponding to each first image pixel point in the background area from the weight map, and before performing point diffusion processing on the first pixel points using the corresponding diffusion weights, the electronic device may also Firstly, the diffusion weight corresponding to the first pixel point is mapped according to the weight mapping curve to obtain the diffusion weight after the mapping of the first pixel point.

The electronic device may perform point diffusion processing on the first pixel by using the diffusion weight after the mapping of the first pixel. That is to say, the electronic device can calculate the value of each second pixel within the diffusion range corresponding to the first pixel according to the pixel value of the first pixel in the first image, the diffusion weight after the mapping of the first pixel, and the first aperture kernel. The target pixel value of . And, the electronic device may calculate the diffusion times of each pixel within the diffusion range corresponding to the first pixel according to the diffusion weight after the mapping of the first pixel and the first aperture kernel.

If the diffusion weights include luminance weights and chrominance weights, the electronic device may map the luminance weights and chrominance weights respectively according to the weight mapping curve.

The above weight mapping curve can be obtained by fusing the first curve and the second curve. Wherein, the first curve may be a global smooth curve, and the second curve may be a local smooth curve stretched around the first brightness level. The first brightness level can be set according to actual service requirements, for example, it can be any value among the brightness levels of 90%-100%.

Exemplarily, the first curve can be represented by the following formula:

Wherein, y4 may represent the diffusion weight after mapping, and x4 may represent the diffusion weight before mapping.

It can be seen from formula (9) that the first curve is a curve with a global smooth transition between 0-255.

Exemplarily, the second curve can be represented by the following formula:

Wherein, y5 may represent the diffusion weight after mapping, and x5 may represent the diffusion weight before mapping.

It can be seen from formula (10) that the second curve is a locally smooth sigmoid curve centered at 95%.

The electronic device may fuse the first curve and the second curve to obtain a weight mapping curve.

Exemplarily, the weight mapping curve can be expressed by the following formula:

y6=α·y4+(1-α)·y5,α∈[0.2,0.4]; Formula (11)

Wherein, y6 may represent the weight mapping curve, y4 may represent the first curve shown in formula (9), and y5 may represent the second curve shown in formula (10).

Please refer to FIG. 10 . FIG. 10 is an example diagram of a first curve, a second curve and a weight mapping curve disclosed by an embodiment. As shown in Figure 10, the first curve 1010 is globally smooth, the second curve 1020 is locally smooth, and the weight mapping curve 1030 combines the characteristics of the first curve 1010 and the second curve 1020, which conforms to the energy distribution of the light source and has a natural transition. The brightness levels can be staggered to form a brightness ladder, so that the diffusion weights corresponding to the first pixels with brightness component values at different brightness levels have obvious differences, which is conducive to optimizing the overall layered transparency of the spot in the blur effect.

In some embodiments, the blurring effect needs to further brighten the light spots, and it may be necessary to further increase the diffusion weight corresponding to the highlighted pixels. The electronic device can optimize the weight mapping curve, and increase the weight of the second half of the weight mapping curve.

Exemplarily, the optimized weight mapping curve can be expressed by the following formula:

y7＝y7·(L-(255-x)), x∈(255-L, 255]; formula (13)

Among them, y7 can represent the optimized weight mapping curve, Brightness can represent the input brightness parameter, which can be set according to actual business requirements, and ApertureSize can represent the blur radius of the first pixel.

Using the optimized weight mapping curve for diffusion weight mapping is beneficial to further brighten the light spots in the blur effect and improve the overall layered transparency of the light spots.

Please refer to FIG. 11 . FIG. 11 is a schematic flowchart of an image processing method disclosed by an embodiment, which can be applied to the aforementioned electronic device. As shown in Figure 11, the method may include the following steps:

1110. Generate a blur intensity map corresponding to the first image, and generate a background mask of the first image according to the blur intensity map.

The electronic device can generate a blur intensity map corresponding to the first image according to the depth information of the full image of the first image, and can generate a background mask by using the blur intensity map corresponding to the first image.

1120. Perform preprocessing on the first image.

The electronic device preprocessing the first image may include: reducing the brightness component value of each first pixel in the background area of the first image; and increasing the color of each first pixel contained in the highlight area in the background area of the first image. Degrees component value to apply color enhancement to highlight areas.

A weight map corresponding to the first image is generated.

The electronic device may calculate the diffusion weight of each pixel according to the luminance component value and the chrominance component value of each pixel in the full image of the first image, so as to generate a weight map corresponding to the first image.

Wherein, the weight map may include a luma weight map and a chroma weight map. The luminance weight map can be calculated based on the luminance component values of each pixel in the full first image, and the chrominance weight map can be calculated based on the luminance component values and chrominance component values of each pixel in the full first image.

When generating the brightness weight map, the electronic device may delete the energy of the large-area high-light area, thereby reducing the brightness weight of the first pixel in the large-area high-light area.

When generating the chromaticity weight map, the electronic device may increase the chromaticity weight corresponding to the first pixel with higher luminance component value and/or chrominance component value.

Each first pixel in the background area of the first image indicated by the background mask is traversed, and the diffusion weight and blur radius corresponding to the first pixel are obtained from the weight map and the blur strength map corresponding to the first image.

1150. Map the diffusion weight corresponding to the first pixel point according to the weight mapping curve to obtain the diffusion weight after the mapping of the first pixel point.

1160. According to the aperture shape parameter, the blur intensity map and the weight map corresponding to the first image, perform point diffusion processing on each first pixel point in the background area of the first image indicated by the background mask, and obtain each pixel in the background area. Diffusion result corresponding to the first pixel.

1170. Superimpose the diffusion results corresponding to each first pixel in the background area, and blur the background area according to the superposition results.

It can be seen that, in the foregoing embodiments, the electronic device can generate a background mask to indicate the range of the background area that needs to be subjected to dot diffusion processing, so that the dot diffusion processing can be performed on each first pixel contained in the background area, so that the bright spots are formed Highlighted light spots, dark spots form hazy diffuse circles. The electronic device can also preprocess the first image before dot diffusion processing, reduce the brightness of the background area to improve the overall contrast of the first image, and at the same time enhance the color of the highlight area to increase the saturation of the light source. Diffusion weight is an important factor affecting point diffusion processing. When generating the brightness weight map, the electronic device deletes the energy of the large-area highlight area and reduces the brightness weight of the first pixel in the large-area highlight area, which can avoid the overexposure of the spot stack. ; When generating the chromaticity weight map, the electronic device may increase the chromaticity weight corresponding to the first pixel with higher luminance component value and/or chrominance component value, so that the light spot can spread more colors. The electronic device can further map the diffusion weight obtained from the weight map through a weight mapping curve that conforms to the energy distribution of the light source and has a natural transition, so as to optimize the overall layered transparency of the light spot in the blur effect. The electronic device can also simulate the aperture shape of different lenses according to different aperture shape parameters, so that the shape of the light spot in the blur effect is more variable.

Please refer to FIG. 12 . FIG. 12 is a schematic structural diagram of an image processing device disclosed in an embodiment. The image processing apparatus shown in FIG. 12 may be applied to any of the aforementioned electronic devices. As shown in FIG. 12 , the image processing apparatus 1200 may include: an acquisition module 1210 , a diffusion module 1220 , and a blurring module 1230 .

The acquiring module 1210 is configured to acquire aperture shape parameters, and acquire a blurring intensity map and a weighting map corresponding to the first image, where the blurring intensity map at least includes blurring radii corresponding to respective first pixels in the background area of the first image , the weight map at least includes diffusion weights respectively corresponding to each first pixel in the background area of the first image;

The diffusion module 1220 is configured to perform point diffusion processing on each first pixel point in the background area of the first image according to the aperture shape parameter, the blur intensity map and the weight map, to obtain a diffusion result corresponding to each first pixel point;

The overlay blur module 1230 is configured to overlay the diffusion results corresponding to each first pixel in the background area, and blur the background area according to the overlay results.

In one embodiment, the diffusion result corresponding to each first pixel point in the background area includes: the diffusion range corresponding to the first pixel point, the target pixel value of each second pixel point within the diffusion range, and the target pixel value of each second pixel point within the diffusion range Diffusion times of each second pixel within ; Wherein, the diffusion range corresponding to the first pixel is centered on the first pixel, the radius of the circumscribed circle of the diffusion range is the blur radius corresponding to the first pixel, and the diffusion range The outer contour is the shape of the aperture indicated by the aperture shape parameter.

In one embodiment, the diffusion module 1220 may include: a first acquisition unit, a first diffusion unit, and a second diffusion unit.

The first obtaining unit is configured to obtain, for each first pixel in the background area of the first image, the blur radius corresponding to the first pixel from the blur strength map, and obtain the blur radius corresponding to the first pixel from the weight map. The diffusion weight corresponding to the pixel point; and, according to the blur radius corresponding to the first pixel point and the aperture shape parameter, determine the first aperture kernel corresponding to the first pixel point, and the first aperture kernel is used to indicate the diffusion range of the first pixel point ;

The first diffusion unit can be used to calculate the value of each second pixel in the diffusion range corresponding to the first pixel according to the pixel value of the first pixel in the first image, the diffusion weight corresponding to the first pixel, and the first aperture kernel. target pixel value;

The second diffusion unit is configured to calculate the number of diffusion times of each second pixel within the diffusion range corresponding to the first pixel according to the diffusion weight corresponding to the first pixel and the first aperture kernel.

In one embodiment, the overlay blur module 1230 may include: an overlay unit and a blur unit.

The superposition unit can be used to superimpose the target pixel value of each second pixel contained in the diffusion range corresponding to each first pixel in the background area to the pixel value superimposition map, and the diffusion range corresponding to each first pixel in the background area The number of diffusions of each included second pixel point is superimposed on the number statistics graph;

The blurring unit can be used to calculate each first pixel according to the pixel value of each first pixel in the superimposed pixel value superposition map in the background area, and the total number of diffusions of each first pixel in the superimposed times statistics map. Pixels are blurred.

In one embodiment, the image processing apparatus 1200 may further include: a background determination module.

The background determination module can be used to remove the portrait area in the first image according to the blur intensity map of the first image; and remove the hair area in the first image from which the portrait area has been removed according to the hair mask, so as to obtain the first A background mask for an image.

Optionally, the background determination module may be further configured to erode the blur intensity map before removing the portrait area in the first image, so as to expand the range of the portrait area. Wherein, the erosion radius corresponding to each pixel in the blur strength map may be positively correlated with the blur radius corresponding to each pixel. And, it can also be used to remove the portrait area in the first image according to the blur strength map after the expansion of the portrait area.

Optionally, the background determination module can also be used to count the area of the hair outer contour in the hair mask before removing the hair region, calculate the expansion radius according to the area ratio of the hair outer contour, and then calculate the expansion radius according to the pixel points of the hair outer contour The blur radius increases the expansion radius; and, it can also be used to expand the hair mask according to the expansion radius to expand the hair area; and, it can also be used to increase the confidence of the pixels in the hair mask to increase the recognition as hair The number of pixels to further expand the hair area. Wherein, the area of the outer contour of the hair region may refer to the number of pixels included in the outer contour of the hair region, and the increase of the blur radius and the expansion radius may be positively correlated.

The background determination module can also be used in the first image from which the portrait area is removed, to subtract the expanded hair area of the hair mask to obtain the background mask of the first image.

In one embodiment, the image processing apparatus 1200 may further include: a preprocessing module.

The preprocessing module can be used to reduce the brightness component value of each first pixel in the background area of the first image. Wherein, the reduction amount of the brightness component value of the first pixel point may have a positive correlation with the brightness of the shooting scene of the first image.

Exemplarily, the preprocessing module can be used to calculate the Gamma parameter for brightness reduction, and perform curve mapping and stretching on the brightness component value of each first pixel point based on the calculated Gamma parameter, so as to reduce the brightness of the first pixel point component value.

The preprocessing module can also be used to increase the chromaticity component value of each first pixel contained in the highlight area in the background area of the first image, so as to enhance the color of the highlight area and improve the color of the center of the light source. Wherein, the increase amount of the chrominance component of the first pixel contained in the highlight area may have a negative correlation with the brightness of the first image shooting scene.

Exemplarily, the preprocessing module may be used to blur the chrominance component value of the first image; and, calculate a Gamma parameter for color enhancement according to the saturation parameter, and use the calculated Gamma parameter parameter to fully optimize the first image The image is color enhanced to obtain the first image after color enhancement; and, calculate the highlight mask of the first image; and, according to the first image after color enhancement and the highlight mask of the first image, the highlight of the first image is calculated Areas are color enhanced.

In one embodiment, the acquiring module 1210 may include: a luma weight generating unit and a chroma weight generating unit.

The luminance weight generation unit is configured to calculate the luminance weight corresponding to each first pixel in the background area of the first image according to the luminance component value of each first pixel in the background area of the first image, so as to obtain a luminance weight map. Wherein, the brightness weight map may include brightness weights corresponding to each first pixel point in the background area, and the brightness weight corresponding to each first pixel point may be positively correlated with the brightness component value of each first pixel point.

The chromaticity weight generating unit may be configured to calculate the chromaticity weight corresponding to each first pixel in the background area of the first image according to the luminance component value and the chrominance component value of each first pixel in the background area of the first image. Wherein, the chromaticity weight map may include chromaticity weights corresponding to each first pixel point in the background area, the chromaticity weights corresponding to each first pixel point may have a positive correlation with the luminance component value of each first pixel point, and each first pixel point The chroma weight corresponding to a pixel may also be positively correlated with the chroma component values of each first pixel.

In one embodiment, the luminance weight generation unit is further configured to set the luminance weight corresponding to the first pixel point whose luminance component value is smaller than the luminance threshold in the background area of the first image to zero; and, according to the background area of the first image The included highlight area area, calculate the brightness weight corresponding to the first pixel contained in the highlight area.

Optionally, the brightness weight generating unit can also be used to set the brightness weight corresponding to each first pixel contained in the highlight region to zero when the area of the highlight region is smaller than the area threshold; and/or, when the area of the highlight region is larger than or equal to the area threshold, according to the area of the highlight area, reduce the brightness component value of each first pixel contained in the highlight area, and use the reduced brightness component value as the brightness weight corresponding to each first pixel contained in the highlight area . Wherein, the reduction amount of the luminance component value of each first pixel contained in the highlight area is positively correlated with the area of the highlight area.

In one embodiment, the luminance weight generating unit is further configured to firstly expand each highlight region to connect adjacent highlight regions when the background region may include two or more highlight regions that are not connected to each other.

In one embodiment, the chromaticity weight generating unit is further configured to calculate the chromaticity value of each first pixel point in the background area according to the chromaticity component value of each first pixel point in the background area; and, for each first pixel in the background area The brightness component value of the point is superimposed with the chrominance component value of each first pixel point in the background area to obtain the component superposition value of each first pixel point in the background area; and, according to the component of each first pixel point in the background area The superposition value determines the chromaticity weight corresponding to each first pixel point in the background area.

Optionally, the chromaticity weight generating unit may also be used to firstly calculate The chromaticity value of the point is stretched; and, after the chromaticity stretching, the luminance component value of each first pixel point in the background area is superimposed with the stretched chromaticity value to obtain the chromaticity value in the background area The component superposition value of each first pixel point.

Wherein, the chromaticity stretch variation amount of the chromaticity value of any first pixel may be positively correlated with the luminance component value of the first pixel.

Optionally, the chromaticity weight generating unit may also be used to perform brightness stretching on the brightness component values of each first pixel point in the background area to obtain the stretched brightness component values of each first pixel point in the background area; and, The component superposition value of the first pixel is compared with the stretched luminance component value of the first pixel, and the maximum value obtained from the comparison is determined as the chromaticity weight corresponding to the first pixel.

Wherein, the brightness stretching variation of each first pixel in the background area is positively correlated with the brightness component value of each first pixel in the background area before stretching.

In one embodiment, the image processing apparatus 1200 may further include: a weight mapping module.

The weight mapping module may be configured to map the diffusion weight corresponding to the first pixel point according to the weight mapping curve to obtain the diffusion weight after the mapping of the first pixel point. Wherein, the weight mapping curve can be obtained by fusing the first curve and the second curve, the first curve can be a global smooth curve, and the second curve can be a local smooth curve stretched around the first brightness level.

Correspondingly, the first diffusion unit included in the diffusion module 1220 can also be used to calculate the first pixel corresponding to The target pixel value of each second pixel point within the diffusion range of .

The second diffusion unit included in the diffusion module 1220 can also be used to calculate the diffusion times of each pixel within the diffusion range corresponding to the first pixel according to the diffusion weight after the mapping of the first pixel and the first aperture kernel.

It can be seen that, implementing the image processing device in the foregoing embodiments, the dot diffusion process can be performed on each first pixel in the background area, so that the bright spots form bright spots and the dark spots form hazy circles of confusion. And, the first image is preprocessed before the point diffusion processing, the brightness of the background area is reduced to improve the overall contrast of the first image, and the color of the highlight area is enhanced to increase the saturation of the light source. And, the energy of the large-area highlight area can also be deleted, and the brightness weight of the first pixel in the large-area highlight area can be reduced, which can avoid stacking and overexposure of the spots, and increase the value of the first pixel with a higher brightness component value and/or chrominance component value. The chromaticity weight corresponding to a pixel makes the light spot diffuse more colors. Moreover, the diffusion weight obtained from the weight map can also be mapped by using a weight mapping curve that conforms to the energy distribution of the light source and has a natural transition, so that the overall layered transparency of the light spot in the blur effect can be optimized. And, the aperture shape of different lenses can be simulated according to different aperture shape parameters, making the shape of the spot in the blur effect more variable. In summary, the image processing device can make the blurring effect more realistic and natural.

Please refer to FIG. 13 . FIG. 13 is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application. As shown in Figure 13, the electronic device 1300 may include:

a memory 1310 storing executable program code;

processor 1320 coupled to memory 1310;

Wherein, the processor 1320 invokes the executable program code stored in the memory 1310 to execute any image processing method disclosed in the embodiments of the present application.

It should be noted that the mobile terminal shown in FIG. 13 may also include components not shown, such as power supply, input buttons, camera, speaker, screen, RF circuit, Wi-Fi module, Bluetooth module, sensor, etc., which will not be described in detail in this embodiment.

The embodiment of the present application discloses a computer-readable storage medium, which stores a computer program, wherein, when the computer program is executed by a processor, any image processing method disclosed in the embodiment of the present application is implemented.

The embodiment of the present application discloses a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause the computer to execute any image disclosed in the embodiment of the present application Approach.

It should be understood that reference throughout the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present application. Thus, appearances of "in one embodiment" or "in an embodiment" in various places throughout the specification are not necessarily referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by this application.

In various embodiments of the present application, it should be understood that the sequence numbers of the above-mentioned processes do not necessarily mean the order of execution. The implementation of the examples constitutes no limitation.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, located in one place, or distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the above-mentioned integrated units are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a memory , including several requests to make a computer device (which may be a personal computer, server, or network device, etc., specifically, a processor in the computer device) execute some or all of the steps of the above-mentioned methods in various embodiments of the present application.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes read-only Memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), programmable read-only memory (Programmable Read-only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, EPROM), One-time Programmable Read-Only Memory (OTPROM), Electronically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory -Only Memory, CD-ROM) or other optical disk storage, magnetic disk storage, tape storage, or any other computer-readable medium that can be used to carry or store data.

An image processing method, device, electronic device, and storage medium disclosed in the embodiments of the present application have been described above in detail. In this paper, specific examples are used to illustrate the principles and implementation methods of the present application. The descriptions of the above embodiments are only used To help understand the method and core idea of the present application. At the same time, for those skilled in the art, based on the idea of this application, there will be changes in the specific implementation and application scope. In summary, the content of this specification should not be construed as limiting the application.

Claims (14)

1. An image processing method, characterized in that the method comprises: Acquiring aperture shape parameters, and obtaining a blurring intensity map and a weighting map corresponding to the first image, the blurring intensity map at least including blurring radii corresponding to each first pixel in the background area of the first image, so The weight map at least includes diffusion weights respectively corresponding to each first pixel in the background area of the first image; According to the aperture shape parameter, the blur intensity map and the weight map, perform point diffusion processing on each of the first pixels in the background area of the first image, to obtain a value corresponding to each of the first pixels Diffusion result; the diffusion result corresponding to each of the first pixel points in the background area includes: the diffusion range corresponding to the first pixel point, and the target pixel value of each second pixel point within the diffusion range , and the diffusion times of each second pixel within the diffusion range; wherein, the diffusion range corresponding to the first pixel is centered on the first pixel, and the radius of the circumscribed circle of the diffusion range is The blur radius corresponding to the first pixel point, the outer contour of the diffusion range is the shape of the aperture indicated by the aperture shape parameter; The diffusion results corresponding to the first pixel points are superimposed, and the background area is blurred according to the superposition results. 2. The method according to claim 1, wherein, according to the aperture shape parameter, the blur strength map and the weight map, each first pixel in the background area of the first image is Perform point diffusion processing to obtain the diffusion results corresponding to each first pixel point, including: For each first pixel in the background area of the first image, obtain the blur radius corresponding to the first pixel from the blur strength map, and obtain the blur radius corresponding to the first pixel from the weight map The diffusion weight corresponding to the first pixel point; A first aperture kernel corresponding to the first pixel is determined according to the blur radius corresponding to the first pixel and the aperture shape parameter, and the first aperture kernel is used to indicate the diffusion of the first pixel scope; According to the pixel value of the first pixel in the first image, the diffusion weight corresponding to the first pixel, and the first aperture kernel, each second pixel in the diffusion range corresponding to the first pixel is calculated. The target pixel value of the pixel point; Calculate the diffusion times of each second pixel within the diffusion range corresponding to the first pixel according to the diffusion weight corresponding to the first pixel and the first aperture kernel. 3. The method according to claim 2, wherein said superimposing the diffusion results corresponding to each first pixel point in the background area, and blurring the background area according to the superimposition results comprises: Superimpose the target pixel values of each second pixel contained in the diffusion range corresponding to each first pixel in the background area to the pixel value overlay map, and include the diffusion range corresponding to each first pixel in the background area The diffusion times of each of the second pixel points are superimposed on the times statistical graph; According to the pixel value of each first pixel point in the background area in the superimposed pixel value superposition map, and the total number of diffusions of each first pixel point in the superimposed number statistics map, the first Pixels are blurred. 4. The method according to claim 1, wherein the weight map comprises: a luminance weight map and a chrominance weight map; the diffusion weights include a luminance weight and a chrominance weight; Weight map, including: Calculate the brightness weight corresponding to each first pixel point according to the brightness component value of each first pixel point in the background area of the first image to obtain a brightness weight map; each first pixel point in the background area of the first image The brightness component value of the pixel is positively correlated with the corresponding brightness weight; Calculate the chromaticity weight corresponding to each first pixel according to the luminance component value and chromaticity component value of each first pixel; wherein, the chromaticity of each first pixel in the background area of the first image Both the component value and the brightness component value are positively correlated with the corresponding color weight. 5. The method according to claim 4, wherein the calculating the brightness weight corresponding to each first pixel point according to the brightness component value of each first pixel point in the background area of the first image comprises : According to the area of the highlight area included in the background area of the first image, calculate the brightness weight corresponding to the first pixel included in the highlight area; the brightness component values of the first pixel included in the highlight area are all greater than or equal to Brightness threshold. 6. The method according to claim 5, wherein the calculation of the brightness weight corresponding to the first pixel contained in the highlight area according to the area of the highlight area included in the background area of the first image includes : If the area of the highlight area is smaller than the area threshold, then setting the brightness weight corresponding to each first pixel contained in the highlight area to zero; and/or, If the area of the highlight area is greater than or equal to the area threshold, then according to the area of the highlight area, reduce the luminance component value of each first pixel contained in the highlight area, and use the reduced luminance component value as Brightness weights corresponding to the respective first pixels included in the highlight area; the decrease amount of the brightness component value of each first pixel included in the highlight area is positively correlated with the area of the highlight area. 7. The method according to claim 4, wherein the calculating the chrominance weight corresponding to each first pixel point according to the luminance component value and chrominance component value of each first pixel point comprises: calculating the chroma value of each first pixel point according to the chroma component value of each first pixel point in the background area; superimposing the luminance component value of each first pixel point in the background area and the chrominance value of each first pixel point to obtain a superimposed component value of each first pixel point in the background area; The chromaticity weight corresponding to each first pixel in the background area is determined according to the component superposition value of each first pixel in the background area. 8. The method according to claim 7, wherein the determining the chromaticity weight corresponding to each first pixel in the background area according to the component superposition value of each first pixel in the background area comprises : performing brightness stretching on the brightness component values of each first pixel in the background area to obtain the stretched brightness component values of each first pixel in the background area; wherein, each first pixel in the background area There is a positive correlation between the brightness stretching variation of the point and the brightness component value of each first pixel point in the background area before stretching; For each first pixel in the background area, compare the component superposition value of the first pixel with the stretched brightness component value of the first pixel, and compare the maximum value Determine the chroma weight corresponding to the first pixel. 9. The method according to claim 7, wherein, in the background area, the luminance component value of each first pixel point is superimposed with the chromaticity value of each first pixel point to obtain Before the component superposition value of each first pixel point in the background area, the method also includes: performing chromaticity stretching on the chromaticity values of each first pixel in the background area to obtain the stretched chromaticity value of each first pixel in the background area; wherein, each first pixel in the background area There is a positive correlation between the chromaticity stretch variation of the pixel and the brightness component value stretched by each first pixel in the background area; And, superimposing the luminance component value of each first pixel point in the background area and the chrominance value of each first pixel point to obtain the superimposed component value of each first pixel point in the background area ,include: Superimpose the luminance component value of each first pixel point in the background area and the stretched chrominance value of each first pixel point to obtain a component superposition value of each first pixel point in the background area. 10. The method according to claim 2, wherein after obtaining the diffusion weight corresponding to the first pixel from the weight map, the method further comprises: Map the diffusion weight corresponding to the first pixel point according to the weight mapping curve to obtain the diffusion weight after the mapping of the first pixel point; the weight mapping curve is obtained by fusing the first curve and the second curve, and the The first curve is a global smooth curve, and the second curve is a local smooth curve stretched around the first brightness level; And, calculating the diffusion range corresponding to the first pixel according to the pixel value of the first pixel in the first image, the diffusion weight corresponding to the first pixel, and the first aperture kernel The target pixel value of each second pixel point in , including: According to the pixel value of the first pixel in the first image, the diffusion weight after the mapping of the first pixel, and the first aperture kernel, calculate each second pixel in the diffusion range corresponding to the first pixel. The target pixel value of two pixels; And, the calculation of the number of diffusions of each pixel in the diffusion range corresponding to the first pixel according to the diffusion weight corresponding to the first pixel and the first aperture kernel includes: Calculate the diffusion times of each pixel within the diffusion range corresponding to the first pixel according to the diffusion weight after the mapping of the first pixel and the first aperture kernel. 11. The method of claim 2, further comprising: Acquiring multiple aperture images with different aperture shapes; Each aperture image in the plurality of aperture images is scaled multiple times to obtain multiple aperture kernels corresponding to each aperture shape, and each aperture kernel corresponds to a virtual blur radius. 12. An image processing device, comprising: An acquisition module, configured to acquire aperture shape parameters, and acquire a blurring intensity map and a weighting map corresponding to the first image, where the blurring intensity map at least includes the respective first pixel points in the background area of the first image corresponding to Blur radius, the weight map at least includes diffusion weights respectively corresponding to each first pixel in the background area of the first image; A diffusion module, configured to perform point diffusion processing on each first pixel in the background area of the first image according to the aperture shape parameter, the blur intensity map, and the weight map, to obtain the corresponding Diffusion results; the diffusion results corresponding to each of the first pixel points in the background area include: the diffusion range corresponding to the first pixel point, the target pixel of each second pixel point within the diffusion range value, and the diffusion times of each second pixel within the diffusion range; wherein, the diffusion range corresponding to the first pixel is centered on the first pixel, and the radius of the circumscribed circle of the diffusion range is the blur radius corresponding to the first pixel, and the outer contour of the diffusion range is the shape of the aperture indicated by the aperture shape parameter; The overlay blurring module is configured to overlay the diffusion results corresponding to the first pixel points in the background area, and blur the background area according to the overlay results. 13. An electronic device, characterized in that it comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is made to implement claims 1 to 11. any one of the methods described. 14. A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images