CN104715451B - A kind of image seamless fusion method unanimously optimized based on color and transparency - Google Patents

Abstract

The invention discloses an image seamless fusion method based on consistent optimization of color and transparency, belonging to the field of image graphics processing. The technical solution adopted is as follows: firstly, in the color channel, the fusion labels of the foreground and background images in the overlapping area are calculated based on the graph cut energy optimization algorithm; secondly, in the transparency channel, an energy function that conforms to the gradient difference distribution of the foreground and background is constructed to solve the transparency; Finally, images are seamlessly blended based on transparency. The method of the invention can obtain a better segmentation effect, thereby affecting the transparency distribution of the foreground and background images in the overlapping area, so that the transparency values are more uniform. Compared with other fusion methods, the present invention can generate better fusion effect.

Description

A method for seamless image fusion based on consistent optimization of color and transparency

technical field

The invention belongs to the field of image and graphic processing, in particular to an image seamless fusion method based on consistent optimization of color and transparency.

Background technique

Image fusion refers to combining two or more images into a new image. Image fusion plays an important role in virtual reality, digital media, and style design. Image fusion techniques can be divided into three levels: pixel-level fusion, feature-level fusion, and decision-level fusion. Pixel-level image fusion is mainly the comprehensive processing of information directly on each source image data, and it is the most basic tool in image fusion. As an important application of pixel-level image fusion, seamless image fusion technology plays a core role in the field of image and video editing.

At present, there have been a large number of researches on image seamless fusion methods, mainly in two categories: one is the fusion method based on multi-resolution, (Zhu Ruihui, Wan Min, Fan Guobin. Image fusion method based on pyramid transformation[J].Computer Simulation, 2008,24(12):178-180), this method uses the Laplacian pyramid decomposition method to decompose the original image at multiple resolutions, and then uses the method based on regional feature measurement to decompose each layer of the image fusion. (Gu Xiafang. Discussion and comparison of image fusion methods based on wavelet transform [J]. China West Science and Technology, 2009, 8(26): 21-22), this paper decomposes the source image by multi-layer wavelet to obtain a series of sub-images, And perform feature selection on the transform domain to create a fused image, and finally reconstruct the fused image through inverse transformation. This type of method is mainly to decompose the original image into multiple resolutions, and recompose the decomposed multi-scale images to achieve fusion transition within the entire image range, which can reduce the sensitivity to registration errors to a certain extent. However, due to signal attenuation caused by multiple filtering, the final composite image will be dark or blurred. The second is the fusion method based on the gradient domain, such as (Pérez P, Gangnet M, Blake A.Poisson image editing[C]//ACMTransactions on Graphics(TOG).ACM,2003,22(3):313-318), the In this paper, the image fusion problem is reduced to the minimization problem of the objective function by using the image gradient field, and the fusion is realized by solving the Poisson equation. This fusion method based on the Poisson equation needs to solve a large sparse linear equation system, which is computationally intensive and memory-intensive, and is not easy to implement. And (Farbman Z, Hoffer G, Lipman Y, et al.Coordinates for instant image cloning[C]//ACM Transactions on Graphics(TOG).ACM,2009,28(3):67), the article proposes a method based on the mean The fast image seamless fusion algorithm of coordinates interpolates the boundary pixels of the fusion area through the mean coordinates of each position in the fusion area to obtain the color value of each pixel in the fusion area. This method has high computational efficiency, is easy to implement and saves memory, and can perform real-time fusion. This kind of fusion method based on gradient field is fused in the color channel. After the foreground object is fused into the background image, the hue will change with the background image.

When the background color and texture of the source images are quite different, the image seamless fusion method can be extended to collage applications, and related work has been carried out. ROTHER et al. (Rother C, Bordeaux L, Hamadi Y, et al. Autocollage [C] // ACM Transactions on Graphics (TOG). ACM, 2006, 25 (3): 847-852), proposed a transparency channel-based Poisson puzzle method, which solves the transparency of the source image in the overlapping area by optimizing the minimum energy function, and performs fusion based on the transparency channel. However, the segmentation effect of this method is poor, and some important information in the source image may be lost in the obtained output mosaic, and the integrity of the foreground object in the source image cannot be guaranteed, and the transition at the boundary line of the composite image is not smooth enough.

Contents of the invention

In order to overcome the above-mentioned defects in the prior art, the object of the present invention is to provide a method for seamless fusion of images based on consistent optimization of color and transparency, which can obtain better segmentation results, thereby affecting the overlap between foreground and background images The transparency distribution of , making its transparency value more uniform.

The present invention is achieved through the following technical solutions:

An image seamless fusion method based on consistent optimization of color and transparency, the technical solution is:

First, use the graph cut energy optimization algorithm to calculate the fusion label of the foreground and background images in the overlapping area in the color channel; secondly, construct an energy function that conforms to the gradient difference distribution of the foreground and background in the transparency channel to solve the transparency; finally, image based on transparency Blends seamlessly.

Include the following steps:

1) Calculate the fusion label

Construct an energy function to calculate the color and transparency of image fusion, solve the energy function step by step, separate the variables, and use the graph cut energy optimization method to solve in the color channel to obtain the fusion label of the overlapping area;

2) Calculation Transparency

Fix the fusion label l(r), separate the unary function containing the variable α(r) from the unified energy function, perform optimization calculations, and obtain the transparency of the foreground and background images in the area to be fused by solving the linear equations;

3) Normalize the transparency value

Normalize the transparency values α _A and α _B of the foreground and background images respectively, so as to facilitate the mixing of the foreground and background images based on transparency, as follows:

4) Image Fusion

The foreground image f _A and the background image f _B are fused based on the normalized transparency α' _A and α' _B in the area to be fused, so as to obtain the fused output image, as shown in the following formula:

f=α' _A f _A +α' _B f _B .

Step 1) described energy function is as follows:

E＝ _min∫∫Ω ‖l(r)-α(r)‖ ² +λ(r)‖▽l‖ ² +w(r)‖▽α‖ ² dr

Among them, Ω is the overlapping area to be fused, and l(r) is a binary function, corresponding to the fusion label of each pixel in the fusion area; with are the foreground contour line of the area to be fused and the closed curve drawn by the user; α(r) is the transparency distribution function; λ(r) and w(r) represent the weight of the fusion label energy item and transparency energy item based on color information, respectively ; ▽l represents the gradient function of the fusion label, ▽α represents the gradient function of the transparency distribution, and dr represents the integral operator.

Step 1) Separate the variables, and the obtained unary function is as follows:

E(l)＝ _min∫∫Ω ‖l(r)-α(r)‖ ² +λ(r)‖▽▽l‖ ²

Among them, Ω is the overlapping area to be fused, and the energy function is solved by graph-cut energy optimization. The corresponding data items and smoothing items are:

E _s (p,q,l _p ,l _q )=|l _p -l _q |·(D(p)+D(q))

D(v)＝‖I ₁ (v)-I ₂ (v)‖ ² +α‖▽I ₁ (v)-▽I ₂ (v)‖ ²

In the above formula, α=2, ▽ represents the gradient operator, p and q in the formula represent the neighboring pixels, l _p and l _q represent the fusion labels corresponding to the neighboring pixels, I ₁ and I ₂ represent the foreground and The background image; the above formula shows that the greater the grayscale and gradient difference between the foreground image and the background image in the region to be fused, the greater the smoothing item, that is, the closer the color and gradient information of adjacent pixels, the fusion label Also keep consistent;

At the same time, in order to prevent the energy function from falling into a local minimum solution, a mandatory data constraint item is added to the boundary pixels of the overlapping area:

In the above formula, A represents the background image, B represents the foreground image, with Respectively represent the foreground contour line of the overlapping area and the closed curve drawn by the user, l _A and l _B represent the labels to which the foreground image and the background image belong, respectively.

When mosaicing source images with large differences in color and texture, if a boundary pixel in the overlapping area falls into an image, the boundary pixel is marked as the label to which the image belongs.

The unary function E _k (α) obtained by separating out the variable α(r) in step 2) is:

in, is the weighting factor, is the average gradient of the foreground image and the background image, ▽I ₁ , ▽I ₂ represent the gradient of the foreground image and the background image at pixel r respectively, σ ² is the mean square gradient of the foreground and background images; l(r) is binary function, l _r represents the fusion label of pixel r, and α(r) represents the transparency value of pixel r; the above formula shows that the transparency distribution of the overlapping area is as consistent as possible with the fusion label result, and the transparency value of adjacent pixels remains approximate;

Substitute the fusion labels of the foreground image and the background image into the above formula to solve, so as to obtain their transparency α _A and α _B in the overlapping area to be fused; the quadratic energy function can be solved using the linear equations shown in the following formula:

right

Among them, Ω is the overlapping area to be fused, N _p is the four-neighborhood of pixel p, w is the weighting factor mentioned above, α(p) is the transparency value of pixel p, and l(p) is the fusion label of pixel p. The target energy function in the above formula can ensure that the transparency value of the two images to be fused changes uniformly in the overlapping area, making the transition at the dividing line smooth after fusion, thus avoiding the blurring and color scattering phenomenon in the fusion based on color channels .

Compared with the prior art, the present invention has the following beneficial technical effects:

The present invention constructs a unified energy function to calculate the labels and transparency of the foreground and background images in the fusion area, and solves the problem step by step. Construct an energy function that conforms to the gradient difference distribution of the foreground and background to solve the transparency of the foreground and background images in the overlapping area, and finally fuse based on the transparency. At the same time, the image seamless fusion algorithm proposed by the present invention can also mosaic the source images with large differences in color and texture. The advantages are mainly reflected in:

(1) The present invention combines color and transparency to seamlessly fuse images, and utilizes the consistent optimization of color and transparency to achieve the effect of color tone maintenance and boundary continuous transition during fusion. Compared with the prior art, the method of the present invention will not change the original color tone of the foreground object after fusion.

(2) The image seamless fusion method proposed by the present invention can be extended to mosaic applications through the improvement and constraints of the boundary conditions. Compared with other mosaic algorithms, the calculation method of the fusion boundary in this invention can obtain better segmentation results, thus Affects the transparency distribution of the foreground and background images in the overlapping area to make the transparency value more uniform, so the transition of the output composite image at the split line is smoother.

Description of drawings

Fig. 1 is a schematic diagram of solving a fusion boundary dividing line based on graph cut optimization in the present invention;

Fig. 2 is the solution result figure of the image seamless fusion dividing line in the specific example of the present invention;

Figures 3, 4, and 5 are comparison experiment results of seamless fusion of source images and target images using different backgrounds; among them, 3-1, 4-1, and 5-1 are foreground images; 3-2, 4-2 And 5-2 is the result figure of prior art processing; 3-3, 4-3 and 5-3 are the result figure of the method of the present invention process;

Fig. 6 is a schematic diagram of solving the graph-cut energy segmentation line based on the jigsaw puzzle;

Fig. 7 The comparison results of the experimental results of the two image puzzles;

Wherein, 7-1 and 7-2 are two selected foregrounds to be spliced; 7-3 is a schematic diagram of the solution result of the dividing line; 7-4 is a result map processed by the prior art; 7-5 is processed by the method of the present invention Result graph;

Fig. 8 is the comparison result figure of the experimental results of multiple image puzzles;

Among them, 8-1 is the result diagram of prior art processing; 8-2 is the result diagram of the method of the present invention.

detailed description

The present invention will be further described in detail below in conjunction with specific embodiments, which are explanations of the present invention rather than limitations.

An image seamless fusion method based on consistent optimization of color and transparency of the present invention establishes a unified mathematical model to calculate the fusion label and transparency of the foreground and background images in the overlapping area, and solves it through a two-step optimization strategy: First, in the color channel, the fusion labels of the foreground and background images in the overlapping area are calculated based on the graph cut energy optimization algorithm, and then in the transparency channel, the transparency is solved according to the energy function that conforms to the gradient difference distribution of the foreground and background, and fusion is performed based on the transparency.

Include the following steps:

1) Calculate the fusion label

Construct an energy function to calculate the color and transparency of image fusion, solve the energy function step by step, separate the variables, and use the graph cut energy optimization method to solve in the color channel to obtain the fusion label of the overlapping area;

Described energy function is as follows:

E＝ _min∫∫Ω ‖l(r)-α(r)‖ ² +λ(r)‖▽l‖ ² +w(r)‖▽α‖ ² dr

Among them, l(r) is a binary function, corresponding to the fusion label of each pixel in the fusion area; with are the foreground contour line of the area to be fused and the closed curve drawn by the user; α(r) is the transparency distribution function; λ(r) and w(r) represent the weight of the fusion label energy item and transparency energy item based on color information, respectively ; The energy function shows that the transparency values of the foreground and background tend to be consistent with the fusion label as much as possible, and the label results and transparency of adjacent pixels in the overlapping area are as similar as possible, so that the transparency of the overlapping area is evenly distributed, and the fusion boundary The line tends to be smooth.

The variable is separated, and the unary function obtained is as follows:

E(l)＝ _min∫∫Ω ‖l(r)-α(r)‖ ² +λ(r)‖▽▽l‖ ²

Among them, Ω is the overlapping area to be fused, and the energy function is solved by graph-cut energy optimization. The corresponding data items and smoothing items are:

E _s (p,q,l _p ,l _q )=|l _p -l _q |·(D(p)+D(q))

D(v)＝‖I ₁ (v)-I ₂ (v)‖ ² +α‖▽I ₁ (v)-▽I ₂ (v)‖ ²

In the above formula, α=2, ▽ represents the gradient operator, and the above formula shows that the greater the gray level and gradient difference between the foreground image and the background image in the region to be fused, the greater the smoothing term, that is, the The closer the color and gradient information are, the more consistent their fusion labels are;

At the same time, in order to prevent the energy function from falling into a local minimum solution, a mandatory data constraint item is added to the boundary pixels of the overlapping area:

In the above formula, A represents the background image, B represents the foreground image, with Respectively represent the foreground contour line of the overlapping area and the closed curve drawn by the user;

When mosaicing source images with large differences in color and texture, the boundary conditions are improved as follows: if a boundary pixel in the overlapping area falls into an image, the boundary pixel is marked as the label to which the image belongs.

2) Calculation Transparency

Fix the fusion label l(r), separate the unary function containing the variable α(r) from the unified energy function, perform optimization calculations, and obtain the transparency of the foreground and background images in the area to be fused by solving the linear equations;

The unary function E(α) obtained by separating out the variable α(r) is:

in, is the weighting factor, is the average gradient of the foreground image and the background image, σ ² is the mean square gradient of the image; the above formula shows that the transparency distribution of the overlapping area is as close as possible to the result of the fusion label, and the transparency values of adjacent pixels remain approximate;

Substitute the fusion labels of the foreground image and the background image into the above formula to solve, so as to obtain their transparency α _A and α _B in the overlapping area to be fused; the quadratic energy function can be solved using the linear equations shown in the following formula:

right

Among them, N _p represents the four-neighborhood of pixel p, and the target energy function in the above formula can ensure that the transparency values of the two images to be fused change uniformly in the overlapping area, making the transition smooth at the dividing line after fusion, thereby avoiding the Blurring and color scattering phenomena that occur in color channel fusion.

3) Normalize the transparency value

Normalize the transparency values α _A and α _B of the foreground and background images respectively, so as to facilitate the mixing of the foreground and background images based on transparency, as follows:

4) Image Fusion

The foreground f _A and the background image f _B are fused based on the normalized transparency α' _A and α' _B in the area to be fused, so as to obtain the fused output image, as shown in the following formula:

f=α' _A f _A +α' _B f _B .

The specific implementation process is as follows:

Step 1. Fusion label calculation

First ask the user to roughly draw a closed curve around the foreground object to be inserted And use the Grabcut (Rother C, Kolmogorov V, Blake A.Grabcut:Interactive foreground extractionusing iterated graph cuts[C]//ACM Transactions on Graphics(TOG).ACM,2004,23(3):309-314) algorithm for segmentation to obtain the edge contour line containing the foreground object The overlapping area is formed by the pixels between the closed curve drawn by the user and the edge contour. We solve the mathematical model established by the separation of variables method. First, separate the variable l(r) from the unified mathematical model, transform it into graphcut energy optimization for solution, and use the color and gradient information of the overlapping area to calculate the corresponding graph cut optimization. The energy item is then solved by the maximum flow minimum cut algorithm, and the obtained minimum cut set is the segmentation line, and the fusion label of each pixel in the overlapping area is obtained at the same time. The boundary conditions when optimizing the energy function based on graph cuts are considered as follows: For seamless image fusion, pixels located on the closed curve specified by the user in the overlapping area are marked as belonging to the background label. The data energy term is infinite, while the data energy belonging to the foreground label term is 0; conversely, pixels lying on the foreground contour line are marked as belonging to the foreground label. The data energy term is infinite, while the data energy term belonging to the background label is 0:

is the foreground contour line, is the closed curve drawn by the user, l _A and l _B are the labels represented by the background and foreground images respectively, and the schematic diagram of solving the segmentation line based on graph cut optimization is shown in Figure 1. Figure 2 shows the results of the seamless fusion of images with the Mother's Day themed poster as the background of the dividing line. For image mosaic applications, if the pixel on the boundary line of the overlapping area is located inside an image, the data energy item marked as belonging to this image label is infinite, and the data energy item marked as belonging to other labels is 0. For example, the schematic diagram for solving the puzzle-based graph cut dividing line shown in Figure 6, the boundary constraints are as follows:

Step 2. Transparency calculation

We fix l(r), separate the variable α(r) from the unified mathematical model, obtain a quadratic energy function that conforms to the gradient difference distribution of the foreground and background, and perform partial differential derivatives on it to obtain a linear equation about the transparency distribution function , and the first-order equations of each pixel in the overlapping area are combined to form a linear equation system to solve the transparency of the foreground and background images in the overlapping area. The transparency is a real-valued function, α(r)∈[0 1]. For the seamless fusion of images, it is assumed that the labels of the foreground and the background are 1 and 0 respectively. If the label of a certain pixel after segmentation is 1; then the transparency of the foreground image at this pixel is close to 1, while the transparency of the background image at this pixel is close to On the contrary, if a pixel is marked as 0 after segmentation, the transparency of the background image at this pixel is approximately 1, while the transparency of the foreground image at this pixel is approximately 0; the above shows that the transparency of the foreground and background images in the overlapping area The value tends to be consistent with the fusion label.

Step 3. Normalize the transparency value

Normalize the transparencies of the foreground and background images obtained in step 2 in overlapping regions to obtain their contrasting transparencies in overlapping regions.

Step 4. Image Fusion

Foreground and background images are weighted blended with normalized transparency to obtain a fused generated image. We selected source images and target images of different backgrounds for seamless fusion experiments, and compared with literature [13] (Chen T, Zhu J Y, Shamir A, et al.Motion-Aware Gradient Domain Video Composition[J].Image Processing, IEEE Transactions on, 2013, 22(7): 2532-2544), the experimental results are compared, as shown in Figures 3, 4, and 5. Among them, 3-1, 4-1 and 5-1 are foreground images; 3-2, 4-2 and 5-2 are result images processed by prior art (document [13]); 3-3, 4-3 And 5-3 is the result figure that the method of the present invention handles; From the experimental results, compared with other image fusion algorithms, because the algorithm proposed in the present invention combines color and transparency channel to fuse, can not change the original tone of the foreground image On the basis of , the color mutation problem of saturated pixels is effectively avoided, and at the same time, the algorithm in the present invention makes the transparency values evenly distributed, so the blending boundary transition is smoother.

The algorithm in the present invention is extended to puzzle applications through the constraints of boundary conditions. Figure 7 and Figure 8 are the comparative experimental results of two and multiple image puzzles respectively. We have made a comparison with the existing technology, and the literature [1] is ( Pérez P, Gangnet M, Blake A.Poisson image editing[C]//ACM Transactions on Graphics (TOG).ACM,2003,22(3):313-318); literature [5] is (Rother C, Bordeaux L, Hamadi Y, et al.Autocollage[C]//ACM Transactions on Graphics(TOG).ACM,2006,25(3):847-852); Among them, 7-1 and 7-2 are two selected images to be spliced 7-3 is a schematic diagram of the solution result of the dividing line; 7-4 is the result figure processed by the prior art (document [1]); 7-5 is the result figure processed by the method of the present invention; 8-1 is the prior art (Document [5]) the result figure of processing; 8-2 is the result figure of the method of the present invention process. Among them, the results of the two image puzzles show that the boundaries after fusion are relatively abrupt, and color scattering and blurring are prone to occur. However, the segmentation result obtained in the present invention is more reasonable, and the transition at the segmentation line is smoother. The experiment results of multiple image collages show that when the color and texture of the source images are quite different, the present invention can realize the effect of continuous border transition after collage compared with other methods.

In summary, the method of the present invention establishes a unified mathematical model to calculate the color and transparency of image fusion, and adopts a two-step optimization strategy to solve the problem: first, calculate the fusion label in the color channel through the graph cut optimization algorithm; then optimize the The model solves for optimal transparency for image fusion. Compared with other fusion methods, the present invention can generate better fusion effect.

Claims (5)

1. An image seamless fusion method based on color and transparency consistency optimization is characterized in that,

the method comprises the following steps:

1) computing fusion tags

Constructing an energy function for calculating color and transparency during image fusion, solving the energy function step by step to separate out variables, and solving in a color channel by using a graph cut energy optimization method to obtain a fusion label of an overlapping area;

2) calculating transparency

fixing a fusion label l (r), separating a unitary function containing a variable α (r) from the unified energy function, performing optimization calculation, and respectively obtaining the transparency of the foreground image and the background image in the region to be fused by solving a linear equation set, wherein α (r) is the transparency value of a pixel r;

3) normalizing the transparency value

the transparency values alpha of the foreground and background images are respectively taken_A，α_BNormalization was performed as follows:

<mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <msubsup> <mi>&amp;alpha;</mi> <mi>A</mi> <mo>&amp;prime;</mo> </msubsup> <mo>=</mo> <mfrac> <msub> <mi>&amp;alpha;</mi> <mi>A</mi> </msub> <mrow> <msub> <mi>&amp;alpha;</mi> <mi>A</mi> </msub> <mo>+</mo> <msub> <mi>&amp;alpha;</mi> <mi>B</mi> </msub> </mrow> </mfrac> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msubsup> <mi>&amp;alpha;</mi> <mi>B</mi> <mo>&amp;prime;</mo> </msubsup> <mo>=</mo> <mfrac> <msub> <mi>&amp;alpha;</mi> <mi>B</mi> </msub> <mrow> <msub> <mi>&amp;alpha;</mi> <mi>A</mi> </msub> <mo>+</mo> <msub> <mi>&amp;alpha;</mi> <mi>B</mi> </msub> </mrow> </mfrac> </mrow> </mtd> </mtr> </mtable> </mfenced>

4) image fusion

For foreground image f_AAnd a background image f_Bbased on normalized transparency α 'in the region to be fused'_A、α'_BAnd performing fusion to obtain a fused output image, which is shown as the following formula:

f＝α'_Af_A+α'_Bf_B。

2. the method for image seamless fusion based on consistent optimization of color and transparency according to claim 1, wherein the energy function in step 1) is as follows:

<mrow> <mi>E</mi> <mo>=</mo> <mi>m</mi> <mi>i</mi> <mi>n</mi> <mo>&amp;Integral;</mo> <msub> <mo>&amp;Integral;</mo> <mi>&amp;Omega;</mi> </msub> <mo>|</mo> <mo>|</mo> <mi>l</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>-</mo> <mi>&amp;alpha;</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> <mo>+</mo> <mi>&amp;lambda;</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>|</mo> <mo>|</mo> <mo>&amp;dtri;</mo> <mi>l</mi> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> <mo>+</mo> <mi>w</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>|</mo> <mo>|</mo> <mo>&amp;dtri;</mo> <mi>&amp;alpha;</mi> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> <mi>d</mi> <mi>r</mi> </mrow>

<mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mi>l</mi> <mo>(</mo> <mi>r</mi> <mo>)</mo> <mo>=</mo> <mn>1</mn> <mo>,</mo> <mi>r</mi> <mo>&amp;Element;</mo> <msub> <mo>&amp;part;</mo> <mrow> <mi>o</mi> <mi>u</mi> <mi>t</mi> </mrow> </msub> </mtd> </mtr> <mtr> <mtd> <mi>l</mi> <mo>(</mo> <mi>r</mi> <mo>)</mo> <mo>=</mo> <mn>0</mn> <mo>,</mo> <mi>r</mi> <mo>&amp;Element;</mo> <msub> <mo>&amp;part;</mo> <mrow> <mi>i</mi> <mi>n</mi> </mrow> </msub> </mtd> </mtr> <mtr> <mtd> <mrow> <mi>l</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>&amp;Element;</mo> <mo>{</mo> <mn>0</mn> <mo>,</mo> <mn>1</mn> <mo>}</mo> </mrow> </mtd> </mtr> </mtable> </mfenced>

wherein, Ω is an overlapping region to be fused, l (r) is a fusion label, which is a fusion label of each pixel r in the fusion region and is a binary function;andrespectively representing the foreground contour line of the region to be fused and a closed curve drawn by a user, wherein alpha (r) is the transparency value of a pixel r, and lambda (r) and w (r) respectively represent the weights of a fusion label energy item and a transparency energy item based on color information;a gradient function representing the fusion tag is provided,a gradient function representing the transparency distribution, dr representing an integration operator.

3. The method for image seamless fusion based on consistent optimization of color and transparency according to claim 2, wherein the variables are separated in step 1), and the obtained univariate function is as follows:

<mrow> <mi>E</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> <mo>=</mo> <mi>m</mi> <mi>i</mi> <mi>n</mi> <mo>&amp;Integral;</mo> <msub> <mo>&amp;Integral;</mo> <mi>&amp;Omega;</mi> </msub> <mo>|</mo> <mo>|</mo> <mi>l</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>-</mo> <mi>&amp;alpha;</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> <mo>+</mo> <mi>&amp;lambda;</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>|</mo> <mo>|</mo> <mo>&amp;dtri;</mo> <mi>l</mi> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> </mrow>

wherein omega is an overlapping area to be fused, the energy function is solved by graph cut energy optimization, and a corresponding data item E_d(p,l_p) And a smoothing term E_s(p,q,l_p,l_q) Respectively as follows:

<mrow> <msub> <mi>E</mi> <mi>d</mi> </msub> <mrow> <mo>(</mo> <mi>p</mi> <mo>,</mo> <msub> <mi>l</mi> <mi>p</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mo>-</mo> <mo>&amp;dtri;</mo> <msub> <mi>I</mi> <msub> <mi>l</mi> <mi>p</mi> </msub> </msub> </mrow>

E_s(p,q,l_p,l_q)＝|l_p-l_q|·(D(p)+D(q))

<mrow> <mi>D</mi> <mrow> <mo>(</mo> <mi>v</mi> <mo>)</mo> </mrow> <mo>=</mo> <mo>|</mo> <mo>|</mo> <msub> <mi>I</mi> <mn>1</mn> </msub> <mrow> <mo>(</mo> <mi>v</mi> <mo>)</mo> </mrow> <mo>-</mo> <msub> <mi>I</mi> <mn>2</mn> </msub> <mrow> <mo>(</mo> <mi>v</mi> <mo>)</mo> </mrow> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> <mo>+</mo> <mi>&amp;alpha;</mi> <mo>|</mo> <mo>|</mo> <mo>&amp;dtri;</mo> <msub> <mi>I</mi> <mn>1</mn> </msub> <mrow> <mo>(</mo> <mi>v</mi> <mo>)</mo> </mrow> <mo>-</mo> <mo>&amp;dtri;</mo> <msub> <mi>I</mi> <mn>2</mn> </msub> <mrow> <mo>(</mo> <mi>v</mi> <mo>)</mo> </mrow> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> </mrow>

in the above formula, α is 2,representing a gradient operator, where p, q represent neighborhood pixels, l_p,l_qRepresenting fusion labels corresponding to the neighbourhood pixels, I₁,I₂Representing foreground and background images to be fused;

meanwhile, a mandatory data constraint item is added to the boundary pixel of the overlapping area:

<mrow> <msub> <mi>E</mi> <mi>d</mi> </msub> <mrow> <mo>(</mo> <msub> <mo>&amp;part;</mo> <mrow> <mi>o</mi> <mi>u</mi> <mi>t</mi> </mrow> </msub> <mo>,</mo> <msub> <mi>l</mi> <mi>A</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mi>&amp;infin;</mi> <mo>,</mo> <msub> <mi>E</mi> <mi>d</mi> </msub> <mrow> <mo>(</mo> <msub> <mo>&amp;part;</mo> <mrow> <mi>o</mi> <mi>u</mi> <mi>t</mi> </mrow> </msub> <mo>,</mo> <msub> <mi>l</mi> <mi>B</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mn>0</mn> </mrow>

<mrow> <msub> <mi>E</mi> <mi>d</mi> </msub> <mrow> <mo>(</mo> <msub> <mo>&amp;part;</mo> <mrow> <mi>i</mi> <mi>n</mi> </mrow> </msub> <mo>,</mo> <msub> <mi>l</mi> <mi>A</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mn>0</mn> <mo>,</mo> <msub> <mi>E</mi> <mi>d</mi> </msub> <mrow> <mo>(</mo> <msub> <mo>&amp;part;</mo> <mrow> <mi>i</mi> <mi>n</mi> </mrow> </msub> <mo>,</mo> <msub> <mi>l</mi> <mi>B</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mi>&amp;infin;</mi> </mrow>

in the above equation, a denotes a background image, B denotes a foreground image,andare respectively provided withForeground contour lines representing overlapping areas and closed curves drawn by the user, |_A、l_BRespectively representing the labels to which the foreground image and the background image belong.

4. The method as claimed in claim 3, wherein when the source image with color and texture difference is subjected to mosaicing, if a boundary pixel of the overlap region falls into an image, the boundary pixel is marked as a label to which the image belongs.

5. the method for seamless image fusion based on consistent optimization of color and transparency according to claim 2, wherein the step 2) is to separate the univariate function E obtained by the inclusion of the variable α (r)_kand (α) is:

<mrow> <msub> <mi>E</mi> <mi>k</mi> </msub> <mrow> <mo>(</mo> <mi>&amp;alpha;</mi> <mo>)</mo> </mrow> <mo>=</mo> <munder> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> <mi>&amp;alpha;</mi> </munder> <munder> <mo>&amp;Sigma;</mo> <mrow> <mi>r</mi> <mo>&amp;Element;</mo> <mi>&amp;Omega;</mi> </mrow> </munder> <mo>|</mo> <mo>|</mo> <mi>l</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>-</mo> <mi>&amp;alpha;</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> <mo>+</mo> <mi>w</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>|</mo> <mo>|</mo> <mo>&amp;dtri;</mo> <mi>&amp;alpha;</mi> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> </mrow>

wherein,in order to be a weighting factor, the weighting factor,is the average gradient of the foreground image and the background image,representing the gradient, σ, of the foreground and background images at pixel r, respectively²the mean square gradient of the foreground and background images, l (r) is a fusion label, which refers to the fusion label of each pixel r in a fusion area and is a binary function, α (r) represents the transparency value of the pixel r;

respectively substituting the fusion labels of the foreground image and the background image into the formula to solve to obtain the transparency α of the foreground image and the background image in the overlapped region to be fused_Aand alpha_B(ii) a The quadratic energy function is solved using a system of linear equations as shown below:

to pair

Wherein omega is an overlapping region to be fused, N_pthe method comprises the steps of representing four neighborhoods of a pixel p, wherein w is a weighting factor, p and q represent neighborhood pixels, α (p) represents a transparency value of the pixel p, α (q) represents a transparency value of the pixel q, and l (p) represents a fusion label of the pixel p.