JPH11317909A - Picture synthesis method/device and data recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the picture quality of the boundary part of a synthesis picture by synthesizing texture signals between a picture element to be processed and a corresponding picture element in a second picture space corresponding to it based on the synthesis ratio of the texture signal of the picture element to be processed and generating a synthesis picture signal forming a third picture space. SOLUTION: When a form signal is inputted from an input terminal 2100a, a boundary position judgment means 2104 judges the boundary position of an object. In the case of the picture element of the boundary position, a synthesis ratio generation means 2106 calculates the synthesis ratio of the picture to be processed. A picture synthesis means 2110 obtains texture data of a synthesis picture by using texture data of a foreground picture being a processing object inputted from a second input terminal 2102, background picture texture data inputted from a third input terminal 2116 and the synthesis ratio calculated by the synthesis ratio generation means 2106. Switches 2122 and 2124 change over a picture element synthesis means 2112 near a boundary and a picture element synthesis means 2114 in the boundary.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image synthesizing apparatus, an image synthesizing method, and a data recording medium. It relates to a recording medium storing a program.

[0002]

2. Description of the Related Art In recent years, a multimedia era has been entered in which voice, image, and other data are handled in an integrated manner. Has become a feature of multimedia. Generally, multimedia means not only characters but also graphics, sounds, and especially images, etc. are simultaneously associated with each other. To make the above-mentioned conventional information media the target of multimedia, the information must be expressed in digital form. Is an essential condition.

However, when the information amount of each of the above information media is estimated as a digital information amount, the information amount per character is 1 to 2 bytes in the case of a character, while the information amount per character is 64 kb / s in the case of voice. (Telephone quality), and more than 100Mb per second (current television reception quality) for moving images, it is actually impossible to handle the huge amount of information handled by information media such as television in digital form. Not a target. For example, a videophone is an integrated services digital network (ISDN) having a transmission speed of 64 kbps to 1.5 Mbps.
al Network), but it is impossible to send the video of a TV camera directly through ISDN.

Therefore, what is required is an information compression technique. For example, in the case of a videophone, ITU-T
(International Telecommunication Union Telecommunication Standardization Sector) The H.261 standard moving image compression technology is used. Further, according to the information compression technology of the MPEG1 standard, it is possible to store image information together with audio information in a normal music CD (compact disc).

Here, MPEG (Moving Picture Exper)
(ts Group) is an international standard for moving image data compression. In MPEG1, moving image data is compressed to 1.5 Mbps, that is, information of a television signal is compressed to about 1/100. In addition, since the transmission speed targeted by the MPEG1 standard is mainly limited to about 1.5 Mbps, MPEG2 standardized to meet the demand for higher image quality,
Video data is compressed so that its transmission speed is about 2 to 15 Mbps to achieve the current television quality. Furthermore, at present, a working group that has promoted standardization with MPEG1 and MPEG2 (ISO / IEC JTC1 / SC29 / WG11)
Accordingly, MPEG4, which enables encoding and manipulation of image information in units of objects and realizes new functions required in the multimedia age, is being standardized.

One of the features of MPEG4 is that a plurality of objects, for example, a foreground image (object) 102 and a background image (object) shown in FIG.
The point is that encoding processing is separately performed on an image signal for displaying 100.

The image signal corresponding to each object is composed of a shape signal representing the shape of each object and a texture signal representing the texture of each object. In the processing, the encoding process for the shape signal and the encoding process for the texture signal are performed by different encoding methods.

Here, the texture signal includes a luminance signal as information indicating the brightness of the image of the object and a color difference signal as information indicating the color of the image of the object. The resolution is different from the resolution of the luminance signal. That is, the size of each pixel constituting the image space obtained from the luminance signal (the area of one pixel occupied on the display screen) is 1/4 of the size of one pixel in the image space obtained from the color difference signal. ing. Then, the luminance signal and the color difference signal are constituted by pixel values corresponding to each pixel in each image space.

The resolution of the shape signal is the same as the resolution of the luminance signal. That is, the size of one pixel constituting the image space obtained from the shape signal is obtained from the luminance pixel. The size of one pixel constituting the image space to be obtained. The shape signal is composed of pixel values corresponding to each pixel in the image space. Therefore, the luminance signal and the shape signal for a region surrounding one object on the screen (hereinafter, referred to as an object region) have the same number of pixel values.

In the following description, an image space obtained from a luminance signal, a color difference signal, and a shape signal is called a luminance space, a color space, and a shape space, respectively, as necessary.
The pixels that constitute the luminance space, color space, and shape space are called luminance pixels, color pixels, and shape pixels.
The pixel values of the color pixel and the shape pixel are referred to as luminance data, color difference data, and shape data. Further, an image signal corresponding to one object, that is, a shape space, a luminance space, and a color space formed by the shape signal, the luminance signal, and the color difference signal have the same reference position and the same size. Needless to say.

The pixel value of the shape signal is "0 value".
When represented by the binary value of “0 value”, the shape pixel having the pixel value of “0 value” is located outside the image of the object in the shape space, and has the shape of the pixel value of “non 0 value”. The pixel is assumed to be located in the image of the object in the shape space.

On the other hand, at the time of decoding, as shown in FIG. 23, encoded information (encoded texture signal and encoded shape signal) corresponding to each object (foreground image and background image) is decoded and decoded. Foreground image 10 using the transformed shape signal
The texture signal is synthesized between 0 and the background image 102 to reproduce the synthesized image 106. In FIG. 23,
Reference numeral 104 denotes the shape of the object obtained from the shape signal.
In addition, in the texture signal synthesizing process, a synthesizing process for a luminance signal and a synthesizing process for a color difference signal are performed.

As described above, the resolution of the chrominance signal is one-fourth the resolution of the luminance signal and the shape signal. Therefore, in the color difference signal synthesizing process, the decoded shape signal (hereinafter, the shape signal before conversion) is used. ) Cannot be used as it is, and it is necessary to convert the resolution of the shape signal to generate a shape signal corresponding to the resolution of the color difference signal (hereinafter, also referred to as a converted shape signal). In MPEG4, MPEG-4 Video Verif
ication Model Ver8.0 (ISO / IEC JTC1 / SC29 / WG11 N1796,
Use the method described in pp17).

More specifically, since the resolution of the color difference signal is １ ／ of the resolution of the shape signal as described above, one pixel in the color space has four adjacent pixels in the image space obtained from the shape signal before conversion. Will correspond. Further, since the resolution of the decoded shape signal is converted to correspond to the resolution of the color difference signal, one pixel in the color space corresponds to one pixel in the image space obtained from the converted shape signal.

Therefore, in the resolution conversion method, one of the four shape pixels corresponding to one color pixel is set to “0”.
If the pixel has a non-value pixel value, in other words, if there is a shape pixel whose pixel value is “non-zero value” as one of the four shape pixels corresponding to a certain color pixel, The pixel value of one shape pixel corresponding to the certain one color pixel in the image space obtained from the converted shape signal is set to “non-zero value”.

Next, the image synthesizing method will be specifically described with reference to FIG. First, a description will be given of a synthesizing process in the case where the shape signal is a binary signal having one of “0 value” and “non-zero value” as a pixel value. When the luminance signal of the foreground image 102 is combined with the luminance signal of the background image 100 by referring to the shape signal, the pixel value is “non-zero value”.
For the luminance pixel corresponding to the shape pixel (that is, the pixel located in the black display portion in the shape space 104), the pixel value of the foreground image is set as the pixel value of the composite pixel. Further, as for the luminance pixel corresponding to the shape pixel having the pixel value “0 value” (that is, the pixel located outside the black display portion in the shape space 104), the pixel value of the background image is set as the pixel value of the composite image. . Such a combining process is similarly performed on the color difference signal using the above-mentioned converted shape signal.

Next, the shape signal is represented by a pixel value
A description will be given of a synthesis process in the case of a multi-level signal having any of 0, 1, 2, 3,..., 255. When the luminance signal of the foreground image 102 is synthesized with the luminance signal of the background image 100 with reference to the shape signal, the luminance signal of the foreground image 102 corresponds to a shape pixel having the minimum value of the multilevel signal as a pixel value.
The luminance pixel in the composite image has the pixel value of the background image as its pixel value, and corresponds to the shape pixel having the maximum value of the multi-level signal as the pixel value. The luminance pixel in the composite image has the foreground as its pixel value. It will have the pixel value of the image.

When the shape signal is a multi-valued signal, the color difference signal synthesis process is performed in the same manner as the luminance signal synthesis process, and the maximum value or the minimum value of the converted shape signal after resolution conversion is determined by the pixel. Corresponding to the shape pixels having as values,
The color pixel in the composite image has the pixel value of the foreground image or the background image as its pixel value. In addition,
In this case, the average value of the pixel values of the four shape pixels corresponding to one color pixel is calculated as one shape pixel corresponding to the one color pixel in the image space obtained from the converted shape signal. Pixel value.

When the shape signal is a multi-level signal,
A resolution conversion process different from the case where the shape signal is a binary signal is performed. That is, when the shape signal has an intermediate value between the maximum value and the minimum value in the multilevel signal as a pixel value, the foreground image and the background corresponding to the foreground image are used as the weighting coefficient using the shape data (the pixel value of the shape signal). A pixel value of a synthesized pixel is obtained by taking a weighted average of pixel values of a texture signal (luminance signal, color difference signal) between the image and the image. As a result, the background image is displayed while being transmitted through the foreground image. The following is an example of an arithmetic expression for performing the above-mentioned weighted averaging.

Pel = (alpha × fgpel + (MAX−alpha) × bgpel) / MAX (Equation 1) Here, each element of the equation 1 is as follows. pel: Pixel value after composition alpha: Composition ratio to pixel value of foreground image (shape data) fgpel: Pixel value of foreground image (luminance data or color difference data) bgpel: Pixel value of background image (luminance data or color difference data) MAX : The maximum value of the shape signal. If alpha has a value from 0 to 255, MA
X becomes 255. Also, here, the calculation for obtaining the weighted average according to the above equation 1 includes a process of rounding the calculation result, but the calculation according to the equation 1 may be performed by truncating decimal places in the calculation result.

In the calculation for obtaining the weighted average, the pixel value (luminance data or color difference data) of the foreground image is multiplied in advance by the pixel value (shape data) of the corresponding shape signal, and There are also ways to reduce multiplication. In that case, Equation 1 is represented as Equation 2 below. pel = fgpel '+ (1−alpha ′) × bgpel (Equation 2) alpha ′: synthesis ratio (shape data) to pixel value of foreground image fgpel ′: pixel value of foreground image multiplied by alpha ′ (luminance data or (Color difference data) Note that alpha ′ takes a value from 0/255 to 255/255. In the above equation 2, if fgpel ′ = fgpel × alpha ′, the same operation result as in the case of MAX = 1 in the above equation 1 can be obtained.

[0022]

However, when the foreground image and the background image are combined using the binary shape signal, the pixel value of the pixel is determined at the boundary between the foreground image and the background image in the combined image. If the luminance data and the color difference data take values far apart from each other, there is a problem that the boundary looks unnatural.

This problem will be described with reference to an example shown in FIG. FIG. 24A shows a synthesized image 106 obtained by synthesizing the foreground image and the background image, and FIG. 24B shows the boundary between the foreground image and the background image in the synthesized image. Area 20 of predetermined size (8 × 8 pixels) including
0 is shown in a zoomed-up manner, and in particular, each pixel included in the area 200 is shown.

[0024] As shown in FIG.
In the boundary portion between the foreground image and the background image, for example, between the pixel 204 and the pixel 206, if the pixel values of both pixels are far apart from each other, the boundary may look unnatural.

FIG. 25 (a) shows, in pixel units, a portion 300 of an image obtained from the shape signal of the foreground image, which corresponds to the portion 200 of the boundary in the composite image 106. FIG. 25B shows a pixel value (shape data) of each pixel included in one pixel line 302 in this portion 300.
Are shown on a coordinate plane where the horizontal axis is the pixel position and the vertical axis is the value of the shape data. Here, possible values of the shape data of each pixel are values from 0 to 255.

When processing for synthesizing the pixel values of the foreground and background pixels corresponding to the shape pixel based on such a shape signal is performed, each pixel (synthetic pixel) constituting the synthesized image
Is obtained (synthesized data). At this time, the synthesized pixel corresponding to the shape pixel whose pixel value is not 0 is
As the pixel value, the pixel value of the corresponding foreground pixel or the value obtained by weighting and averaging the pixel values of the foreground and the background as described above. On the other hand, a composite pixel corresponding to a shape pixel having a pixel value of 0 has the pixel value of the corresponding background pixel as its pixel value.

At this time, when the foreground pixel and the background pixel have discontinuous values as pixel values in the boundary portion 306, for example, the pixel value of the foreground pixel (luminance data or color difference data) and the background If the pixel value (luminance data or color difference data) of the pixel becomes extremely different, the display of the foreground image and the background image on the display screen of the composite image looks unnatural at the boundary. .

Therefore, it is necessary to process the pixel values of the pixels at the boundary portion or the combination ratio used in the composition so that the boundary portion in the composite image does not look unnatural. Further, when the resolution of the shape signal and the resolution of the texture signal are different, a texture signal is synthesized between the foreground image and the background image using the shape signal, or a synthesized image obtained by synthesizing the texture signal is used as a necessary image. Conversion to a screen size (resolution) may result in degradation of image quality.

The case where such image deterioration occurs will be described below with reference to FIGS. FIG. 26 shows 4 ×
Using a shape signal composed of pixel values of four pixels, 2 ×
FIG. 27 is a diagram conceptually illustrating a process of synthesizing a foreground image and a background image composed of pixel values of two pixels. FIG. FIG.

In the conventional processing, the first input terminal 2600
, The foreground image shape signal 2500 is input, and the second, third
The color difference signal 2504 of the foreground image and the color difference signal 2506 of the background image are input to the input terminals 2602 and 2604 of.
In the image format used in MPEG4, the resolution of the chrominance signal is 解像度 of the resolution of the luminance signal or the shape signal, so the shape signal 2500 is converted by the resolution conversion means so that the resolution corresponds to the chrominance signal. 2606
Is subjected to resolution conversion.

Next, using the resolution-converted shape signal 2502, the input color difference signal 2504 of the foreground image and the color difference signal 2506 of the background image are synthesized by the pixel synthesis means 26.
08. Then, the synthesized color difference signal 2508
Since the resolution is different from the luminance signal as described above, during display, the synthesized color difference signal 2508 is converted by the resolution conversion unit 2610 so that the resolution becomes the same as the corresponding luminance signal. A synthesized color difference signal 2510 after the resolution conversion is output from an output terminal 2620.

Usually, the pixel value added by the resolution conversion is calculated based on the pixel value of the pixel adjacent to the pixel having the additional pixel value. Is compared with the pixel values of the pixels other than the portion (black) in the object in the shape signal 2500 before the resolution conversion, the above-described combined color difference signal 2510 indicates that the pixels outside the object are also within the object. It can be seen that the pixel has the pixel value of As a result, the image displayed based on the color difference signal has blurred image quality.

The degradation of the image quality due to the synthesis of the texture signal as described above is caused not only by the color difference signal but also by the synthesis processing of the luminance signal when the resolution is converted, for the same reason as described above, the reproduced image is at the boundary portion. There is a problem that the image quality is deteriorated.

The present invention has been made in order to solve the above-described problems, and an image synthesizing apparatus, an image synthesizing method, and an image synthesizing method capable of improving image quality at a boundary portion in a synthetic image. It is an object of the present invention to provide a data recording medium storing a program for realizing a combining process by software.

[0035]

An image synthesizing method according to the present invention (claim 1) forms a first image space including an object of an arbitrary shape, and a shape signal indicating the shape of the object and a picture of the object. The first image signal including the texture signal indicating the image and the second image signal forming the second image space and including the texture signal indicating the picture pattern of the image are combined to form a third image signal.
A method for generating a composite image signal that forms the image space of the above-mentioned pixel, wherein the arithmetic processing on the shape signal of the pixel located in the target area including the pixel to be processed in the first image space, A combining ratio calculating step of calculating a combining ratio with respect to the texture signal; and, based on the calculated combining ratio, the pixel to be processed and the corresponding pixel in the second image space corresponding to the pixel to be processed. A pixel synthesizing step of synthesizing a texture signal, wherein the synthesized texture signal is used as a texture signal of a pixel corresponding to the pixel to be processed in the third image space.

According to a second aspect of the present invention, in the image synthesizing method according to the first aspect, it is determined whether each pixel is located in the object based on a shape signal of each pixel in the first image space. A shape determining step of determining whether the pixel to be processed is located in the object and calculating the combining ratio using a shape signal of a pixel in the target area when the result of the determination is that the pixel to be processed is located in the object. Is what you do.

According to a third aspect of the present invention, in the image synthesizing method according to the first aspect, it is determined whether each pixel is located inside the object based on a shape signal of each pixel in the first image space. And determining the composition ratio.In the composition ratio calculation step, when the processing target pixel is located in the object, the composition ratio is calculated by averaging the shape signals of the pixels in the target area. It is to be calculated.

According to a fourth aspect of the present invention, in the image synthesizing method according to the first aspect, in the synthesizing ratio calculating step, the arithmetic processing on the shape signal is performed by a power of 2 or a power of 2 in the arithmetic processing. This is performed based on an arithmetic expression modified so as to be as follows.

According to the image synthesizing method of the present invention (claim 5), a first image space including an object of an arbitrary shape is formed.
A first image signal including a shape signal indicating the shape of the object and a texture signal indicating a picture of the object, and a second image signal including a texture signal indicating a picture of an image forming a second image space. A method of generating a combined image signal for forming a third image space by combining, wherein each pixel is located inside the object based on a shape signal of each pixel in the first image space. A shape determination step of determining whether or not the target pixel is located in the object, based on the number of pixels located in the object among the pixels in the target area, A combining ratio calculating step of calculating a combining ratio with respect to the texture signal; and, between the pixel to be processed and a corresponding pixel in the second image space corresponding thereto based on the calculated combining ratio. And a pixel composition step of performing composition processing of the texture signals of the pixels, the texture signals the synthetic, in the third image space,
This is used as a texture signal of a pixel corresponding to the pixel to be processed.

In the image synthesizing method according to the present invention (claim 6), a first image space including an object of an arbitrary shape is formed.
A first image signal including a shape signal indicating the shape of the object and a texture signal indicating a picture of the object, and a second image signal including a texture signal indicating a picture of an image forming a second image space. A method of generating a combined image signal for forming a third image space by combining, wherein each pixel is located inside the object based on a shape signal of each pixel in the first image space. A shape determining step of determining whether or not the pixel is to be processed, and a synthesis ratio of the pixel to be processed to a texture signal is calculated based on a shape signal of a pixel located in a target area including the pixel to be processed in the first image space. A combining ratio calculation step, and as a result of the shape determination, when the pixel to be processed is located outside the object, a texture signal of the pixel to be processed is converted into a texture signal of a pixel in the target area. A pixel information generating step of replacing the texture signal obtained by the averaging, and, based on the calculated combination ratio, the pixel information between the pixel to be processed and the corresponding pixel in the second image space corresponding thereto. A pixel synthesizing step of synthesizing a texture signal of the pixel of the pixel, the synthesized texture signal,
It is used as a texture signal of a pixel corresponding to the pixel to be processed in the third image space.

According to a seventh aspect of the present invention, in the image synthesizing method according to the sixth aspect, in the synthesizing ratio generating step, an average value of shape signals of pixels included in the target area is calculated as the synthesizing ratio. It is.

According to the present invention (claim 8), in the image synthesizing method according to claim 6, in the pixel information generating step, the texture signal of the pixel to be processed is included in an object among the pixels included in the target area. This is to replace the average value of the texture signal of the located pixel.

According to a ninth aspect of the present invention, in the image synthesizing method according to the fifth or sixth aspect, in the combining ratio calculating step, the number of pixels located in the object among the pixels in the target area is determined. The composition ratio calculated in advance and stored in the table is obtained from the table according to the number of pixels in the object obtained from the shape signal of the pixels located in the target area.

According to a tenth aspect of the present invention, in the image synthesizing method according to the first or sixth aspect, in the pixel synthesizing step, both of the combination of the combination ratio value and the texture signal value are performed. A texture signal of a pixel corresponding to the pixel to be processed in the third image space is obtained with reference to a table storing the result of the multiplication.

According to a fifth aspect of the present invention, in the image synthesizing method according to the fifth or sixth aspect, in the pixel synthesizing step, the number of pixels located in the object in the target area and the value of the texture signal value are adjusted. A texture signal of a pixel corresponding to the pixel to be processed in the third image space is obtained with reference to a table storing the result of multiplication of both of the combinations.

An image synthesizing method according to the present invention (claim 12) includes a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape. A first decoded image signal;
And a second decoded image signal including a texture signal indicating a picture pattern of the image forming the third image space to generate a synthesized image signal forming the third image space. A combining ratio calculating step of calculating a combining ratio of the pixel to be processed to a texture signal by performing an arithmetic process on a shape signal of a pixel located in a target region including the pixel to be processed in the image space; And a pixel synthesizing step of synthesizing a texture signal of the pixel between the pixel to be processed and a corresponding pixel in the second image space corresponding to the pixel to be processed. When generating the coded image signal corresponding to the first decoded image signal, referring to the additional information added to the coded image signal, the text in the synthesis processing is referred to. It is intended to switch the synthesis of Sucha signal.

According to a thirteenth aspect of the present invention, in the image synthesizing method according to the twelfth aspect, the additional information is information indicating a size of the target area.

The image synthesizing method according to the present invention (claim 14) includes a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape. A first image signal is combined with a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space. A method for detecting a boundary position of the object based on a shape signal of each pixel in the first image space; and detecting a boundary position in the first image space including a pixel to be processed in the first image space. Based on the shape signal of the pixel located in, the process of calculating the synthesis ratio of the pixel to be processed to the texture signal, near the boundary of the object, only for pixels located inside and outside the object And calculating the combination ratio between the pixel to be processed and the corresponding pixel in the second image space corresponding to the pixel based on the calculated combination ratio. Of the pixel other than the vicinity of the boundary, based on the shape data of these pixels, the pixel to be processed and the corresponding second image space. And a pixel synthesizing step of performing a process of synthesizing texture signals of these pixels with the corresponding pixels in the third step.
Is used as a texture signal of a pixel corresponding to the pixel to be processed in the image space.

The image synthesizing method according to the present invention (claim 15) includes a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape. A first image signal is combined with a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space. The method, based on the size of the object and additional information indicating the position of the object in the first image space, for a specific pixel located inside or outside the object and near the boundary of the object, A process of synthesizing a texture signal of these pixels is performed between a pixel to be processed in the first image space and a corresponding pixel in the second image space corresponding to the pixel to be processed. Oak For peripheral pixels other than the specific pixel, the texture signal of the pixel in the second image space corresponding to these peripheral pixels is used as the texture signal of the pixel corresponding to the peripheral pixel in the third image space. A texture signal obtained by the synthesis processing in the pixel synthesis step is used as a texture signal of a pixel corresponding to the specific pixel in the third image space including a pixel synthesis step.

The image synthesizing method according to the present invention (claim 16) includes a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape. A first image signal is combined with a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space. In the method, when the resolution of the texture signal constituting the first image signal is not equal to the resolution of the shape signal, a resolution conversion process is performed on the texture signal so that the resolution becomes equal to the resolution of the shape information. A resolution converting step, and an image synthesizing step of synthesizing the first image signal and the second image signal using the texture signal subjected to the resolution conversion processing and the shape signal. Tsu is intended to include and-flops.

An image synthesizing method according to the present invention (claim 17) includes a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape. A first image signal is combined with a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space. A resolution of the texture signal forming the first image signal, a resolution of the texture signal forming the second image signal, and a display resolution when displaying the texture signal forming the composite image signal. A resolution changing step of performing a resolution conversion process on the texture signals of the first and second image signals so that the resolution is equal to the display resolution; The first subjected to Zodo conversion process,
An image synthesizing step of synthesizing the first image signal and the second image signal using the texture signal of the second image signal.

An image synthesizing method according to the present invention (claim 18) includes a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape. A first image signal is combined with a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space. A method for calculating a synthesis ratio of a pixel to be processed to a texture signal based on a shape signal of a pixel located in a target area including the pixel to be processed in the first image space; A pixel for performing a texture signal synthesis process of the pixel between the pixel to be processed and a corresponding pixel in the second image space corresponding to the pixel based on the calculated synthesis ratio. And a forming step, a texture signal which is the synthetic, in the third image space,
A first image synthesizing process that is used as a texture signal of a pixel corresponding to the pixel to be processed, and a process between a pixel to be processed in the first image space and a corresponding pixel in the second image space corresponding to the first image combining process. A pixel synthesizing process of synthesizing the texture signal of the pixel at a ratio different from the synthesizing ratio in the first image synthesizing process, and combining the synthesized texture signal in the third image space. A second image synthesis process used as a texture signal of a pixel corresponding to the pixel to be processed,
The first and second image synthesizing processes are switched based on a shape signal of a pixel located in a target region including a pixel to be processed in the first image space.

According to a nineteenth aspect of the present invention, in the image synthesizing method according to the eighteenth aspect, the second image synthesizing process is performed in the pixel synthesizing step. Is used as the texture signal of the pixel corresponding to the pixel to be processed in the third image space, and the texture signal in the second image space corresponding to the pixel to be processed outside the object in the first image space is used. In this processing, a texture signal of a corresponding pixel is used as a texture signal of a pixel corresponding to the pixel to be processed in the third image space.

An image synthesizing apparatus according to the present invention (claim 20) includes a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape. A first decoded image signal;
And a second decoded image signal including a texture signal indicating a picture pattern of the image forming the third image space to generate a synthesized image signal forming the third image space. Combining ratio calculating means for calculating a combining ratio of a pixel to be processed to a texture signal by performing an arithmetic process on a shape signal of a pixel located in a target region including the pixel to be processed in the image space; And a pixel synthesizing unit for synthesizing a texture signal of these pixels between the pixel to be processed and the corresponding pixel in the second image space corresponding to the pixel to be processed. When generating the encoded image signal corresponding to the first decoded image signal, the texture signal in the synthesizing process is referred to by referring to the additional information added to the encoded image signal. Those configured to switch the synthesis method.

The image synthesizing apparatus according to the present invention (claim 21) includes a shape signal indicating the shape of the object and a texture signal indicating a picture of the object, forming a first image space including the object having an arbitrary shape. A first image signal is combined with a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space. A boundary detection means for detecting a boundary position of the object based on a shape signal of each pixel in the first image space;
In the image space of, based on the shape signal of the pixel located in the target region including the pixel to be processed, the processing of calculating the synthesis ratio of the pixel to be processed to the texture signal, near the boundary of the object, And the composition ratio calculating means for performing only on the pixels located in the object, and for the pixels near the boundary, based on the calculated composition ratio,
A process of synthesizing texture signals of these pixels is performed between the pixel to be processed and the corresponding pixel in the second image space corresponding to the pixel to be processed. A pixel synthesizing unit that performs a process of synthesizing a texture signal of the pixel between the pixel to be processed and a corresponding pixel in the second image space based on the shape data of the pixel; The synthesized texture signal is used as a texture signal of a pixel corresponding to the pixel to be processed in the third image space.

The image synthesizing apparatus according to the present invention (claim 22) includes a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape. A first image signal is combined with a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space. The device, based on the size of the object and additional information indicating the position of the object in the first image space, for a specific pixel located inside or outside the object and near the boundary of the object, A process of synthesizing a texture signal of these pixels is performed between a pixel to be processed in the first image space and a corresponding pixel in the second image space corresponding to the pixel to be processed. Oak For peripheral pixels other than the specific pixel, the texture signal of the pixel in the second image space corresponding to these peripheral pixels is used as the texture signal of the pixel corresponding to the peripheral pixel in the third image space. A pixel synthesizing unit is provided, and a texture signal obtained by the synthesizing process in the pixel synthesizing step is used as a texture signal of a pixel corresponding to the specific pixel in the third image space.

The image synthesizing apparatus according to the present invention (claim 23) includes a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape. A first image signal is combined with a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space. A resolution conversion process that, when the resolutions of the texture signal and the shape signal constituting the first image signal are not equal to each other, make the resolution of the texture signal equal to the resolution of the shape information; Resolution converting means, and image synthesizing means for synthesizing the first image signal and the second image signal using the texture signal subjected to the resolution conversion processing and the shape signal. It includes those were.

The image synthesizing apparatus according to the present invention (claim 24) includes a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape. A first image signal is combined with a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space. A resolution of a texture signal constituting the first image signal, a resolution of a texture signal constituting the second image signal, and a display resolution when displaying the texture signal constituting the composite image signal. A resolution changing unit configured to perform a resolution conversion process on the texture signals of the first and second image signals so that the resolution is equal to the display resolution; Subjected to conversion processing said first, second
Image synthesizing means for synthesizing the first image signal and the second image signal using the texture signal of the image signal.

The image synthesizing apparatus according to the present invention (claim 25) includes a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape. A first image signal is combined with a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space. A combination ratio calculating means for calculating a combination ratio of a pixel to be processed to a texture signal based on a shape signal of a pixel located in a target region including the pixel to be processed in the first image space; A first image for performing a texture signal synthesis process of the pixel between the pixel to be processed and a corresponding pixel in the second image space corresponding to the pixel based on the calculated synthesis ratio. A first image synthesizing processing unit that has a synthesizing unit, and outputs the synthesized texture signal as a texture signal of a pixel corresponding to the pixel to be processed in the third image space; The processing of synthesizing the texture signals of these pixels between the pixel to be processed in the image space and the corresponding pixel in the second image space is performed at a ratio different from the synthesis ratio in the first image synthesis processing. A second image synthesizing processing unit having a second pixel synthesizing unit that performs a predetermined process, and outputs the synthesized texture signal as a texture signal of a pixel corresponding to the pixel to be processed in the third image space; Wherein the first and second image synthesizing processes are cut based on shape signals of pixels located in a target area including a pixel to be processed in the first image space. One in which was configured to obtain.

The data storage medium according to the present invention (claim 26) is a data storage medium storing a program for causing a computer to perform an image synthesizing process. An image synthesizing program for causing a computer to perform an image synthesizing process according to any one of the image synthesizing methods is stored.

[0061]

Embodiments of the present invention will be described below. Embodiment 1 FIG. The image synthesizing method according to the first embodiment of the present invention is a method for receiving image signals corresponding to images of a plurality of objects, and performing image signal synthesizing processing with reference to a shape signal corresponding to each image. After performing a smoothing process on the shape data (pixel value of the shape signal) of a pixel located inside the boundary of the image, that is, within the object and adjacent to the image boundary, performing a synthesizing process on the image signal of each of the above objects And
Thereby, the image quality of the composite image can be improved.

FIG. 1 is a diagram showing a flow of processing in the image synthesizing method according to the first embodiment of the present invention. FIG. 2 shows FIG.
FIG. 5 is a diagram schematically showing a specific process in step 410 shown in FIG. In FIG. 2A, reference numeral 302 denotes shape data corresponding to one line of pixels 300 at the boundary portion of the object shown in FIG. 25. The horizontal axis represents the position of a pixel in the image, and the vertical axis represents the value of the pixel shape data. Are shown on the coordinates. Here, the shape data is multivalued data that takes any value from 0 to 255.

Hereinafter, of the shape data 302, boundary pixels (pixels to be processed) located in the object adjacent to the object boundary
A case in which pixel value conversion processing is performed on the shape data 500 of FIG. First, unprocessed image determination step 4
02, it is determined whether or not unprocessed pixels remain.
In 4, foreground pixels to be processed (pixels to be processed)
Enter the shape data. If no unprocessed pixels remain, the combining process ends (step S414).

Next, in the shape data determination step 406, the shape data 500 of the foreground pixel to be processed is 0
Determine if it is a value. If the result of the determination is that the shape data is a zero value, in a background pixel output step 408, the pixel value (texture data) of the background pixel corresponding to the target foreground pixel position is output as the texture data of the composite pixel. Then, the synthesizing process for the target foreground pixel ends. Here, when the output destination of the texture data of the synthesized pixel is the storage unit of the texture data of the background pixel, the process of step 408 is not performed.

Subsequently, as a result of the determination in the step 406, if the shape data 500 of the target foreground pixel is a non-zero value, the shape data calculation step (composition ratio calculation step) 410 The shape data of the foreground pixel to be processed is calculated by a predetermined method using the shape data of all pixels in the area, that is, the area of a predetermined size including the target pixel to be processed (target area). here,
The target region is a region surrounded by a thick black line including a boundary portion 500 of the image of the object shown in FIG. 2B, and a region 502 composed of 3 × 3 pixels where the target pixel is located at the center.
It is.

The arrangement of the pixels included in the target area is not limited to 3 × 3 pixels, but may be a pixel arrangement represented by M × N (M and N are natural numbers). It is assumed that the pixel arrangement (vertical and horizontal size, number of pixels, etc.) in the target area may be changed according to the image. Further, in the processing of the luminance data and the color difference data constituting the texture data, the pixel arrangement in the target area may be different.

As a specific method for calculating the shape data, the shape data of the pixel to be processed is obtained by averaging the shape data of nine pixels in the target area 502. As a result, the shape data 504 of the pixel to be processed 506 becomes the level 50 as shown in FIG.
4 becomes smaller. In such a method, the target area 500
By calculating the shape data of the pixel to be processed in consideration of the shape data of the pixels in the above, the change in the shape data of a plurality of pixels arranged on both sides of the image boundary of the object can be moderated.

Lastly, in the pixel synthesizing step 412, the shape data 504 calculated as described above is used as a synthesizing ratio used in synthesizing the texture data, and the shape data 504 is used for the target foreground pixel and background pixel texture data. Then, the weighted averaging process is performed using Expression 1 above, and the texture data synthesized between the foreground image and the background image is output as the texture data of the pixel of the synthesized image corresponding to the target pixel.

According to the first embodiment as described above, only the shape data of the pixel located adjacent to the boundary inside the image of the object is used, and the distribution of the shape data of the pixel near the boundary is determined. Smooth the shape data to make it smooth,
The shape data obtained by this smoothing is used as the synthesis ratio, and the texture data is synthesized between the foreground pixel located near the inside of the boundary and the corresponding background pixel. The image quality at the boundary between the foreground image and the background image can be improved. In addition,
In the first embodiment, even if the shape data is multi-valued data that randomly takes any value from 0 to 255,
It may be binary data having either a zero value or a non-zero value.

For example, when the shape data is binary data having either a 0 value or a non-zero value,
As shown in a table 508 in FIG.
The number of pixels whose shape data has a non-zero value among the pixels in
A value (count / range-co) obtained by dividing the number of pixels having a non-zero value (count) by the number of pixels in the target area 502 (range-count)
unt) multiplied by non-zero value shape data (alpha-value)
The shape data of the pixel to be processed can also be calculated. Here, the alpha-value used in the expression of Table 508 is 255 in the example of FIG. 2, but is not limited to this and may take an intermediate value within the range of 0 to 255. And When the shape data is multivalued data as described above, an average of the shape data of each pixel in the target area may be used as the shape data of the pixel to be processed.

Embodiment 2 FIG. 3 is a diagram for explaining an image synthesizing method according to the second embodiment, and schematically shows processing of a synthesizing ratio calculation step 410 in this synthesizing method. The image synthesizing method according to the second embodiment includes a synthesizing ratio calculating step 4 in the image synthesizing method according to the first embodiment.
At the time of calculating the shape data in 10, additional processing is performed so that the distribution of the composition ratio of the pixels located near the boundary portion becomes smoother. The other processing is exactly the same as the processing procedure described in the first embodiment, and a description thereof will not be repeated.

In FIG. 3A, 300, 302, 50
Reference numerals 0 and 502 are the same as those shown in FIGS. In the combination ratio calculation step 410 according to the second embodiment, a process of changing the combination ratio (shape data) of the pixels located within the boundary of the object is performed. The pattern of the pixel array in the target area (the area composed of 3 × 3 pixels) including the pixel to be processed is shown in FIG.
In most cases, in addition to the pattern of the target region 502 composed of × 3 pixels, the pattern of the pixel array 610, 612, or 614 shown in FIG.

For a target area having such a pixel array pattern, the number of pixels whose shape data is non-zero is calculated, and if the calculated number is a certain number or less, the number of pixels to be processed is determined. If the shape data of the pixel is set to 0 and otherwise (when the number of calculations exceeds a certain number), the shape data of the pixel to be processed is changed from 0 to the maximum value of the shape data (FIG. 3).
In (b), conversion is performed to take the intermediate value of 255).

More specifically, the synthesis ratio (shape data) of the pixel to be processed is calculated using the conditional expression shown in Table 608 of FIG. According to the conditional expression shown in Table 608, when the number of pixels having non-zero value shape data in the target region including the pixel to be processed indicates a value equal to or smaller than the threshold,
The synthesis ratio (shape data) of the pixel to be processed is set to 0 value. In other cases, the number of pixels having non-zero value shape data included in the target region and all pixels included in the target region Is used to calculate the combination ratio so that the shape data of the pixel to be processed becomes an intermediate value from 0 to a non-zero value (255 in FIG. 3B). With this conditional expression, the value of the combination ratio corresponding to the pixel located at the boundary of the image of the object can be changed, and the image quality of the boundary portion of the foreground image in the composite image obtained by superimposing the foreground image and the background image can be improved. be able to.

The threshold value used in the conditional expression shown in Table 608 is changed depending on the size of the target area. The alpha_value used in the conditional expression of Table 608 is 2 in the example of FIG.
It is 55, but may take an intermediate value from 0 to 255.

According to the second embodiment, the number of pixels whose shape data is non-zero in the target area including the boundary of the object is determined, and the shape data of the pixel located at the boundary of the object is determined. On the other hand, the foreground and the background are synthesized using the obtained synthesis ratio, and the foreground image and the background image in the synthesized image are processed so that the change of the value (that is, the value of the synthesis ratio) becomes smoother. Can further improve the image quality of the boundary portion.

Embodiment 3 FIG. 4 is a diagram for explaining an image synthesizing method according to the third embodiment of the present invention, and shows a flow of processing of the image synthesizing method according to the third embodiment. FIG.
(a) to FIG. 5 (e) show the shape information generation step 71 shown in FIG.
FIG. 7 is a diagram schematically illustrating processing in a pixel information generation step 714 of FIG.

The image synthesizing method according to the third embodiment is as follows.
In a method of receiving image signals of a plurality of objects and performing a synthesizing process of the image signals, a synthesis ratio with respect to a pixel at a boundary portion of the foreground image is obtained by a smoothing process with reference to shape data of the boundary pixel, and thus obtained. Using the combined ratio
The texture signal is synthesized between the foreground image and the background image. At this time, the texture data of the pixel outside the boundary of the foreground image is generated using the texture data of the pixel inside the boundary, thereby improving the image quality of the synthesized image. It is intended.

In FIG. 5, reference numeral 302 denotes one line of pixels including the boundary of the region 300 shown in FIG. 25. The shape data of the boundary pixel 306 located outside the object boundary within the one line of pixels 302 is shown. The case of performing the conversion process will be described. FIG. 5D shows the region 300, that is, the region 800 in the texture space corresponding to the region including the object boundary in the shape space.

First, an unprocessed pixel determination step 70
In 2, it is determined whether there is an unprocessed pixel. If there is, in a pixel input step 704, a target area centered on the target foreground pixel (described as a predetermined area in FIG. 4) is determined. The pixel texture data and shape data are input. Here, reference numeral 306 denotes a shape pixel to be processed (pixel to be processed), and reference numeral 804 denotes a texture pixel (a pixel in a texture space) corresponding to the shape pixel.
The target area is an area of a predetermined size in the shape space and the texture space centered on the pixels 306 and 804. FIGS. 5B and 5D show these areas 80.
2,806 is shown as a region composed of 3 × 3 pixels surrounded by a thick line. Note that this area is not limited to an area composed of 3 × 3 pixels, but may be an area composed of M × N (M and N are natural numbers) pixels. (Size, number of pixels, etc.) may be changed. Further, in the processing of the luminance data and the chrominance data constituting the texture data, the size of the predetermined area may be different depending on the case.

Next, in the shape information determination step 706,
It is determined whether any of the pixels in the target area 802 centered on the foreground pixel to be processed has non-zero value shape data. If none of the pixels has shape information of a non-zero value, in the background pixel output step 708, the texture data of the background pixel corresponding to the foreground pixel 804 to be processed is output as the texture data of the composite pixel. Here, if the output destination of the texture data of the synthesized pixel is a portion storing the texture data of the background pixel, the processing in step 708 is not performed.

On the other hand, if any of the pixels in the target area 802 has non-zero value shape data, in the combining ratio calculation step 710, the input shape data of all the pixels in the target area 802 is deleted. Then, the sum is divided by the number of pixels in the target area 802 (9 in FIG. 5).
The composition ratio as a result of the division (shape data 8 in FIG. 5 (c))
08) is used as a weighting coefficient when synthesizing texture data between the foreground image and the background image in a pixel synthesizing step 716 described later.

In the following shape information determination step 712,
It is determined whether or not the shape data of the pixel to be processed 306 is 0 value. If the shape data of the pixel to be processed 306 in the shape space is 0 value, the texture data of the pixel (corresponding texture pixel) 804 in the texture space corresponding to the pixel to be processed is indeterminate. Then, the texture data of the texture pixel corresponding to the shape pixel whose shape data is a non-zero value in the predetermined region 806 of the texture space (the region corresponding to the target region of the shape space) is added. Then, the average value obtained by averaging the sum of the texture data with the number of texture data pixels whose shape data of the corresponding shape pixel is a non-zero value is used as the texture data of the texture pixel 804. Here, the texture pixel whose shape data of the corresponding shape pixel has a non-zero value is two pixels with a pattern in the predetermined area 806 shown in FIG. 5D, and the sum of the texture data is divided by the texture pixel. The obtained value is used as texture data of the corresponding texture pixel 804. FIG. 5E shows the value (the above average value) 820 of the texture data of the corresponding texture pixel 804.

On the other hand, if the shape data of the shape pixel (pixel to be processed) 306 to be processed is a non-zero value, there is texture data in the corresponding target texture pixel, so the process of step 714 is performed. Absent. further,
If the texture data of the corresponding texture pixel has been supplemented with any value in advance, step 714
Is not performed. For example, if the value of the encoding distortion included in the decoded texture signal is held and the value is not far from the value of the adjacent pixel, the process of step 714 may not be performed. .

Finally, in the pixel synthesizing step 716, the foreground pixel to be processed and the background pixel to be processed are determined by using the synthesis ratio calculated in step 710 (shape data 808 of the pixel to be processed shown in FIG. 5C) as a weighting coefficient. The texture data is synthesized between the pixels. The synthesis is performed using the above (Equation 1). In FIG. 5, after the shape information calculation step 710 of the target pixel, the processing in the determination step 712 of the shape data of the pixel to be processed is performed, but these processing may be performed in parallel. .

As described above, in the third embodiment, the texture data and the shape data are smoothed for pixels outside the object located at the boundary of the object. It may be blurry,
As in the first and second embodiments, the value of shape data of pixels near the boundary in the object is not reduced, so that the size (vertical and horizontal size, the number of pixels, etc.) of the object is very small. Can combine an image of a foreground object with a background image while avoiding it from shrinking. Therefore, the combination processing of the first and second embodiments and the combination processing of the third embodiment may be switched and used according to the input image. Further, since the texture data is composed of the luminance data and the chrominance data, the switching between the synthesizing processing of the first and second embodiments and the synthesizing processing of the third embodiment is performed independently with the luminance data and the chrominance data. It may be.

In the third embodiment, even if the shape data is multi-valued data that randomly takes any value from 0 to 255, the shape data has either 0 value or non-zero value. It may be value data. In addition, the first embodiment
Although all the synthesizing methods shown in 3 are performed using Expression 1, Expression 2 can also be used for the above-described synthesis processing.
In this case, assuming that the calculated combination ratio (that is, obtained using the shape data of the pixels located in the target region including the pixel to be processed) is β, β / alpha ′ is multiplied by the foreground value fgpel ′. Thus, a similar process can be performed.

Embodiment 4 The image synthesizing method according to the fourth embodiment of the present invention is configured to speed up the processing of the synthesizing ratio calculation step shown in the first to third embodiments, and the other processing procedures are the same as in the first to third embodiments. This is the same as the processing procedure described in 3. In the synthesizing methods performed in the first to third embodiments, the synthesizing ratio is obtained by calculation. However, in the case where the shape data of the pixel to be processed is a binary value of 0 value and non-zero value, FIG. The conversion formula in Table 508 shown in FIG.
Based on the conversion formula in Table 608 shown in (c), the shape data can be obtained after calculating the number of shape pixels having a non-zero value. Since this calculation involves division and multiplication, the amount of calculation becomes extremely large when processing is performed for each pixel.

In the fourth embodiment, an area composed of 3 × 3 pixels surrounded by a thick black line in FIG. 6 is set as a target area (predetermined area), and the number of pixels having non-zero values as shape data is determined. The combination ratios corresponding to (0-9) are calculated in advance and stored in the table 900.

In the actual calculation, the number of pixels whose shape data is non-zero in the target area is first calculated, and based on this number of pixels, the corresponding combination ratio is referred from the table. By (applying), division and multiplication can be reduced. Here, Embodiments 1 and 3
Although a table that takes a value corresponding to the combining method is described as an example, the same processing can be performed in the second embodiment.

Embodiment 5 In the image synthesizing method according to the fifth embodiment, not only the synthesizing ratio but also a value obtained by multiplying the synthesizing ratio and the pixel information is stored as a table. Can be simplified. In general, pixels often take the same combination ratio (shape data) value at the boundary portion of an object, and the same applies to texture data.

Therefore, the multiplication value of the composition ratio and the texture data is previously calculated and stored in a table, and the values of the table are referred to based on the combination ratio and the value of the texture data, thereby performing redundant division and multiplication. Processing can be reduced. For the above equation 1, FIG.
Is performed, and the multiplication process in the expansion formula is replaced with a process that refers to the table 1000, whereby the amount of calculation can be reduced.

Further, when the composition ratio of the pixel to be processed is obtained from the number of pixels having non-zero-valued shape data, the above equation 1 is expanded as shown in FIG. Is stored in the table 1100, the composition ratio of the pixels to be processed can be calculated from the number of pixels having non-zero value shape data in the target area and the value of the texture data. In this case, the number of indexes in a table prepared in advance can be reduced.

For example, rather than using an array of “texture values of pixels” and “synthesis ratio” as an index in the above table, it is located in an object in a target area including a “texture value of pixels” and a “pixel to be processed”. Using the array of "the number of pixels" reduces the number of arrays as indices. Specifically, when the texture value of the pixel and the combination ratio (shape data) take values from 0 to 255, using an array of “texture value of pixel” and “composition ratio”, 256 × 256 = 65536 When an array of “texture values of pixels” and “number of pixels” is used, even if the texture data of pixels takes 256 values, the number of pixels in the object is 0 to 0. Since there are ten to nine, the number of index arrays may be 256 × 10 = 2560.

The pixel information or the pixel value of the pixel in FIGS. 7 and 8 means texture data, and the shape information means shape data. Also in the above equation 2, by referring to the table for the term of the term to be multiplied by the background pixel value (texture data),
The amount of calculation can be reduced in the same manner as in the above equation 1. Further, it goes without saying that the combining process using the tables as in the fourth and fifth embodiments can also be applied to the combining method in the second embodiment.

Embodiment 6 FIG. FIG. 10 is a block diagram for explaining a decoding / synthesizing apparatus according to Embodiment 6 of the present invention. This decoding / synthesizing apparatus encodes an image signal corresponding to each object to which additional information indicating whether or not to perform a smoothing process of shape data or the like on a boundary pixel as described in the first to fifth embodiments is added. The encoded data obtained by the decoding is decoded.

FIG. 9 shows a data structure of a bit stream which is encoded data to which the additional information 1250 has been added. In the bit stream 1200 having this data structure, first, there is header information 1204 that describes the number of data included in the data structure, etc., then there is additional information 1250, and then there is an object unit. Encoded data 1206, 1208 for each time,
1210 and 1214 are arranged in order.

The encoded data 1206 is divided into header information 1218 describing information for decoding the data, encoded shape data (shape information) 1220, and encoded texture data (pixel information) 1222. .
Here, the shape data is coded by an arithmetic coding method such as JBIG and further subjected to variable length coding, and the texture data is coded by a coding method such as DCT and further subjected to variable length coding.

In the bit stream 1200 having this data structure, when combining the header information 1204 with the portion described in the first to fifth embodiments,
Additional information (Blend flag) 12 indicating whether to smooth texture data or shape data at the boundary portion
50 is added by 1 bit. This makes it possible to switch whether or not to perform the image synthesis processing as described in the first to fifth embodiments for each object. The header information indicates the vertical and horizontal sizes (for example, 3 × 3 pixels) and the number of pixels of the target area (predetermined area) described in each of the embodiments.

Further, by adding a plurality of bits of the additional information, it is possible to easily realize the above-described smoothing processing method and information required for the processing. For example, by adding the plane pattern of the target area and the like shown in the first to fifth embodiments as additional information, the plane pattern (the vertical and horizontal size of the area, the number of pixels, etc.) of the target area is changed for each image to be synthesized. can do.

Further, additional information (Blend flag) 1250
Are inserted between the header information 1204 and the encoded data 1206, but each of the encoded data 1206, 1208,
By inserting the data in the middle of 1210 and 1214, the processing can be switched from the data in the middle.

Further, the encoded data 12 for each unit time
Similar additional information (Blend flag) is also included in each header information 1218 included in each of 06, 1208, 1210, and 1214.
By adding 1250, it is also possible to perform the combining process as described in Embodiments 1 to 5 only on encoded data at a specific time.

Although the above-mentioned additional information is described in the bit stream 1200 of the texture data, it may be given as side information separate from the bit stream 1200.

Next, a decoding / synthesizing apparatus according to the sixth embodiment will be described. The decoding / synthesizing apparatus 1400 according to the sixth embodiment.
Receives the bit stream 1200 and separates the encoded data of the header information, the shape data, and the texture data included in the bit stream 1200 from the encoded data. And a combining processing unit 1408 having a plurality of combining units and combining the coded data by a predetermined combining unit based on additional information included in the header information.

Here, the synthesizing processing unit 1408 includes a first selection switch 1412 for selecting, from the plurality of synthesizing units, a synthesizing unit that supplies the output of the image decoding unit 1404 based on the additional information. A second switch 1414 for selecting and outputting one of the outputs of the plurality of combining means based on the additional information. Further, at least one of the plurality of synthesizing units in the synthesizing processing unit 1408 is configured to perform the synthesizing process by the image synthesizing method according to any of the first to fifth embodiments.

Next, the operation will be described. Input terminal 14
When the bit stream 1200 is input from 00a,
This bit stream 1200 is transmitted to the data separation unit 14.
At 02, the data is divided into encoded data such as shape data and pixel data. Each of the divided encoded data is supplied to a decoding unit 1
Decoded at 404. On the other hand, the bit stream 1
The additional information 1250 included in the
08, and switches 1412 and 1414 select a combining unit using a corresponding combining method.

Here, a plurality of decoding means 1404 may be provided, and decoding processing may be performed in parallel on bit streams for a plurality of input objects. Background pixel information that has already been decoded, generated, or stored is
The signal is input from the second input terminal 1406, and
08. The synthesis processing unit 1408 performs a synthesis process of a foreground image and a background image by a synthesis method corresponding to each pixel data and shape data decoded by the image decoding unit 1404. The synthesized data is output from an output terminal 1410.

Next, a decoding / synthesizing apparatus according to a modification of the sixth embodiment will be described with reference to FIG. The decoding / synthesizing device 1500 of this modification includes the additional information 1250
The image synthesis is performed by decoding a bit stream, which is used in the synthesis method of each of the above embodiments and indicates a plane pattern of a target area and to which other additional data necessary for synthesis is added.

That is, the decoding / synthesizing device 1500 receives the first bit stream corresponding to the foreground image, separates the encoded data obtained by encoding the shape data and the texture data from the bit stream, and performs a synthesizing method. And a data separating unit 1502 for separating additional information necessary for performing synthesis, and an image decoding unit 1504 for decoding encoded data output from the separating unit.

Further, the decoding / synthesizing device 1500 includes a synthesizing method determining unit 1506 for calculating information necessary for synthesizing using the additional information, and a decoding method for determining the decoded shape data, texture data, and synthesizing method 1506. Based on the information of the input synthesis method, the second input terminal 15
An image synthesizing unit 1508 for synthesizing the texture data of the foreground image and the texture data of the background, which has already been decoded, generated, or held, input from the image processing unit 12.

Here, the image synthesizing means 1508 is configured to perform the synthesizing process by any one of the image synthesizing methods according to the first to fifth embodiments as the image synthesizing process.

Next, the operation will be described. Input terminal 15
When the bit stream 1200 is input from 00a, the data separating unit 1502 divides the encoded data obtained by encoding the shape data and the texture data from the bit stream, and performs a combining method and a combining method. In this case, additional information required is separated. Then, the additional information is input to the combining method determining unit 1506. On the other hand, the divided encoded data is decoded by the image decoding unit 1504.

The synthesizing method determining means 1506 determines
Information necessary for the composition is calculated using the input additional information. In the pixel synthesizing unit 1508, the decoded shape data and texture data, the synthesizing method determining unit 1
506 and the second input terminal 1
The composition processing is performed using the background texture data that has been input from 512 and has already been decoded, generated, or held. The synthesized data is output from output terminal 1510.

Embodiment 7 FIG. 12 to 15 are diagrams for explaining an image synthesizing method according to the seventh embodiment of the present invention. In the image synthesizing method according to the seventh embodiment, when synthesizing a texture signal between a foreground image and a background image, the image signal is synthesized with reference to additional information indicating the shape of an object in the foreground. This is to reduce the arithmetic processing.

FIG. 12 conceptually shows a process of synthesizing a background image and a foreground image. In the figure, 1600 is a background image screen (background texture space) based on a background texture signal, and 1602 is a foreground image based on a foreground texture signal. A screen (foreground texture space) 1604 is a foreground shape screen (foreground shape space) based on a foreground shape signal.
Background image screen 100, foreground image screen 10 in FIG.
2, the same as the foreground shape screen 104.

In FIG. 12, the proportion occupied by objects in the foreground image screen 1602 is less than half of the whole. In such a case, only the pixels included in a region 1606 surrounding the object (hereinafter, also referred to as an object region) or pixels outside the object adjacent to the object region 1606 need to be determined in the synthesis processing. Background image 1 for pixels other than
600 texture data (values corresponding to each pixel of the texture signal) may be used as it is.

Accordingly, the vertical and horizontal sizes 1608 and 1610 of the object area 1606 obtained from the shape signal forming the foreground image screen 1604 and the vertical and horizontal dimensions indicating the distance of the object area 1606 from the upper left of the foreground shape screen 1604 are displayed. The information indicating displacements 1612 and 1614 of the foreground image screen 1602 is received as additional information at the time of the synthesis processing, and the synthesis processing is performed only for the periphery of the rectangular area 1606 including the object in the foreground image screen 1602. Is determined whether it is necessary.

FIGS. 13 and 15 are block diagrams showing the arrangement of an apparatus for performing an image synthesizing process by the synthesizing method according to the seventh embodiment. An image synthesizing device 170 according to the seventh embodiment shown in FIG.
0 receives the first bit stream corresponding to the foreground image, separates the encoded data obtained by encoding the shape data and the texture data from the bit stream, and separates the additional information necessary for performing synthesis. Data separating means 1702 for decoding the encoded data output from the separating means, and additional information obtaining means 1706 for obtaining and outputting the separated additional information. I have.

The decoding / synthesizing apparatus 1700 includes a pixel combination unit 1710 in a rectangular area for synthesizing a texture signal corresponding to a pixel in the object area between a background image and a foreground image. A combination processing unit 1708 that combines the foreground image and the background image based on the additional information from 1706 is provided.

Here, the image synthesizing means 1710 is configured to perform the synthesizing process by any one of the image synthesizing methods according to the first to fifth embodiments as the image synthesizing process. Further, the synthesizing unit 1708 transmits the decoded data from the image decoding unit 1704 to the synthesizing unit 1710.
Switch 1716 for turning on and off the supply of the image data based on the additional information from the additional information acquiring unit 1706, and selecting one of the texture signal of the background image and the output data of the synthesizing unit 1710 based on the additional information. A switch 1718.

Next, the operation will be described. An input terminal 1700a of the image synthesizing device 1700 has a bit stream 12 encoded with a texture signal and a shape signal.
Encoded data having a data structure such as 00 is input. The input coded data is converted into data
Is separated into each encoded data. On the other hand, information indicating the size of the shape of the object and the position in the screen is input to the additional information acquisition unit 1706.

Each of the encoded data corresponding to the texture signal and the shape signal is decoded by the image decoding means 1704. Note that a plurality of decoding units 1704 may be provided, and decoding processes on a plurality of encoded data may be performed in parallel.

The decoded shape signal and texture signal are input to the pixel synthesizing unit 1708 via the switch 1716, and the texture signal of the background image is input to the synthesizing unit 1708 via the second input terminal 1714. The composition of the texture signal is performed between the background image and the foreground image.

That is, the synthesis processing unit 1708 uses the information such as the size of the shape of the input object and switches 1716 and 1718 to determine whether the pixel is within the rectangular area (object area). A texture signal is synthesized between foreground images and output, and for pixels outside the rectangular area, the texture signal of the background image is output as it is. Here, the texture signal of the background image may be already decoded or generated and stored in the memory, or the coded data may be decoded by means similar to the decoding means 1704.

The synthesis processing unit 1708 refers to additional information indicating the position and size of the object on the foreground screen, which is separately input, based on the input texture signal of the background and foreground and the shape signal of the foreground. The pixels constituting the foreground screen are determined inside and outside the rectangular area, and the switches 1716 and 1718 are used to determine whether
The combining process is performed by switching between the combining process inside the rectangular region by 0 and the combining process outside the rectangular region. The background texture data may be used for pixels outside the rectangular area.

FIG. 15 shows a configuration of an image synthesizing apparatus according to a modification of the seventh embodiment. This image synthesizing apparatus 1701 is different from the image synthesizing apparatus shown in FIG. A memory 1802 for holding information on the texture signal and the shape signal of the screen and the size of the object on the foreground screen is added.

The image synthesizing apparatus 1 having the memory 1802
In 701, when synthesizing a still image, the amount of processing can be reduced by using already decoded data. In the memory 1802, in addition to the decoded shape signal and texture signal, the size and position of the rectangular area (object area) on the shape screen obtained at the time of decoding are displayed.
Alternatively, it holds additional information indicating the boundary position of the rectangular area, and inputs the information to the pixel synthesizing unit 1710 when synthesizing an image, thereby obtaining the same information as the image synthesizing apparatus 1700 shown in FIG. Perform processing. By transmitting information such as the position of the boundary of the rectangular area to the synthesis processing unit 1708 in this manner, it is possible to reduce the processing of determining whether the pixel to be processed is located at the boundary.

Further, information such as the position and size of the rectangular area to be referred to when performing the above-mentioned image synthesis is stored in the additional information acquisition unit 1.
706 without using the decoding unit 1704
708 may be sent directly. By such a combining process, it is possible to limit an area in the foreground screen where it is necessary to determine whether or not to perform the texture data combining process between the foreground and the background. Complicated arithmetic processing can be reduced. Further, information such as the size and position of the foreground screen to be synthesized and displayed on the background screen may be described in the additional information.

This will be described with reference to FIG. 14, the same symbols as those in FIG. 12 are the same as those described in the seventh embodiment. Background texture signal (screen 1600), foreground texture signal (screen 1602)
When the foreground image and the background image are combined using the foreground shape signal (screen 1604), the combined image is enlarged and displayed, or the combined image is displayed on another combining screen. Or may be. In that case, the composite image 160
0a on position 2704 on another composite screen 2700a
By similarly inputting the 2702 and the image sizes 2708 and 2706, the display position and the display size can be changed according to the synthesized image.

Embodiment 8 FIG. 16 and 17 are diagrams for explaining an image synthesizing method according to the eighth embodiment of the present invention. The image synthesizing method according to the eighth embodiment reduces redundant judgment processing of whether or not to perform the synthesizing process by checking the positions of the pixels inside and outside the object in advance when performing the synthesizing process. is there. FIG. 16 and FIG.
4 shows a shape screen obtained by a shape signal of a predetermined image. One square in the drawing corresponds to one pixel, and a colored pixel (eg, pixel 1902) indicates that the pixel is inside the object, and a non-colored pixel (eg, 1904)
) Indicates that the object is outside the object.

The image synthesizing method described in the first, second, and third embodiments of the present invention uses the information of the pixels in the target regions 1910, 1910 centered on the pixel to be processed to be subjected to the synthesis processing. This is for calculating the composition ratio of the processed pixels and the texture data.

By the way, as described in the first to third embodiments, the portion which needs to be processed for the pixel shape data at the boundary between the foreground and the background is, for example, the process for the pixel only in the object is shown in FIG. The pixel is a pixel adjacent to the part indicated by the white arrow 1906 or the part indicated by the white arrow 1906. Also, for pixels located inside the object without being adjacent to the boundary, as indicated by the white arrow 2002 in FIG. 17, or for pixels located outside the object, indicated by the black arrow 2004, Form 1
It is not necessary to perform the processing on the pixels at the boundary portion by the image synthesis method 3.

Accordingly, as shown in FIGS. 16 and 17, the shape signal (shape screen 1 obtained from the shape signal)
900, 2000) are searched for the start and end points of the portion that requires processing on the pixels at the boundary, and the start and end points of the pixel positions that do not require processing on the pixels at the boundary. Then, processing is performed separately for the boundary portion and the other portion using the information on the searched pixel position. Accordingly, redundant processing can be reduced as compared with the case where the determination processing is performed for each pixel.

FIG. 18 is a block diagram for explaining an image synthesizing apparatus for synthesizing images by retrieving the shape signals shown in FIGS. This image synthesizing device 21
00 is a boundary position determining unit 2104 for determining the boundary position of the object based on the shape signal input from the input terminal 2100a, and a target of the combining process based on the determination result of the boundary position by the boundary position determining unit 2104. And a combination ratio generation unit 2106 for calculating the combination ratio of the pixel to be processed. Also, the image synthesizing device 2100 includes the texture data (pixel value of the texture signal) of the foreground image to be processed input from the second input terminal 2102 and the background image input from the third input terminal 2116. Texture data (pixel value of texture signal)
And a combination processing unit 2110 that combines textures based on the combination ratio calculated by the combination ratio generation unit 2106 to obtain texture data of a combined image.

The synthesizing unit 2110 includes a pixel synthesizing unit 2112 near the boundary for synthesizing pixels located near the boundary, and a pixel synthesizing unit within the boundary for synthesizing pixels located outside the boundary. 2114 based on the information of the boundary position from the boundary position determination means 2108
A switch 2122 for supplying the texture data of the foreground image to be processed from the input terminal 2102 to one of the pixel synthesizing unit 2112 and the pixel synthesizing unit 2114;
Based on the information on the boundary position, the pixel synthesizing unit 2112 is used.
And a switch 2124 for selecting any one of the output of the pixel synthesizing unit 2114 and the texture data of the background image input from the third input terminal 2116.

First, the input terminal 2 of the synthesizer 2100
When the shape signal is input from 100a, the input shape signal is used to determine the boundary position of the object by the boundary position determination unit 2104. If the boundary position determining unit 2104 determines that the pixel is located at the boundary position, the combining ratio generating unit 2106 calculates the combining ratio of the pixel to be processed which is the target of the combining process. In the synthesis processing unit 2110, the second
, The texture data of the foreground image to be processed (pixel value of the texture signal) input from the input terminal 2102, and the texture data of the background image input from the third input terminal 2116 (the pixel value of the texture signal).
Using the composition ratio calculated by the composition ratio generation unit 2106, the texture data of the composite image is obtained.

At the time of the synthesizing process, the switches 2122 and 2124 cause the synthesizing unit 21 to operate according to the information on the boundary position input from the boundary position judging unit 2108.
The switching between the combining process of the pixels located near the boundary by 12 and the combining process by the combining unit 2114 other than the vicinity of the boundary is performed.

As described above, the amount of processing can be reduced by performing the process of calculating the combination ratio required for the pixels near the boundary only for the necessary pixels. Note that, in the calculation processing of the combination ratio by the combination ratio generation unit 2106, the combination ratio may be calculated by the pixel combination unit 2112 near the boundary instead of the combination ratio generation unit, and the combination processing may be performed.

Further, the image synthesizing method according to the eighth embodiment is combined with the image synthesizing method according to the seventh embodiment, that is, the image synthesizing method according to the eighth embodiment indicates the size of the shape of the object. By performing a process of searching for a shape signal according to the eighth embodiment in a rectangular area including a shape of an object using additional information, redundant arithmetic processing required for determination of inside and outside of the object is further reduced. Can be.

Embodiment 9 FIG. FIG. 19 is a diagram for explaining an image synthesizing method according to the ninth embodiment of the present invention. In the image synthesizing method according to the ninth embodiment, when the image signal of the foreground image having a different resolution of the texture signal and the shape signal is synthesized with the image signal of the background image, the color difference as the texture signal is adjusted according to the resolution of the shape signal. The resolution of the signal is converted, and the color difference signal is synthesized between the foreground image and the background image.

FIG. 19 conceptually shows a process of synthesizing the color difference signals of the foreground image and the background image after the resolution conversion. 2200, 2204, and 2206 in FIG.
Are the shape pixels 2500 of the foreground image in FIG. 26, respectively.
Color pixel 2504 of foreground image, color pixel 2506 of background image
Is the same as

Hereinafter, description will be made with reference to FIG. First, with reference to the pixel value (shape data) of the foreground shape pixel 2200, the pixel value (color difference data) of the corresponding color pixel 2204
Is converted to the same size (resolution) as the shape data to obtain a color difference pixel 2202. The resolution conversion method is the method specified in MPEG4 (MPEG-4 Video Verifi
cation Model Ver8.0 (ISO / IEC JTC1 / SC29 / WG11 N1796)
Medium 2.2.2 filtering process) shall be used. Similarly, regarding the pixel value (color difference data) of the background color pixel 2206, when the size at the time of synthesis is different, the 2 × 2 size color pixel 2206 is converted to the 4 × 4 size color pixel 2204 by resolution conversion. Ask.

Foreground and background color pixels 22 whose resolution has been converted
The color difference data is synthesized between the foreground and the background using the pixel values of 02, 2204 and the shape pixel 2202. In addition,
It is assumed that a synthesized color pixel 2210 is obtained by this synthesis. When the synthesized color pixel 2210 is compared with the synthesized color pixel 2510 obtained by performing the resolution conversion after the conventional synthesis, the pixel value of the pixel in the object of the synthesized color pixel 2210 affects the pixel outside the object. You can see that it is not.

Therefore, as shown in FIG. 26, a process of once lowering the resolution of a high-resolution shape signal, synthesizing a color difference signal using the shape signal with the reduced resolution, and increasing the resolution of the synthesized color difference signal is performed. As shown in FIG. 19, the deterioration of the image is reduced when the color difference signals are combined in accordance with the high resolution (in this case, the shape signal) as compared with the case of performing the combination.

FIG. 20 is a conceptual diagram showing, for each pixel, shape data constituting a shape signal and luminance data constituting a luminance signal as a texture signal. 8 × 8 in FIG.
The shape pixel 800 and the 8 × 8 luminance pixel 300 are shown in FIG.
And the same as the texture pixel 800 shown in FIG. 5 and the texture pixel 800 shown in FIG.

In FIG. 20, reference numerals 2302 and 2304 represent two color pixels corresponding to the luminance pixel 800. Although the shape pixel 800 shown in FIG. 20 has the same number of pixels as the luminance pixel 300, the number of color pixels 2302 and 2304 is ４ of the number of pixels of the shape pixel 800 and the luminance pixel 300. There is no pixel correspondence between the color difference signal, the luminance signal and the shape signal.

Therefore, when the foreground image and the background image are combined, the color difference signal is subjected to resolution conversion in accordance with the number of pixels of the shape signal to obtain 8 × 8 color pixels 2306 and 2308. This resolution conversion method is defined in MPEG4 (MPEG-4 Video Verification Model Ver8.0 (ISO / IE
C JTC1 / SC29 / WG11 N1796) Medium 2.2.2 filtering proces
s) may be used. Specifically, the resolution conversion is performed by interpolating the pixels by weighting and averaging adjacent pixels.

When the resolution of the pixel at the boundary between the inside and outside of the object is converted, the color difference data of the pixel outside the object has an indefinite value, so that the color difference data of the pixel inside the object is used as the corresponding color difference data. The resolution conversion is performed by using. Then, a synthesis process between the foreground image and the background image is performed on the color difference data subjected to the resolution conversion with reference to the corresponding shape data. For the method of synthesis, the above-described synthesis method of the embodiment of the present invention may be used.

In this embodiment, when the resolution of the color difference signal and the resolution of the shape signal are different, the synthesizing method of performing resolution conversion and performing synthesis has been described, but the texture signal (luminance signal, color difference signal) and the shape signal are converted. Alternatively, the foreground and the background may be combined and displayed after the resolution is converted to a size adapted to the screen to be displayed. The method for converting the resolution of the texture signal in this case may be the method described above. For the resolution conversion of the shape signal, the method described in MPEG-4 (MPEG-4 Video Verification Model Ver8.0 (ISO /
(IEC JTC1 / SC29 / WG11 N1796) 3.2.5 size conversion)
May be used.

Hereinafter, a synthesizing apparatus that performs a synthesizing process after performing a resolution conversion will be described with reference to FIG. The image synthesizing device 2400 converts the shape signal input from the first input terminal 2400 into a first resolution conversion unit (shape information resolution conversion) that performs resolution conversion in accordance with the size (resolution) of the screen to be displayed. Means) 2406, and second resolution conversion means (pixel information resolution conversion means) 2408 for performing resolution conversion on the texture signal input from the second input terminal 2402 according to the size (resolution) of the shape signal. , A third resolution conversion unit (pixel information resolution conversion unit) 241 that performs resolution conversion on the background texture signal input from the third input terminal 2404.
0. The image synthesizing device 2400
Are the first, second, and third resolution conversion means 2406,
It has a pixel synthesizing unit 2412 that receives outputs of 2408 and 2410 and synthesizes a texture signal between a foreground image and a background image, and outputs a synthesized texture signal to an output terminal 2420.

Next, the operation will be described.

First, a shape signal of a foreground image representing the shape of an object is input from a first input terminal 2400a of the device 2400, and a texture signal of the foreground image corresponding to the shape signal is input from a second input terminal 2402. You. From the third input terminal 2404, a texture signal of a background image is input.

The shape signal input from the first input terminal 2400 can be converted by the first resolution conversion means 2 if necessary.
Through 406, resolution conversion processing is performed according to the size (resolution) of the screen to be displayed. Second input terminal 240
The texture signal input from 2 is subjected to resolution conversion processing by the second resolution conversion means 2408 according to the size (resolution) of the shape signal. Further, if necessary, the texture conversion signal of the background input from the third input terminal 2404 is subjected to resolution conversion processing by the third resolution conversion means 2410. Then, the first, second, and third resolution conversion means 2406, 2408, 241
The output of 0 is input to the pixel synthesizing unit 2412, and a texture signal synthesizing process is performed between the foreground image and the background image. The synthesized texture signal is output to an output terminal 242.
Output from 0.

In the resolution conversion, not only the above-described enlargement processing, but also, when the size of the screen to be displayed is small, the input shape signal and texture signal are reduced to an appropriate image size. The combining process may be performed later. Furthermore, only the foreground or the background can be enlarged or reduced and synthesized as needed. Further, the enlargement and reduction processing may be performed independently for the luminance signal and the color difference signal constituting the texture signal.

In the ninth embodiment, when the resolution of the chrominance signal is 解像度 of the resolution of the luminance signal, that is, the resolution of the chrominance signal is の of the resolution of the luminance signal in both the horizontal and vertical directions. Although a case has been described, as in the ninth embodiment, in the image synthesizing method in which the color difference signal is resolution-converted and then the color difference signal is synthesized between the foreground image and the background image, The ratio of the resolution of the color difference signal is not limited to the above. For example, even when the horizontal resolution of the chrominance signal is １ ／ of the resolution of the luminance signal and the vertical resolution of the chrominance signal is the same as the resolution of the luminance signal, the same image synthesis as in the ninth embodiment is performed. Processing can be performed.

Embodiment 10 FIG. Furthermore, an image synthesizing program for realizing the synthesizing process by the image synthesizing method shown in each of the above embodiments is recorded on a data storage medium such as a floppy disk, so that the image synthesizing program shown in each of the above embodiments is described. The processing can be easily performed in an independent computer system.

FIG. 22 is an explanatory diagram of a case where the image synthesizing process of the first to ninth embodiments is performed by a computer system using a floppy disk storing the encoding or decoding program. Fig. 22 (a)
Shows the external appearance, cross-sectional structure, and floppy disk of the floppy disk viewed from the front, and FIG. 22B shows an example of the physical format of a floppy disk D which is a recording medium body. A floppy disk D is built in a case F, and a plurality of tracks Tr are formed concentrically from the outer periphery toward the inner periphery on the surface of the disk, and each track is divided into 15 sectors Se in an angular direction. ing. Therefore, in the floppy disk storing the program, data as the program is recorded in an area allocated on the floppy disk D.

FIG. 22C shows a configuration for recording and reproducing the above program on the floppy disk D. When the above program is recorded on the floppy disk FD, data as the above program is transferred from the computer system Cs to the floppy disk drive FDD.
Write through. When the encoding or decoding device is constructed in a computer system using a program in the floppy disk FD, the program is read from the floppy disk FD by the floppy disk drive FDD and transferred to the computer system.

In the above description, the description has been made using a floppy disk as a data recording medium. However, the same can be done using an optical disk. Further, the recording medium is not limited to this, and the present invention can be similarly implemented as long as the program can be recorded thereon, such as an IC card or a ROM cassette.

Further, in the image synthesizing method according to the above-described embodiment, after the shape data or the texture data of the pixels arranged over the boundary portion is smoothed for the pixels located at the boundary portion, , Synthesis
Instead of judging whether a pixel is located at a boundary portion by one pixel, the boundary portion is determined in advance, and then the synthesizing method of the present invention is applied to the boundary portion. , Redundant judgment processing can be reduced.

Since the texture signal is composed of the luminance signal and the color difference signal, the size of the target area including the pixel to be processed in the synthesis processing for the luminance signal and the pixel to be processed in the synthesis processing for the color difference signal are determined. May be set independently of the size of the target region, or different synthesis processes may be performed for the luminance signal and the color difference signal.

Further, in each of the above embodiments, the processing of synthesizing the luminance data or the color difference data between the foreground image and the background image is performed by the arithmetic processing defined by (Equation 1). By using a modification in which the multiplier or divisor in the arithmetic processing is a power of two, the arithmetic processing can be simplified. In other words, the quotient of the product of the multiplicand and the multiplier or the quotient of the divisor of the dividend can be easily obtained by the bit shift processing of the multiplicand or the dividend in the arithmetic unit.

Specifically, in the above (Equation 1), when alpha is 0
If the value is in the range of <alpha <255, it can be modified as in the following Expression 3.

Pel = (alpha × fgpel + (256−alpha) × bgpel) / 256 (Expression 3) In Expression 3, when alpha = 0, pel = bgpel
, When alpha = 255, pel = fgpel.

In the above (Equation 1), if alpha is alpha <
If the value is in the range of 0, it can be modified as in the following Expression 4. pel = ((alpha + 1) × fgpel + (255−alpha) × bgpel) / 256 (Expression 4) However, in Expression 4, when alpha = 0, pel = bgpel
Becomes Also, the expressions for obtaining the pixel value pel after the combination shown in FIGS. 2C, 3C, 7 and 8 are simplified in the same manner as described above to simplify the arithmetic processing. be able to.

(Embodiment 11) FIG. 28 is a view for explaining an image synthesizing method and an image synthesizing apparatus according to Embodiment 11 of the present invention. The image synthesizing method according to the eleventh embodiment is similar to the image synthesizing method according to the first embodiment, and forms a first image space including an object having an arbitrary shape. And a second image signal that forms a second image space and includes a texture signal that indicates a picture pattern of an image to form a third image space. This is a method for generating a composite image signal.

In the image synthesizing method according to the eleventh embodiment, the first and second image synthesizing methods different from each other are performed based on the shape signals of the pixels located in the target area including the pixel to be processed in the first image space. 2 is switched.

Here, the first image synthesizing process includes a synthesizing ratio calculating step and a pixel synthesizing step, similarly to the image synthesizing method of the first embodiment, and the synthesized texture signal is converted into the third image space. , Is used as a texture signal of a pixel corresponding to the pixel to be processed.

In the combining ratio calculating step, the combining ratio of the pixel to be processed to the texture signal is determined based on the shape signal of the pixel located in the target area including the pixel to be processed in the first image space. Is calculated.
In the pixel synthesizing step, a texture signal synthesizing process of these pixels is performed between the pixel to be processed and the corresponding pixel in the second image space corresponding to the pixel to be processed based on the calculated synthesizing ratio. Will be

In the second image synthesizing process, in the pixel synthesizing step, the texture signal of the pixel to be processed in the object in the first image space is converted into the pixel signal in the third image space. Is performed for use as a texture signal of a pixel corresponding to.

FIG. 28 is a block diagram showing a configuration of an image synthesizing apparatus for performing a synthesizing process by the image synthesizing method according to the eleventh embodiment. Based on the shape type signal obtained from the foreground image signal, the image synthesizing device 3500 converts the predetermined region in the first embodiment, that is, the region of a predetermined size including the target pixel to be processed (target region) into A region where all the pixels are located outside the object (hereinafter referred to as an extra-object block), a region where all the pixels are located inside the object (hereinafter referred to as an intra-object block), and a pixel located outside the object. The image processing apparatus includes a boundary block determining unit 3510 that determines which of the areas includes a pixel located in the object (hereinafter, referred to as a boundary block). Here, the value of the shape data of the pixel outside the object is “0”, and the value of the shape data of the pixel inside the object is “255”.

The image synthesizing device 3500 receives the shape signal of the foreground and receives the boundary block determining means 3510.
The boundary pixel determination means 35 that determines whether each pixel in the boundary block is a pixel in the object based on the determination result in
20 and the number of pixels in the object in the boundary block are calculated based on the determination result of the boundary pixel determination unit 3520, and the synthesis control is performed according to whether the number of pixels in the object is equal to or greater than a predetermined threshold value. A synthesizing method determining unit 3530 that outputs a signal 3506.

Further, the image synthesizing device 3500 has first and second pixel synthesizing means 3540 and 3550 for synthesizing respective texture signals (color signals) between the background image and the foreground image. . Here, the first pixel synthesizing unit 3540 is configured to realize a synthesizing process by the image synthesizing method according to the first embodiment.

That is, the first pixel synthesizing means 3540
Is a composition ratio calculation means for calculating a composition ratio of a foreground processed pixel to a texture signal based on a foreground shape signal, and the foreground processed pixel based on the calculated composition ratio. Pixel synthesizing means for synthesizing the texture signals of these pixels with the corresponding pixels of the corresponding background, and synthesizing the synthesized texture signals with the pixels corresponding to the pixels to be processed in the synthesized image. It is configured to output as a texture signal.

Also, the second pixel synthesizing means 3550
The texture signal of the pixel to be processed in the object of the foreground image is used as the texture signal of the pixel corresponding to the pixel to be processed in the composite image, and corresponds to the pixel to be processed outside the object of the foreground image. The texture signal of the corresponding pixel in the background image is represented by
The configuration is used as a texture signal of a pixel corresponding to the pixel to be processed.

The image synthesizing device 3500
A pre-switch 3560a for supplying a foreground color signal to one of the first and second pixel synthesizing means 3540 and 3550 based on a synthesizing control signal 3506 from the synthesizing method judging means 3530; One of the output signals of the first and second pixel synthesizing means 3540 and 3550 is selected based on the synthesis control signal 3506 from the means 3530, and the selected output signal is output to the output terminal 3500e as a color signal of a synthesized image. Output switch 3560b.

Reference numerals 3500a to 3500d are first to fourth input terminals, respectively. First input terminal 3500
a, a foreground shape signal is input to the second input terminal 3500b, a foreground color signal is input to the third input terminal 3500c, and a background color signal is input to the fourth input terminal 3500d. It is supposed to be.

Next, the operation will be described. When the shape type signal 3501 obtained from the foreground image signal is input to the first input terminal 3500a of the image synthesizing device 3500, the boundary block determining unit 3510 determines a predetermined type including a target pixel to be processed. Size area (here 3x3
A block consisting of pixels) is a region where all pixels are located outside the object (external block), a region where all pixels are located inside the object (intra-object block), and pixels and objects located outside the object. A determination is made as to which of the regions (boundary blocks) includes pixels located inside the body.

The boundary pixel determining means 3520 of the image synthesizing device 3500 receives the foreground shape signal 3502 input to the second input terminal 3500b, and based on the determination result by the boundary block determining means 3510, For each pixel in the boundary block, a determination is made as to whether or not this is a pixel in the object.

Then, in the synthesizing method judging means 3530 of the image synthesizing apparatus 3500, the boundary pixel judging means 352
Based on the determination result of 0, only for the boundary block, the number of pixels in the object in the block is calculated,
A synthesis control signal 3506 is output according to whether the number of pixels in the object is equal to or greater than a predetermined threshold.

At this time, the foreground switch 3560a of the image synthesizing device 3500 outputs the foreground color signal input to the third input terminal 3500c to the synthesizing control signal 35
06, it is controlled so as to be supplied to one of the first and second synthesizing means 3540 and 3550, and the rear switch 3560b of the image synthesizing device 3500 is turned on by the synthesizing control signal 3506. Synthetic means 3
It is controlled to select one of the outputs 540 and 3550.

For example, as a result of the judgment by the synthesizing method judging means 3530, if the number of pixels in the object in the boundary block is equal to or larger than a predetermined threshold value, the foreground color signal 3503 is transmitted via the front switch 3560a. Then, the output is supplied to the first pixel synthesizing unit 3540, and the output of the pixel synthesizing unit 3540 is output to the output terminal 3500e as the synthesized color signal 3505 via the subsequent switch 3560b. In the first pixel synthesizing unit 3540, similarly to the image synthesizing method of the first embodiment, the synthesis ratio of the pixel to be processed of the foreground to the texture signal is calculated based on the shape signal of the foreground, and the calculated synthesis ratio is calculated. , The texture signal of these pixels is synthesized between the pixel to be processed in the foreground and the corresponding pixel in the background corresponding to the foreground.

On the other hand, as a result of the judgment by the synthesizing method judging means 3530, if the number of pixels in the object in the boundary block is not equal to or larger than the predetermined threshold value, the foreground color signal 3503 is transmitted via the front switch 3560a. Second
, And the output of the pixel synthesizing unit 3550 is output to the output terminal 3500e as a synthesized color signal 3505 via the subsequent switch 3560b. In the second pixel synthesizing unit 3550, a texture signal of a pixel to be processed in the object of the foreground image is output as a texture signal of a pixel corresponding to the pixel to be processed in the composite image. A texture signal of a corresponding pixel in the background image corresponding to the outside pixel to be processed is output as a texture signal of a pixel in the composite image corresponding to the pixel to be processed.

In the eleventh embodiment having such a configuration, after the texture data of the pixel to be processed to be processed is smoothed for the pixels arranged across the boundary between the foreground image and the background image, A first synthesis process for synthesizing texture data between the foreground image and the background image, and outputting texture data of a pixel to be processed in the object of the foreground image as a texture signal of a corresponding pixel in the synthesized image; A texture signal of a corresponding pixel in the background image corresponding to a pixel to be processed outside the object in the foreground image,
The switching between the second synthesizing process that outputs the texture signal of the corresponding pixel in the synthesized image and the second synthesizing process is performed based on the shape signal of the foreground. There is an effect that it is possible to avoid blurring of the boundary portion while suppressing occurrence of blurring.

In the eleventh embodiment, the first
Although the above-described pixel synthesizing means performs the synthesizing process by the image synthesizing method of the first embodiment, the first pixel synthesizing means may be any of the embodiments other than the first embodiment, for example, the second embodiment. Alternatively, the composition processing may be performed by any one of the image composition methods described above.

Further, by recording an image synthesizing program for realizing the synthesizing process by the image synthesizing method shown in the eleventh embodiment on a data storage medium such as a floppy disk, the above-mentioned embodiment is realized. Similarly to the image combining processes 1 to 9, the image combining process of the eleventh embodiment can be easily performed by an independent computer system.

[0187]

As described above, according to the image synthesizing method according to the present invention, the texture data of the pixel to be processed to be processed is determined for the pixels arranged across the boundary between the foreground image and the background image. Alternatively, since the texture data is synthesized between the foreground image and the background image after the shape data is smoothed, the image quality at the boundary between the synthesized foreground and background can be improved.

Further, according to the data storage medium of the present invention, a program for causing a computer to perform the image synthesizing process by the above-described image synthesizing method is stored. The image synthesis processing that can improve the image quality of the image can be realized by software.

[Brief description of the drawings]

FIG. 1 is a diagram for describing a processing flow in an image synthesizing method according to Embodiment 1 of the present invention.

FIG. 2 is a diagram for conceptually explaining the image synthesizing method according to the first embodiment.

FIG. 3 is a diagram conceptually illustrating an image synthesizing method according to a second embodiment of the present invention.

FIG. 4 is a diagram for explaining a flow of processing in an image synthesizing method according to Embodiment 3 of the present invention.

FIG. 5 is a diagram for conceptually explaining an image synthesizing method according to the third embodiment.

FIG. 6 is a schematic diagram for explaining an image synthesizing method according to Embodiment 4 of the present invention.

FIG. 7 is a diagram for explaining an image synthesizing method according to a fifth embodiment of the present invention.

FIG. 8 is a diagram for explaining an image synthesizing method according to a modification of the fifth embodiment.

FIG. 9 is a diagram showing a data structure of a bit stream handled in an image combining method according to a sixth embodiment of the present invention.

FIG. 10 is a diagram showing a data storage medium storing a program for realizing the processing by the image synthesizing method according to each embodiment of the present invention by a computer system.

FIG. 11 is a block diagram showing a configuration of an image synthesizing apparatus according to a modification of the sixth embodiment.

FIG. 12 is a schematic diagram conceptually illustrating an image synthesizing method according to a seventh embodiment of the present invention.

FIG. 13 is a block diagram showing an image synthesizing device according to the seventh embodiment.

FIG. 14 is a diagram conceptually illustrating a modification of the image synthesizing method according to the seventh embodiment.

FIG. 15 is a block diagram showing an image synthesizing device according to a modification of the seventh embodiment.

FIG. 16 is a diagram conceptually illustrating an image synthesizing method according to an eighth embodiment of the present invention.

FIG. 17 is a diagram conceptually illustrating an image synthesizing method according to the eighth embodiment.

FIG. 18 is a block diagram showing an image synthesizing device according to the eighth embodiment.

FIG. 19 is a schematic diagram for explaining an image synthesizing method according to a ninth embodiment of the present invention.

FIG. 20 is a schematic diagram for explaining an image synthesizing method according to the ninth embodiment.

FIG. 21 is a block diagram showing an image synthesizing device according to the ninth embodiment.

FIG. 22 is a block diagram showing a configuration of an image synthesizing apparatus according to Embodiment 6 of the present invention.

FIG. 23 is a diagram illustrating a process of combining a plurality of images in a conventional image combining process.

FIG. 24 is a diagram for explaining an image synthesized by the image synthesis processing method shown in FIG. 23;

FIG. 25 is a schematic diagram showing shape data corresponding to the composite image shown in FIG. 24;

FIG. 26 is a diagram for explaining a process of synthesizing an image and performing resolution conversion in a conventional image synthesizing method.

FIG. 27 is a block diagram showing an apparatus for performing the resolution conversion processing shown in FIG. 26.

FIG. 28 is a block diagram showing an image synthesizing apparatus according to Embodiment 8 of the present invention.

[Explanation of symbols]

402 Unprocessed image determination step 404 Shape information input step 406 Shape information determination step 410 Shape information calculation step (composition ratio calculation step) 412 Pixel composition step 1200 Bit stream 1400, 1500, 1700, 2100, 2400,
3500 Image synthesis device 1402, 1502, 1702 Data separation means 1404, 1504, 1704 Image decoding means 1408, 1508, 2110 Image synthesis means 1412, 1414, 1716, 1718, 2122
2124 Switch 1506 Synthesis method determination means 1706 Additional information acquisition means 1708 Synthesis processing unit 1710 Pixel synthesis processing means 1802 Memory 2104 Boundary position determination means 2106 Synthesis ratio generation means 2112 Pixel synthesis means near boundary 2114 Pixel synthesis means within boundary 2406 Shape information Resolution conversion means 2408, 2404 Pixel information resolution conversion means 3500a, 3500b, 3500c, 3500d First to fourth input terminals 3500e Output terminal 3501 Boundary block determination means 3510 3520 Boundary pixel determination means 3530 Synthesis method determination means 3540 First pixel Combining means 3550 Second pixel combining means 3560a Front switch 3560b Rear switch Cs Computer system FD Floppy disk FDD Floppy disk drive

Claims (26)

[Claims] 1. A first image signal including a shape signal indicating a shape of an object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape, and a second image space. Forming a third image space by synthesizing a second image signal including a texture signal indicating a picture pattern of the image to form a third image space. A calculating process for a shape signal of a pixel located in a target region including the pixel to be processed, a combining ratio calculating step of calculating a combining ratio of the pixel to be processed to a texture signal, based on the calculated combining ratio, The pixel to be processed;
And a pixel synthesizing step of synthesizing texture signals of these pixels with corresponding pixels in the second image space corresponding thereto, and converting the synthesized texture signals in the third image space into An image synthesizing method, wherein the method is used as a texture signal of a pixel corresponding to the pixel to be processed. 2. The image combining method according to claim 1, wherein a shape determination step of determining whether each pixel is located in the object based on a shape signal of each pixel in the first image space. In the combining ratio calculating step, when the result of the determination is that the pixel to be processed is located in the object, the combining ratio is calculated using a shape signal of a pixel in the target region. Synthesis method. 3. The image synthesizing method according to claim 1, wherein, based on a shape signal of each pixel in the first image space, a shape determining step of determining whether each pixel is located inside the object. In the combining ratio calculating step, when the result of the determination is that the pixel to be processed is located in the object, the combining ratio is calculated by averaging the shape signals of the pixels in the target region. Image composition method. 4. The image synthesizing method according to claim 1, wherein in the synthesizing ratio calculating step, the arithmetic processing on the shape signal is performed by using a multiplier or divisor of 2 in the arithmetic processing.
An image synthesizing method, wherein the image synthesizing method is performed based on an arithmetic expression modified so as to be a number of powers of. 5. A first image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape, and a second image space. Forming a third image space by synthesizing a second image signal including a texture signal indicating a picture pattern of the image to form a third image space. Based on the shape signal of the pixel, a shape determination step of determining whether each pixel is located inside the object, and, as a result of the determination, when the target pixel is located in the object, a pixel in the target area Calculating a combination ratio of the pixel to be processed with respect to the texture signal based on the number of objects located in the object; and calculating the combination ratio based on the calculated combination ratio. And iodine,
And a pixel synthesizing step of synthesizing texture signals of these pixels with corresponding pixels in the second image space corresponding thereto, and converting the synthesized texture signals in the third image space into An image synthesizing method, wherein the method is used as a texture signal of a pixel corresponding to the pixel to be processed. 6. A first image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape, and a second image space. Forming a third image space by synthesizing a second image signal including a texture signal indicating a picture pattern of the image to form a third image space. A shape determining step of determining whether or not each pixel is located inside the object based on a shape signal of each pixel; and a pixel located in a target area including the pixel to be processed in the first image space. Based on the shape signal of the above, the synthesis ratio calculating step of calculating the synthesis ratio of the pixel to be processed to the texture signal, and as a result of the shape determination, when the pixel to be processed is located outside the object, the pixel to be processed A pixel signal generating step of replacing the texture signal of the pixel signal with a texture signal obtained by averaging the texture signals of the pixels in the target area; based on the calculated combination ratio, the pixel to be processed;
And a pixel synthesizing step of synthesizing texture signals of these pixels with corresponding pixels in the second image space corresponding to the texture signal. An image synthesizing method, wherein the method is used as a texture signal of a pixel corresponding to the pixel to be processed. 7. The image synthesizing method according to claim 6, wherein in the synthesizing ratio generating step, an average value of shape signals of pixels included in the target area is calculated as the synthesizing ratio. . 8. The image synthesizing method according to claim 6, wherein, in the pixel information generating step, the texture signal of the pixel to be processed is replaced with the texture signal of a pixel located in the object among the pixels included in the target area. An image synthesizing method characterized by replacing with an average value. 9. The image synthesizing method according to claim 5, wherein in the synthesizing ratio calculating step, a table is calculated in advance according to the number of pixels located in the object among the pixels in the target area. And calculating the combination ratio stored in the table from the table according to the number of pixels in the object obtained from the shape signal of the pixels located in the target area. 10. The image synthesizing method according to claim 1, wherein in the pixel synthesizing step, a table storing a result of multiplying all combinations of the value of the synthesizing ratio and the value of the texture signal is stored. An image synthesizing method, wherein a texture signal of a pixel corresponding to the pixel to be processed in the third image space is obtained. 11. The image synthesizing method according to claim 5, wherein, in the pixel synthesizing step, both of the combination of the number of pixels located in the object in the target area and the value of the texture signal are used. An image synthesizing method, wherein a texture signal of a pixel corresponding to the pixel to be processed in the third image space is obtained by referring to a table storing a result of the multiplication. 12. A first decoded image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape, and a second image. By combining a second decoded image signal including a texture signal indicating a picture pattern of an image to form a space,
A method for generating a composite image signal for forming a third image space, comprising: performing an arithmetic operation on a shape signal of a pixel located in a target area including a pixel to be processed in the first image space; A combining ratio calculating step of calculating a combining ratio of the pixel to the texture signal; and, based on the calculated combining ratio, the pixel to be processed;
A pixel synthesizing process of synthesizing a texture signal of these pixels with a corresponding pixel in a second image space corresponding thereto, wherein the pixel synthesizing step corresponds to the first decoded image signal. An image synthesizing method, wherein a method of synthesizing a texture signal in the synthesizing process is switched by referring to additional information added to the encoded image signal when generating the encoded image signal. 13. The image synthesizing method according to claim 12, wherein the additional information indicates a size of the target area. 14. A first image signal, comprising a shape signal indicating the shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape;
A method for generating a composite image signal for forming a third image space by synthesizing a second image signal including a texture signal indicating a picture pattern of an image to form a second image space; A boundary detection step of detecting a boundary position of the object based on a shape signal of each pixel in one image space; and a shape signal of a pixel located in a target region including the pixel to be processed in the first image space. A combining ratio calculating step of performing a process of calculating a combining ratio of the pixel to be processed with respect to the texture signal based only on pixels located inside and outside the object near the boundary of the object; For nearby pixels, based on the calculated combination ratio, the texture signal of these pixels is placed between the pixel to be processed and the corresponding pixel in the corresponding second image space. Is performed, and for pixels other than the vicinity of the boundary, the pixel to be processed and the corresponding pixel in the second image space corresponding to the pixel are processed based on the shape data of these pixels. And a pixel synthesizing step of performing a process of synthesizing texture signals of these pixels between the pixels. The synthesized texture signal is converted into a texture signal of a pixel corresponding to the pixel to be processed in the third image space. An image synthesizing method characterized by being used as an image. 15. A first image signal, comprising a shape signal indicating the shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object of an arbitrary shape;
A method for generating a composite image signal for forming a third image space by synthesizing a second image signal including a texture signal indicating a picture pattern of an image forming a second image space, the method comprising: Based on the size of and the additional information indicating the position of the object in the first image space, for a specific pixel located inside or outside the object and near the boundary of the object,
Performing a process of synthesizing a texture signal of these pixels between the pixel to be processed in the first image space and a corresponding pixel in the second image space corresponding thereto;
For the peripheral pixels other than the specific pixel in the first image space, the texture signal of the pixel in the second image space corresponding to these peripheral pixels is converted into the texture signal corresponding to the peripheral pixel in the third image space. A pixel signal corresponding to the specific pixel in the third image space, wherein a texture signal obtained by the combining process in the pixel combining step is used as a texture signal of a pixel corresponding to the specific pixel in the third image space. An image synthesizing method characterized by the following. 16. A first image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape;
A method for generating a composite image signal for forming a third image space by synthesizing a second image signal including a texture signal indicating a picture pattern of an image to form a second image space; A resolution conversion step of performing a resolution conversion process on the texture signal when the resolution of the texture signal is not equal to the resolution of the shape signal so that the resolution of the texture signal is equal to the resolution of the shape information; An image synthesizing method, comprising an image synthesizing step of synthesizing the first image signal and the second image signal using the texture signal subjected to the conversion process and the shape signal. 17. A first image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape;
A method for generating a composite image signal for forming a third image space by synthesizing a second image signal including a texture signal indicating a picture pattern of an image to form a second image space; When the resolution of the texture signal forming the first image signal, the resolution of the texture signal forming the second image signal, and the display resolution at the time of displaying the texture signal forming the composite image signal are different, the first and second image signals are different. A resolution changing step of performing a resolution conversion process on the texture signal of the second image signal so that the resolution is equal to the display resolution; and a texture signal of the first and second image signals subjected to the resolution conversion process. And the first image signal and the second
An image synthesizing step of performing an image signal synthesizing process. 18. A first image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape;
A method for generating a composite image signal for forming a third image space by synthesizing a second image signal including a texture signal indicating a picture pattern of an image to form a second image space; A combining ratio calculating step of calculating a combining ratio of the pixel to be processed to a texture signal based on a shape signal of a pixel located in a target area including the pixel to be processed in the image space of 1; A pixel synthesizing process of synthesizing a texture signal of the pixel between the pixel to be processed and a corresponding pixel in the second image space corresponding to the pixel to be processed. Is used as a texture signal of a pixel corresponding to the pixel to be processed in the third image space, and Pixel synthesis in which texture signal synthesis processing of these pixels is performed at a ratio different from the synthesis ratio in the first image synthesis process between the processing pixel and the corresponding pixel in the second image space corresponding thereto. And a second image synthesis process using the synthesized texture signal as a texture signal of a pixel corresponding to the pixel to be processed in the third image space. An image synthesizing method, wherein the first and second image synthesizing processes are switched based on a shape signal of a pixel located in a target area including a pixel to be processed. 19. The image synthesizing method according to claim 18, wherein in the second image synthesizing process, in the pixel synthesizing step, a texture signal of a pixel to be processed in the object in the first image space is converted into the second image synthesizing signal. 3
In the image space of the above, using as a texture signal of the pixel corresponding to the pixel to be processed, the texture signal of the corresponding pixel in the second image space corresponding to the pixel to be processed outside the object in the first image space, An image synthesizing method, which is used as a texture signal of a pixel corresponding to the pixel to be processed in the third image space. 20. A first decoded image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape, and a second image. By combining a second decoded image signal including a texture signal indicating a picture pattern of an image to form a space,
An apparatus for generating a composite image signal that forms a third image space, comprising: performing an arithmetic operation on a shape signal of a pixel located in a target area including a pixel to be processed in the first image space; Combining ratio calculating means for calculating a combining ratio of a pixel to a texture signal; and, based on the calculated combining ratio, the pixel to be processed;
Pixel synthesizing means for synthesizing texture signals of these pixels with corresponding pixels in a second image space corresponding thereto, and the pixel synthesizing means corresponds to the first decoded image signal. An image synthesizing device configured to switch a method of synthesizing a texture signal in the synthesizing process by referring to additional information added to the encoded image signal when generating the encoded image signal. . 21. A first image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape;
An apparatus that combines a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space, Boundary detection means for detecting a boundary position of the object based on a shape signal of each pixel in the first image space; and a shape signal of a pixel located in a target region including the pixel to be processed in the first image space. A combination ratio calculating means for performing a process of calculating a combination ratio of the pixel to be processed to a texture signal based on the only the pixels near the boundary of the object, within the object and located at the object. For the neighboring pixels, the texture signals of these pixels are synthesized between the pixel to be processed and the corresponding pixel in the corresponding second image space based on the calculated synthesis ratio. For pixels other than the vicinity of the boundary, the pixel to be processed and the corresponding pixel in the second image space corresponding to the pixel are processed based on the shape data of these pixels. A pixel synthesizing unit for performing a process of synthesizing a texture signal of these pixels, and using the synthesized texture signal as a texture signal of a pixel corresponding to the pixel to be processed in the third image space. An image synthesizing device, characterized in that: 22. A first image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape;
An apparatus for generating a combined image signal for forming a third image space by combining a second image signal including a texture signal indicating a picture pattern of an image, forming a second image space, Based on the size of and the additional information indicating the position of the object in the first image space, for a specific pixel located inside or outside the object and near the boundary of the object,
Performing a process of synthesizing a texture signal of these pixels between the pixel to be processed in the first image space and a corresponding pixel in the second image space corresponding thereto;
For the peripheral pixels other than the specific pixel in the first image space, the texture signal of the pixel in the second image space corresponding to these peripheral pixels is converted into the texture signal corresponding to the peripheral pixel in the third image space. A pixel synthesizing unit configured to generate a texture signal of a pixel to be processed, and a texture signal obtained by the synthesizing process of the pixel synthesizing step is used as a texture signal of a pixel corresponding to the specific pixel in the third image space. An image synthesizing apparatus characterized by the above-mentioned. 23. A first image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape;
An apparatus that combines a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space, Resolution conversion means for performing a resolution conversion process on the texture signal when the resolution of the texture signal is not equal to the resolution of the shape signal so as to make the resolution equal to the resolution of the shape information; An image synthesizing apparatus, comprising: image synthesizing means for synthesizing the first image signal and the second image signal using the texture signal subjected to the conversion processing and the shape signal. 24. A first image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape;
An apparatus that combines a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space, When the resolution of the texture signal forming the first image signal, the resolution of the texture signal forming the second image signal, and the display resolution at the time of displaying the texture signal forming the composite image signal are different, the first and second image signals are different. Resolution changing means for performing a resolution conversion process on the texture signal of the second image signal so that the resolution is equal to the display resolution; and a texture signal of the first and second image signals subjected to the resolution conversion process. And the first image signal and the second
An image synthesizing device, comprising: image synthesizing means for synthesizing the image signal. 25. A first image signal including a shape signal indicating a shape of the object and a texture signal indicating a picture of the object, forming a first image space including an object having an arbitrary shape;
An apparatus that combines a second image signal that forms a second image space and includes a texture signal indicating a picture pattern of an image to generate a combined image signal that forms a third image space, A combining ratio calculating means for calculating a combining ratio of a pixel to be processed to a texture signal based on a shape signal of a pixel located in a target area including the pixel to be processed in the image space of 1; On the basis of,
A first pixel synthesizing unit for synthesizing a texture signal of the pixel between the pixel to be processed and a corresponding pixel in the second image space corresponding to the pixel to be processed; A first image synthesis processing unit that outputs a texture signal of a pixel corresponding to the pixel to be processed in the third image space; a pixel to be processed in the first image space; A second pixel synthesizing means for synthesizing texture signals of these pixels with corresponding pixels in the second image space at a ratio different from the synthesizing ratio in the first image synthesizing process; Outputting a synthesized texture signal as a texture signal of a pixel corresponding to the pixel to be processed in the third image space;
Wherein the first and second image synthesizing processes are switched based on a shape signal of a pixel located in a target area including the pixel to be processed in the first image space. An image synthesizing apparatus characterized in that: 26. A data storage medium storing a program for causing a computer to perform an image synthesizing process, wherein the program is an image synthesizing process according to any one of claims 1 to 19. A data storage medium characterized in that a computer is configured to perform the following.