JP2007019956A - Image synthesizing apparatus and image synthesizing method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve a matter that a child screen does not cover an important image region of a parent screen completely, and a user can determine what kind of image about an important region of the parent screen. <P>SOLUTION: An image synthesizing apparatus 100 is provided with an image input 101 from which an image is input, a region detector 102 which detects the important region from the input image, an image synthesizer 103 which synthesizes a plurality of input images, and an image output 104 which outputs a synthesized image. In the image synthesizer 103, transparent synthesizing of the input image is performed when overlapping is generated with the important region detected by the region detector 102. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image composition apparatus and an image composition method for composing a plurality of moving images and still images on one screen.

  In some conventional image display devices, a small area screen is formed in the entire display screen, and an image different from the image displayed on the entire display screen is displayed on the small area screen.

  For example, Patent Document 1 describes a technique called picture-in-picture (PinP) that displays a thumbnail-like child screen in a parent screen. This technology superimposes a child screen in which a moving image or a still image is compressed in the moving image or a still image in the moving image or the same image as the parent screen with the same aspect ratio as the parent screen. Further, the place where the child screen is superimposed can be set to an arbitrary place on the parent screen.

  FIG. 8 is a block diagram showing a configuration of a picture-in-picture generation apparatus that is a conventional image synthesis apparatus.

  In FIG. 8, the picture-in-picture generation apparatus 10 includes an image enlargement unit 11, an image reduction unit 12, and an image composition unit 13.

The image enlargement unit 11 enlarges an image composed of 176 × 144 pixels as shown in FIG. 3 to 352 × 288 pixels, for example, and the image reduction unit 12 is composed of 352 × 288 pixels as shown in FIG. Image to 176 × 144. The image composition unit 13 synthesizes the image (child screen) output from the image reduction unit at a specific position of the image (parent screen) output from the image enlargement unit as shown in FIG.
JP-A-6-30335

  However, in such a conventional image synthesizing apparatus, there is a case where the child screen covers and hides the important image area of the parent screen, and in this case, the important area of the parent screen cannot be seen. There's a problem.

  The present invention has been made in view of such a point, and the user does not completely cover the important image area of the parent screen, and the user can determine what kind of image the important area of the parent screen is. An object is to provide an image composition apparatus and an image composition method that can be determined.

  The image composition apparatus of the present invention includes an image input means for inputting an image, an area detection means for detecting an area satisfying a predetermined condition from the image input by the image input means as an important area, and an input by the image input means. When synthesizing the plurality of images, if the input image overlaps the important area, an image synthesizing unit that transparently synthesizes the input image is employed.

  The image composition method of the present invention includes an image input step, a step of detecting a region satisfying a predetermined condition from the input image as an important region, and an input image when combining a plurality of input images. Includes an image synthesis step of transparently synthesizing the inputted image.

  From another viewpoint, the present invention is a program for causing a computer to execute the steps of the image composition method.

  According to the present invention, the image to be synthesized can be transparently synthesized so that the important area of the image to be synthesized is not completely obscured by the image to be synthesized. The user can discriminate. Thus, it is possible to understand both the image content of the composite image (for example, the child screen) and the image content of the important region of the composition source (for example, the main screen), and to provide a user-friendly device that is user-friendly. Can do.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of an image composition device according to Embodiment 1 of the present invention. The image composition device according to the present embodiment is an example applied to a digital camera.

  In FIG. 1, an image composition apparatus 100 includes an image input unit 101 that inputs an image, an area detection unit 102 that detects an important region from the input image, and an image composition unit 103 that combines a plurality of input images. The image output unit 104 mainly outputs a synthesized image.

  The image input unit 101 includes a plurality of image input units (1), (2),..., (N), and can input a plurality of images. For example, an image from an input device such as a camera, an image decoded by a decoding device such as a TV tuner, a DVD player, or a VTR, an image data downloaded from a network, an ITU-T used for a TV phone, etc. An image obtained by decoding a bitstream encoded according to a standard such as recommendation H.261, H.263, or MPEG-4 can be used as an input. When inputting an image, enlargement / reduction processing, contour enhancement processing, color conversion (YUV → RGB conversion, etc.), color increase / reduction processing, D / A conversion processing, and the like may be performed.

  The area detection unit 102 detects an area that satisfies a predetermined condition as an important area from the input image stored by the image input unit 101. The detected important area information is transferred to the image composition unit 103. The target object of the important area may be the whole person, a human face, or a partial area such as a hand, foot, eye, nose, ear, or mouth. Moreover, it may be not only a human but also an animal or an artificial object such as a car or a train. As a method for detecting these important areas, for example, a template generated from an object collected in advance is prepared. Then, the correlation of each part of the image is obtained by superimposing the template on the input image by enlarging or reducing the template. As a result of applying a template to each part of an image, there is a method of detecting a highly correlated part having a correlation with a template or a predetermined threshold or more as an important region. In addition, as a separate input device, an important region may be detected using a distance image from a range finder capable of measuring distance, or an object may be detected using information from a temperature sensor.

  The image synthesis unit 103 synthesizes the plurality of images input by the image input unit 101 and the important area information detected by the area detection unit 102. Here, the image composition unit 103 can perform the transmission composition, and performs the transmission composition when it overlaps with the important region detected by the region detection unit 102. The transmission composition is, for example, weighted composition when two images are combined. This weight is commonly referred to as the transmission value. For example, the transmission composition of the pixel value A and the pixel value B is expressed by the following equation (1).

Composite pixel value = pixel value A × transmission value + pixel value B × (1−transmission value) (1)
Further, at the time of composition, enlargement / reduction processing, contour enhancement processing, color conversion (YUV → RGB conversion, etc.), color increase / reduction processing, D / A conversion processing, etc., for a plurality of images input by the image input unit 101 May be applied.

  The image output unit 104 receives the composite image synthesized by the image synthesis unit 103 and outputs it to a display device such as an LCD. Note that conversion to the NTSC or PAL (phase alternation line) format may be performed to enable output to the TV screen.

  Hereinafter, the operation of the image synthesizing apparatus 100 configured as described above will be described.

  Here, as in a video teleconference system, an image captured by the camera of the local station and a bitstream encoded by the MPEG-4 standard are received from the other party, and the decoded image is combined into a picture-in-picture composition. It is assumed that the information is displayed.

  2 and 3 are diagrams illustrating an example of an input image of the image composition device 100, and FIGS. 4 and 5 are diagrams illustrating an image composition example of the image composition device 100. FIG.

  An image captured by the local camera is input by the image input unit 101. An example of the input image is shown in FIG. Also, a bit stream encoded by the MPEG-4 standard from the game is decoded by a decoding device (not shown) and input by the image input unit 101. An example of the input image is shown in FIG. At the time of picture-in-picture synthesis, the local camera image is reduced to an image having a size of 352 × 288 pixels to 176 × 144 pixels, and is synthesized as coordinates (154, 110) as a child screen. It is assumed that the game decoded image is enlarged from a size of 176 × 144 pixels to 352 × 288 pixels and is synthesized as a parent screen as shown in FIG.

  The area detection unit 102 detects an important area of the game decoded image input by the image input unit 101. In FIG. 3, it is assumed that the important area is detected as an area shown in a solid line frame surrounding the person.

  The image composition unit 103 determines from the information indicating the important region from the region detection unit 102 that the sub-screen composition position overlaps with the important region. Perform synthesis. The transmission value in this example is 0.5, but the value may be changed adaptively.

  The image output unit 104 outputs the combined image combined by the image combining unit 103 to a display device such as an LCD.

  FIG. 6 is a flowchart illustrating an image processing method of the image composition device 100. In the figure, S indicates each processing step of the flow.

  6, the image input process (step S1), the area detection process (step S2), the image composition process (step S3), and the image output process (step S4) are respectively the image input unit 101 and the area detection unit shown in FIG. 102, the image synthesizing unit 103, and the image output unit 104.

  The operation will be described more specifically based on the above assumption.

  An image photographed by the camera of its own station is input by the image input process in step S1. An example of the input image is shown in FIG. Also, the bit stream encoded by the MPEG-4 standard from the game is decoded by the decoding device and input by the image input process (step S1). An example of the input image is shown in FIG.

  Next, in the area detection process in step S2, an important area of the game decoded image input by the image input process (step S1) is detected. In FIG. 3, it is assumed that the important area is detected as an area indicated within a solid frame.

  Next, in the image composition process in step S3, it is determined from the important area information detected by the area detection process (step S2) that the child screen composition position overlaps with the important area, and the child composition as shown in FIG. In the case of screen composition, transparent composition is performed. The transmission value in this example is 0.5, but the value may be changed adaptively.

  Finally, in the image output process (step S4) in step S4, the synthesized image synthesized by the image synthesis process (step S3) is output to a display device such as an LCD, and this flow is finished.

As described above, according to the first embodiment, an image to be combined can be transparently combined so that an important region of the image to be combined is not completely obscured by the image to be combined. It is possible to understand the image contents of the important area. In other words, since the important area of the main screen is transparently synthesized, it is excellent in that the user can determine what kind of image the important image area is without completely obscuring the important image area. Have The present invention having such excellent features is preferably applied to an image device in a TV conference system, a stationary cable videophone device, a portable videophone device in a mobile communication system, and the like.
(Embodiment 2)
FIG. 7 is a diagram showing an image composition example of the image composition apparatus according to Embodiment 2 of the present invention. Since the components are the same as those in the first embodiment (FIG. 1), the description thereof is omitted.

  The difference from the first embodiment is that the image composition unit 103 does not transparently compose the entire child screen as shown in FIG. 5, but overlaps the important region detected by the region detection unit 102 as shown in FIG. The purpose is to synthesize only the area.

  6 is different from the flowchart of FIG. 6 in the image composition process (step S3), as shown in FIG. 5, the entire child screen is not transparently synthesized, but the area detection process (step S2) as shown in FIG. Only the region that overlaps the important region detected by is transparently synthesized.

  As described above, according to the second embodiment, since only the important region of the parent screen is transparently combined, the image portion to be transparent combined is smaller than that of the first embodiment, and combined with the image content of the combined image. This has an excellent effect in that the user can further understand the image contents of both of the original important area image contents.

  The above description is an illustration of a preferred embodiment of the present invention, and the scope of the present invention is not limited to this.

  Further, the present invention can be applied to any apparatus as long as it is an electronic apparatus having an image composition apparatus and method. For example, in addition to digital cameras and digital video cameras, videophone devices, mobile phones with cameras, personal digital assistants such as PDA (Personal Digital Assistants), information processing devices such as personal computers, cameras (built-in / external) It is applicable also to the apparatus provided with. Further, the image to be combined may be any image data.

  Furthermore, the present invention can be incorporated as a mobile terminal, for example, and can also be realized as a mobile communication system including an image composition device and method.

  In each of the above embodiments, the names “image composition apparatus and method” are used. However, this is for convenience of explanation, and it is needless to say that an image display apparatus, an image composition method, an image processing method, and the like may be used.

  Further, it goes without saying that the image display shown in FIGS. 2 to 6 is an example, and other examples may be used.

  Further, the type, number, connection method, and the like of each circuit unit constituting the image composition device are not limited to the above-described embodiments.

  The image composition apparatus and method described above can also be realized by a program for causing these image composition apparatuses and methods to function. This program is stored in a computer-readable recording medium.

  An image composition apparatus and method according to the present invention provides a plurality of moving images and still images on one screen in an image device in a TV conference system, a stationary cable videophone device, a portable videophone device in a mobile communication system, and the like. It is useful as an image composition apparatus and an image composition method to be combined on top.

1 is a block diagram showing a configuration of an image composition device according to Embodiment 1 of the present invention. The figure which shows the input image of the image synthesizing | combining apparatus which concerns on the said Embodiment 1. FIG. The figure which shows the input image of the image synthesizing | combining apparatus which concerns on the said Embodiment 1. FIG. The figure which shows the image composition example of the image composition apparatus which concerns on the said Embodiment 1. FIG. The figure which shows the image composition example of the image composition apparatus which concerns on the said Embodiment 1. FIG. Flow chart showing processing of the image processing method of the image composition device according to the first embodiment. The figure which shows the image composition example of the image composition apparatus which concerns on Embodiment 2 of this invention. Block diagram showing the configuration of a conventional image composition device

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image composition apparatus 101 Image input part 102 Area | region detection part 103 Image composition part 104 Image output part

Claims (6)

An image input means for inputting an image;
An area detecting means for detecting an area satisfying a predetermined condition from the image input by the image input means as an important area;
When combining a plurality of images input by the image input means, the image combining means for transparently combining the input images when the input image overlaps the important region is provided. Image composition device.   2. The image synthesizing apparatus according to claim 1, wherein the image synthesizing unit transparently synthesizes only the input image portion in an area overlapping the important area.   The region detecting means applies a template generated from an object collected in advance to each portion of an input image and detects an image portion having a correlation equal to or higher than a predetermined threshold as the important region with the template. The image composition device according to claim 1 or 2, wherein Inputting an image;
Detecting a region satisfying a predetermined condition from the input image as an important region;
An image synthesizing method comprising: an image synthesizing step for transparently synthesizing the input image when the input image overlaps the important region when synthesizing the plurality of input images.   5. The image synthesizing method according to claim 4, wherein in the image synthesizing step, only the input image portion in an area overlapping the important area is transparently synthesized. A program for causing a computer to execute the steps according to claim 4 or 5.

Images