JP4565504B2 - Image synthesizer - Google Patents

Description

  The present invention relates to an image synthesizing apparatus that obtains a similar effect by synthesizing an image obtained by continuously taking an effect of multiple exposure photographing, which is a photographing method of a silver salt camera, at a constant exposure period in an electronic imaging apparatus.

  As a conventional silver halide camera photographing method, there is a method called multiple exposure in which a plurality of images are obtained by exposing a plurality of times on the same film. To perform multiple exposure, set the valve (long exposure because the shutter is open) with the front of the lens covered with a black curtain, and then perform exposure only while removing the front curtain in front of the lens. After exposure, cover the front of the lens again with a black screen. Multiple exposure images can be obtained by repeating this operation a plurality of times.

  A digital camera is known as an electronic imaging device. This digital camera can set a recording mode and a reproduction mode, and in the recording mode state, an image pickup device such as a CCD arranged behind the lens outputs a photoelectric conversion output for one screen every fixed period. The photoelectric conversion output is sampled and held by a sample and hold circuit, converted to digital data by an A / D converter, color processed by a color process circuit, and digital luminance and color difference multiplexed signal (YUV data) are converted into DMA ( Direct Memory Access) is output to the controller. The DMA controller once writes the YUV data output of the color process circuit into the buffer inside the DMA controller using the synchronization signal, memory write enable, and clock output of the color process circuit, and DMA-transfers it to the DRAM via the DRAM I / F.

  After completing the DMA transfer of the YUV data to the DRAM, the CPU reads this YUV data from the DRAM via the DRAM I / F and writes it to the VRAM via the VRAM controller. A digital video encoder (hereinafter simply referred to as a video encoder) periodically reads the YUV data from the VRAM via the VRAM controller, generates a video signal based on the data, and outputs the video signal to the display device. As a result, an image based on the image information captured from the current CCD is displayed on the display device.

  In this way, when the shutter key is operated at the timing at which recording and storage are desired while the image is currently displayed on the display device, a signal is generated. Then, the CPU immediately stops the path from the CCD to the DRAM after the DMA transfer of the YUV data for one screen captured from the current CCD to the DRAM in response to this signal, and shifts to the record storage state. In this recording and storage state, the YUV data for one frame written in the DRAM by the CPU is called a basic block of 8 pixels vertically by 8 pixels horizontally for each component of Y, Cb, and Cr via the DRAM I / F. The data is read in units and written to the JPEG circuit, and the code data compressed by the JPEG circuit is read from the JPEG circuit and written to the flash memory, which is a nonvolatile memory. Then, along with the compression processing of one frame of YUV data and the completion of writing all the compressed data to the flash memory, the CPU activates the path from the CCD to the DRAM again.

  In the playback mode, the CPU stops the path from the CCD to the DRAM, and when the image selection key or the like is pressed, the CPU reads a specific frame of code data from the flash memory, writes it to the JPEG circuit, and decompresses it with the JPEG circuit. One frame of YUV data is developed into VRAM via the VRAM controller in units of 8 × 8 basic blocks obtained by processing. Then, the video encoder generates a video signal based on the YUV data for one frame developed in the VRAM and displays it on the display device.

  Since the digital camera as described above can handle captured images as image data, an effect like multiple exposure has been realized by combining all of a plurality of images obtained by a plurality of shootings.

However, in order to perform multiple exposure in such a conventional silver salt camera, the front of the lens is covered with a black curtain or the like (the shutter is in an open state because the shutter is open), and the front of the lens is set. Repeating the operation to remove or cover the black curtain multiple times is an extremely time-consuming task, such as obtaining a multiple-exposure image. In digital cameras, the shutter is a valve (like the silver salt camera, because the shutter is open). It was difficult to set the time exposure state) to shoot and to superimpose an image on the CCD because of discharge of electric charges .

The present invention has been made in view of such a conventional problem, and an object of the present invention is to provide an image composition device capable of obtaining a composite image by a very simple operation .

In order to solve the above-described problems, the invention described in claim 1 is directed to an imaging unit that captures a subject and continuously acquires a plurality of image data, and a predetermined value for each image data acquired continuously by the imaging unit. Determining means for determining whether or not there is a pixel having a level equal to or higher than the first reference luminance level, and synthesizing means for combining the image data determined by the determining means that there is a pixel having a level equal to or higher than the predetermined first reference luminance level It is characterized by having.

According to a second aspect of the present invention, in the first aspect of the present invention, the combining unit adds a luminance level obtained by adding the luminance levels of the pixels at the same position between the image data to be combined. The brightness level is set to the pixel position .

According to a third aspect of the present invention, in the second aspect of the present invention, when the summed luminance level exceeds a predetermined maximum luminance level, the synthesizing unit determines a pixel position that exceeds the predetermined maximum luminance level. The luminance level is set to the maximum luminance level.

According to a fourth aspect of the present invention, in the first aspect of the present invention, the synthesizing unit compares the luminance levels of the pixels at the same position between the image data to be synthesized, and determines the larger one. The brightness level is set to the brightness level of the pixel position.

According to a fifth aspect of the present invention, in the invention according to any one of the second to fourth aspects, the synthesizing unit has a luminance level of a pixel at a position that coincides between the respective image data to be synthesized. When both are below a predetermined second reference luminance level, the luminance level at the pixel position is set to an average value of the luminance levels of the pixels that are both below the predetermined second reference luminance level. Yes.

According to the present invention, a composite image can be easily obtained, and photography of fireworks, celestial bodies (meteors), and the like can be performed.

Further, it is possible to prevent the dark part of the image from becoming bright, and it is possible to obtain a composite image with high contrast.

  Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of the apparatus. In the recording mode, an image captured by the CCD 1 is sent to the YUV processor 3 by the CCD controller 2 that controls the CCD. Image data that has undergone color process processing by the YUV processor 3 is sent to the image memory 4 via the memory controller 5. At the same time, the captured image data for displaying the through image is also sent to the video encoder 6, which generates a video signal based on the image data sent from the memory controller 5 and outputs it to the video output unit 7. . The key processing unit 10 processes key input.

  Here, when the automatic multiple exposure mode is set, the CPU 8 determines whether or not there is a pixel of a determination level or higher in the captured image, and if there is even one such pixel, the captured image. Is the target of image composition. As described above, since the pixels having the determination level or higher exist, the images to be combined are sequentially combined on the image memory 4. When the automatic multiple exposure mode ends, the combined image data is written and stored in the recording medium 9. In this way, since the image synthesis is performed by discriminating pixels above a certain level from the captured image, a synthesized image can be easily obtained.

  FIG. 2 shows the timing of exposure of the CCD 1 of this apparatus, transfer of the captured image, and discrimination of the synthesis target image. The exposure of the CCD 1 and the image transfer to the image memory 4 are performed with the frame pulse F output every frame period as a reference. In this embodiment, it is set to obtain one image with an exposure time of 4 frames. When the exposure for four frame times is completed, the image is immediately transferred from the CCD 1 to the image memory 4. Of the images transferred every four frames, an image including a pixel having an image combination determination level or higher is an object to be combined. Instead of synthesizing all images, only the necessary images are selected and synthesized, so that efficient image synthesis can be performed without complications. Further, although the exposure time is fixed to a 4-frame cycle, the exposure time is not limited to 4 frames.

  FIG. 3 shows a method for determining an image to be synthesized as to whether or not there is a pixel of a certain level or higher in the image captured from the CCD 1. For the sake of clarity, each pixel is assumed to have monochrome levels of 0 to 255 (8 bits). The level of each pixel corresponds to the luminance or shading of the pixel. The captured image 11 is composed of 4 to 13 level pixels. The determination level is set to 100, and the image 11 is an image that is not an object of image synthesis because there are no pixels of level 100 or higher. The captured image 12 is composed of pixels of levels 2 to 156. The image 12 has five pixels of level 100 or higher. Accordingly, since there is one or more pixels of level 100 or higher, the image 12 is a target for image synthesis.

  As another determination method, the determination may be made based on the average value of the levels of all the pixels in the image, or may be determined by the number of pixels such as how many pixels of a certain level or more exist in the image. In addition, as described above, FIG. 3 illustrates an example of a monochrome image (when each pixel has one level). However, a similar image is applied to a color image in which each pixel is composed of three types of RGB. The synthesis method can be used. In this case, each pixel level for discriminating the image to be synthesized uses a value obtained by adding three values of RGB for each pixel, and in composition of the image, each RGB value is set for two corresponding pixels. What is necessary is just to add separately and to obtain | require the RGB value after a synthesis | combination.

  FIG. 4 shows an image composition method. A method of obtaining a composite image by simple addition of levels will be described. The image 13 is an image that is an image synthesis target and is stored in the image memory 4 in advance, and the image 14 is a new image synthesis target image. Pixels corresponding to images 13 and 14 are added in order to obtain a composite image. For example, in the upper left pixel, image 13 has level 11 and the corresponding upper left pixel of image 14 has level 8. When these are added, it becomes level 19 as in the upper left pixel of the image 15. Similarly, looking at the pixel on the right side, the level is 9 in the image 13 and the level 7 in the image 14, so the level is 16 as shown in the image 15. Similarly, when the process is repeated for each pixel, an image 15 synthesized from the image 13 and the image 14 is obtained.

  Although the pixel level does not exist in FIG. 4 and the pixel level becomes 255 or more as a result of the addition, the pixel of the image 13 is level 181 and the pixel level of the image 14 is 177, and the addition result is 258. In this case, the portion 3 which is 255 or more of the added result is trimmed to 255. As described above, by determining the image to be combined, it is possible to perform image combining by simple pixel level addition. Further, as a method of combining images, the levels of the corresponding pixels in the two images to be combined may be compared and the level of the larger pixel may be selected.

  FIG. 5 is a flowchart showing the processing operation of the CPU. First, the CPU counts the frame counter to determine whether or not four frames of exposure time have elapsed (S1). If four frames have passed, the image data from the CCD 1 is transferred to the image memory 4 (S2). The image data fetched into the image memory 4 is sent to the video encoder 6 for through image display (S3). Next, as described with reference to FIG. 3, it is determined whether or not there is a pixel of a determination level or higher in the captured image (S4). As a result, if even one pixel above the judgment level exists, that image is set as an image to be combined, and is added for each pixel as described in FIG. (S5). As described above, images are taken continuously at a constant exposure cycle, and only images with pixels that are above the discrimination level in the captured image are targeted for synthesis. Equivalent images can be obtained. Especially effective for shooting fireworks and celestial bodies (meteors).

  FIG. 6 shows a method for obtaining a composite image with high contrast. Since the simple addition of levels, which is the synthesis method shown in FIG. 4, is performed even for the level of a low-level pixel, if the image synthesis is repeated many times, the dark part of the image (the part with the low value of the pixel level) ) Will gradually become brighter. Even if the pixel level is low, a constant reference level is set, and if the levels of the corresponding pixels in the two images to be combined are both below a certain level, simple addition is not performed, but the average value of the levels of both pixels How to take.

  A constant reference level 16 is assumed. The level of the upper left pixel of the image 16 is 11, and the level of the upper left pixel of the corresponding image 17 is 8. Here, since both pixels are below the reference level 16, (11 + 8) /2=9.5, and the fractional part is rounded down, the average value of both pixels is 9, and the level of the pixel of the synthesized image 18 is 9. . Similarly, since both pixels on the right side are also below the reference level 16, (9 + 7) / 2 = 8, the average value of both pixels is 8, and the level of the pixel of the synthesized image 18 is 8. If at least one of the two pixels is at the reference level 16 or higher, simple addition is performed as shown in FIG. In the pixel in the second row from the top of the image 16 and the third column from the left, the pixel level is 167 and the corresponding pixel level of the image 17 is 5. Here, since one pixel is above the reference level 16, simple addition is performed, and the level of the pixel of the image 18 synthesized by 167 + 5 = 172 becomes 172.

  By changing the compositing method by providing a fixed reference level for the low pixel level in this way, it is possible to prevent the dark part of the image from becoming bright due to repeated image compositing, and an image with high contrast can be obtained. Can be obtained. In the above embodiment, the average value of the levels of both pixels is used if the levels of the corresponding pixels in the two images to be combined are both below a certain level. However, the levels of the corresponding pixels are both constant. In the following cases, a method of selecting a level of a pixel having a smaller level may be used.

It is a block diagram of this invention apparatus. It is a figure which shows the timing of an automatic multiple exposure process. It is a figure which shows the discrimination method of the image of a synthetic | combination object. It is a figure which shows the synthetic | combination method of an image. It is the flow which showed the process of CPU. It is a figure which shows the synthetic | combination method which obtains an image with high contrast.

Explanation of symbols

1 CCD
2 CCD control unit 3 YUV processor 4 Image memory 5 Memory controller 6 Video encoder 7 Video output unit 8 CPU
9 Recording medium 10 Key processing part

Claims (5)

An imaging means for imaging a subject and obtaining a plurality of image data continuously;
Determining means for determining whether or not there is a pixel having a predetermined first reference luminance level or higher for each image data continuously acquired by the imaging means;
Synthesizing means for synthesizing each image data determined by the determining means that there is a pixel equal to or higher than the predetermined first reference luminance level;
An image composition device comprising:   2. The luminance level at the pixel position is set to a luminance level obtained by adding luminance levels of pixels at positions that coincide with each other between the image data to be synthesized. Image synthesizer.   The synthesizing unit, when the added luminance level exceeds a predetermined maximum luminance level, sets a luminance level at a pixel position exceeding the predetermined maximum luminance level to the maximum luminance level. Item 3. The image composition device according to Item 2.   The synthesizing unit compares the luminance levels of pixels at positions that coincide with each other among the image data to be synthesized, and sets the larger luminance level as the luminance level of the pixel position. Item 6. The image composition device according to Item 1. The synthesizing unit sets the luminance level of the pixel position when both the luminance levels of the pixels at the positions that coincide with each other between the image data to be synthesized are equal to or lower than a predetermined second reference luminance level. 5. The image synthesizing apparatus according to claim 2, wherein the image synthesizing apparatus is set to an average value of luminance levels of pixels that are equal to or lower than a predetermined second reference luminance level.

Images