JP2009010566A - Method and imaging apparatus for expanding dynamic range of photographic image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate an image having no false contour and no sense of incongruity when synthesizing images photographed by changing exposure quantities to be a high dynamic image and to automatically set thresholds of gray values suitable to divide a gray scale image into areas according to the high and low of the gray values. <P>SOLUTION: An image of a reference exposure value among a plurality of images with different exposure values is converted into a gray scale image, the gray scale image is divided into a plurality of areas corresponding to the high and low of gray values according to thresholds, classifying the plurality of areas by colors, the color composition of each pixel is calculated in an image subjected to equalization filter processing, and gray values of a plurality of images with different exposure values are selected in accordance with the color component of each pixel to synthesize the images. In addition, an object boundary image described from an edge generated from the gray scale image, an exposure boundary image describing an exposure boundary being the boundary of an area divided by the high and low of the gray values, and set points rule are used to calculate overlapping points about each pixel, and gray values at which overlapping points become maximum within the range of prescribed gray values are defined as a threshold. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a method and an imaging apparatus for expanding a dynamic range of a photographic image.

  There is a difference between a photograph taken with a film camera or a digital camera, and a visual image when a person looks directly at the subject. One of the factors is that the dynamic range in photography differs from the dynamic range in human vision, and the dynamic range in human vision is much wider than in photography.

  Humans adjust the amount of light incident on the retina by changing the size of the pupil, preventing the image of the gaze area from being too bright or too dark, and in various gaze areas that have been altered by eye movements. The amount of incident light can be appropriately maintained by appropriately changing the size of the pupil according to the brightness. It is considered that the image stored in the brain is temporally synthesized by using such adjustment ability to generate a pseudo high dynamic range image that exceeds the dynamic range of the photoreceptor.

  The dynamic range of the photosensitive material used in photography and the image pickup device (CCD, CMOS) of a digital camera is narrower than such a pseudo-hyperdynamic range in human vision. Even if an image is recorded with a CCD or CMOS that has a dynamic range greater than that of human vision, image information that is similar to that of human vision is generated, even though unclear whitening and blackout occur. It is difficult to do.

  For example, consider the case where a landscape such as that shown in FIG. A bowl is placed on the handrail on the front side, the mountain looks thin as a distant view, and the sky is above it. In the middle is a landscape of buildings, trees, etc. in the middle, the sun is shining from the right front side, it is backlit, the front side of the bowl is shaded, and the cityscape has many shaded parts The contrast is high.

  In such a landscape, when shooting with automatic exposure using a digital camera with center-weighted average metering, the exposure is adjusted so that the brightness of the area centered on the center is appropriate, and overexposure occurs in the sky. As a result, black spots are seen in the dark parts of the streets and behind the bowls. This is due to the narrow dynamic range of the image sensor and output device built in the digital camera.

  It is possible to change the way the image is taken by changing the exposure amount by exposure correction. To make the front bowl and townscape brighter, adjust the overexposure. In this case, the sky and the circular thin-looking mountain range will also become white and will not be visible. Conversely, to make it possible to see a circular mountain or the appearance of clouds in the sky, correct the exposure to the under side. In this case, the pots and the streets become darker.

  Thus, within the range of the dynamic range of the image sensor of the digital camera, there is a restriction that a particularly bright part or a particularly dark part must be difficult to see in a captured image for an image with high contrast. . When a human sees the same scene directly, the bright and dark parts can be visually recognized by the ability to adjust with the eyes. Since images recorded with a camera are recorded without such adjustments, when shooting a high contrast landscape as a single image, overexposure and underexposure occur. When the exposure correction is performed with emphasis on the bright or dark part in the angle of view, the tendency of overexposure or underexposure is not eliminated in the part that is not important.

  When shooting an image of a high-contrast landscape, the overexposure and underexposure will occur. For the same landscape, take multiple shots with different exposures and combine them to create a single image with no overexposure or underexposure. There is a known technique for making an image. This is done using a digital camera having an auto bracket function capable of continuously performing a plurality of times of photographing with the same scene while changing the exposure amount with a digital camera having an exposure amount auto bracket function.

  The following document describes the expansion of the dynamic range by combining images taken by multiple exposures.

  In Patent Document 1, light from one lens system is split into a plurality of images by an optical distributor, input from each image input means to the image processing means, and a shadow portion is sharpened by one image input means. , A dynamic range expansion device is described in which a non-shadow portion is clearly input by the other image input means, and these sharp portions are combined into one image by the combining means.

  Patent Document 2 describes an imaging apparatus that generates a wide dynamic range composite image by combining a plurality of image signals having different exposure amounts.

  In Patent Document 3, in a digital camera having a function of performing gradation enlargement processing for synthesizing a plurality of image signals photographed with different exposure amounts, the luminance of a measurement object is detected and measured by a photometric element having a wider dynamic range than an image sensor. It is shown that it is determined whether or not the gradation enlargement process is performed based on the luminance difference between the central portion and the peripheral portion.

Non-Patent Document 1 describes an HDR method for restoring a wide range of luminance signals from a plurality of images with different shutter speeds and apertures. In order to accurately calculate a characteristic curve of a camera, The number of images is required, and the shutter speed and aperture information of each image must be acquired. Also, the luminance information calculated in this way is not necessarily close to a human visual image, and the generated HDR image has luminance information of 8 bits or more, so that it is displayed on a monitor compliant with 8 bits. I can't.
JP-A-6-105224 JP 2000-92378 A JP 2002-290824 A Paul E.M. Devevec et al., “Recovering High Dynamic Range Radiance Map from Photographs, Computer Graphics Proc.” Annual Conference series (1997), pp. 197 369-378 When generating a pseudo high dynamic image using the auto bracket function, an exposure value (EV) obtained by automatic exposure is used as a reference exposure image, and the exposure value is lowered from the reference exposure value for the same subject. Take three images in succession: an image (underexposed image), a reference exposure image, and an image with an exposure value higher than the reference exposure value (overexposure image), and combine these three images with different exposure values. Think about expanding the dynamic range. Here, expanding the dynamic range does not mean widening the dynamic range of the image sensor, but rather increasing the brightness range on the subject side by image composition so that there is no overexposure or underexposure. The image synthesized in this way is more accurately called a pseudo high dynamic range image.However, based on this understanding, the image synthesized in such a way as to increase the luminance range on the subject side will be described below. This is called a dynamic range image.

  At the time of image composition, it is necessary to set an image area selected from three images. For this purpose, a gray value range defined by a lower limit value and an upper limit value is designated in the gray value range 0 to 255 in the image of the reference exposure image. For example, when the lower limit value is 100 and the upper limit value is 220, when synthesizing a high dynamic range image, paying attention to the gray value of the reference exposure image, if the gray value of the pixel in the image is within the specified range, the standard exposure The gray value of the image. If the gray value is in a range lower than the specified range, the gray value in the overexposed image is used. If the gray value is in the higher range, the gray value in the underexposed image is used. In this way, the gray value of each pixel is selected. In this manner, a high dynamic range image is synthesized from images with three exposure values.

  The same applies when shooting a landscape as shown in FIG. 1. For the same landscape, three images of an underexposed image, a reference exposure image, and an overexposure image are taken with the auto bracket function and specified for the reference exposure image. As shown in FIG. 2A, when the lower limit value of the tone value range is 100 and the upper limit value is 220, this image is higher than the specified range α in the range in which the tone value is specified. The region β is divided into three regions γ lower than the specified range. In the region α, the gray value in the reference exposure image is selected, in the region β, the gray value in the underexposure image is selected, and in the region γ, the gray value in the overexposure image is selected.

  Here, the specified gradation value range is set to 100 to 220, but the lower limit value for specifying the gradation value range is set to 40, and the upper limit value 120 is set according to the condition of the gradation value in the reference exposure image, the contrast in the subject, and the like. Then, the regions α, β, and γ depending on the gray value become different as shown in FIG. In this way, the composition of the regions α, β, and γ depending on the gray value changes, so that the synthesized high dynamic range image gives a different impression.

  In a high dynamic range image obtained by combining multiple exposures using the digital camera with different exposure amounts for the same subject, an underexposed image, a reference exposed image, and an overexposed image can be selected as a single image. The boundary of the image area in which the density value is selected from different images is the boundary of the image with different exposure amount, and the pseudo contour that cannot be seen as the original one image due to the discontinuity And the synthesized image becomes unnatural and uncomfortable with respect to the boundary between regions selected from images with different exposure amounts.

  In Patent Documents 1 and 3 and Non-Patent Document 1, no particular consideration is given to pseudo contours generated during such image composition. In Patent Document 2, when the exposure amount ratio of a plurality of images is not a set value, the discontinuity of the gradation of the next portion is likely to occur. Therefore, the actual exposure amount ratio is obtained and this exposure is performed. Although the composition processing is performed by multiplying the image data with a small amount, there is no technique that can effectively process the discontinuity at the next stitch that can generally occur in the synthesized image.

  In this way, when combining multiple images shot with a conventional digital camera with different exposure amounts, it is effective for creating a high dynamic range image that can be said to be close to human vision by making the pseudo contour inconspicuous. No measures were taken.

  Also, the threshold value (upper limit value, lower limit value) of the tone value range that specifies which tone value of the underexposed image, the reference exposed image, or the overexposed image is selected to be combined as a high dynamic range image. Although how to set affects the synthesized image, a method for appropriately performing the threshold value of the gray value range has been demanded.

  The present invention has been made to solve the above-mentioned problems, and a method for expanding the dynamic range of a photographic image according to the present invention is a method of shooting a plurality of exposure values obtained by changing the exposure value for the same subject. A method for expanding the dynamic range of a photographic image by combining the image data of the image of the image, wherein an image of a reference exposure value among a plurality of images having different exposure values is converted into a grayscale image; Dividing the converted grayscale image into a plurality of regions according to the level of the gray value according to a set gray value threshold; and color-coding the plurality of regions of the divided image in the gray scale image; Generating an image obtained by performing an averaging filter process on the color-coded image, and obtaining a color component of each pixel in the image subjected to the averaging process It and the said selecting image synthesizing gray value of a plurality of different images of the exposure value according to the color component of each pixel, is made of.

  When the grayscale image is divided into a plurality of areas based on a gray value threshold, an object is depicted with an edge as an object boundary by applying an edge detection algorithm to the grayscale image obtained by converting the reference exposure value image A boundary of an area obtained by generating a boundary image and dividing the grayscale image according to the level of the gray value with the gray value as a parameter for the gray scale image obtained by converting the reference exposure value image as a parameter An exposure boundary image depicting the exposure boundary is generated, score rules are set for the object boundary in the object boundary image and the exposure boundary in the exposure boundary image, and the reference value is set for each gray value as a parameter. An overlap score is obtained for each pixel in the image based on the score rule, and the overlap score is determined within a predetermined gray value range. Said gray value becomes larger as the threshold value in the gray value range in accordance with the gray-scale image of the level of gray values may be divided into a plurality of regions.

  The image pickup apparatus according to the present invention is an image pickup apparatus having a function of generating an image with an expanded dynamic range by combining a plurality of images obtained by shooting with different exposure values by an auto bracket function. , Setting means for setting photographing conditions and gray value thresholds, storage means for storing a plurality of image data with different exposure values photographed and image data converted therefrom, and exposure as a reference A gray-scale processing unit that converts an image of a value into a gray-scale image, and the converted gray-scale image is converted into a plurality of regions according to the level of the gray value based on the gray-value threshold value set by the setting unit. An area division color coding processing unit for color-dividing each area, and an average file for performing an averaging filter process on the color-coded image generated by the area division color coding processing unit Image composition for obtaining a color component of each pixel in the image obtained by the processing unit and the averaging filter processing unit, selecting gray values of a plurality of images having different exposure values according to the color component of each pixel, and performing image composition Part.

  Appropriate threshold automatic setting for automatically setting an appropriate threshold for dividing the grayscale image into a plurality of areas by the threshold of the gray value when the setting means is set to automatically set the threshold of the gray value An appropriate threshold value automatic setting unit that applies an edge detection algorithm to a grayscale image obtained by converting the reference exposure value image to generate an object boundary image in which the edge is depicted as an object boundary. An image generation unit and a boundary between regions obtained by dividing the grayscale image according to the level of the gray value by using the gray value as a parameter and the gray value as a threshold for the gray scale image obtained by converting the reference exposure value image An exposure boundary image generation unit that generates an exposure boundary image depicting a certain exposure boundary, and an object boundary in the object boundary image and the exposure boundary image A predetermined score rule table for the exposure boundary and an overlap score for each pixel in the image of the reference exposure value for each gray value as a parameter is obtained based on the score rule table to obtain a predetermined score. You may make it comprise the threshold value selection means which selects the gray value with which the said overlap score becomes the maximum in a gray value range as a threshold value in the said gray value range.

  In the present invention, an image having a reference exposure value among a plurality of images having different exposure values is converted into a gray scale, and the image is subjected to low-pass filter processing by dividing and color-dividing the image with a threshold value of gray value. Since the original image is synthesized, it is possible to suppress the generation of a pseudo contour at the boundary between regions selected from images having different exposure values, and to generate a high dynamic range image without a sense of incongruity. Further, in the generation of such a high dynamic range image, it is possible to automatically set a gray value threshold appropriate for dividing a gray scale image into regions based on the gray value.

[Generate high dynamic range image]
Consider a case where three images with different exposure values, which are underexposed, reference exposed, and overexposed, are shot using the auto bracket function. Normally, underexposure is -1EV and overexposure is + 1EV. However, some digital cameras have variable exposure values that can be set to ± 1 / 3EV and ± 1 / 2EV. In some cases, it is possible to select a case where three images are taken and a case where five images are taken. Hereinafter, a case where three images of -1EV, 0EV, and + 1EV are taken will be described.

For a landscape such as that shown in FIG. 1, three images of underexposed (-1 EV), standard exposure (0 EV), and overexposed (+1 EV) are taken, and the 0 EV image is first converted into a grayscale image. Here, considering the human visual characteristics of perceiving yellow as bright and blue as dark, I = 0.299 × R + 0.587 × G + 1114 × B (1)
(However, R: red component, G: green component, B: blue component).

  Next, two threshold values (upper limit value and lower limit value) in the range of the gray value are designated in order to set a region for selecting the gray value in the 0 EV image from the three images. Here, the upper limit value is set to 220 and the lower limit value is set to 100. Then, the image of 0EV has the upper limit value 220 and the lower limit value 100 as the boundaries, as shown in FIG. 2A, the region α in the range in which the gray value is specified, the region β higher than the specified range, and the specified value. The region is divided into three regions γ that are lower than the specified range. Select the gray value in the standard exposure image for area α, the gray value in the underexposed image in area β, and the gray value in the overexposed image in area γ, and select from images with different exposure values at the boundary of the area. The adjacent pixels are adjacent to each other, and a pseudo contour is generated in the synthesized image, which is unnatural and uncomfortable.

  Therefore, in the present invention, by performing a process of blending and smoothing pixels in the vicinity of the boundary, a pseudo contour is not generated at the boundary. Therefore, when the region α is divided into three regions α, β, and γ, the region α is displayed in green (G), the region β is displayed in red (R), and the region γ is displayed in blue (B). The color display indicates which pixel of the exposure value is applied when combining different images. The pixel where the pixel of the image of -1EV is applied is expressed as a color component, and the pixel of the image of red and 0EV is applied when (R, G, B) = (1, 0, 0). The pixel is green when (R, G, B) = (0, 1, 0), and the pixel where the pixel of the image of +1 EV is applied is (R, G, B) = (0, 0, 1) It is blue.

  For example, as shown in FIG. 3 for an RGB color-displayed image divided into three regions α, β, and γ by boundary lines as shown in FIG. 2, which is an image showing regions for selecting gray values. By using the 3 × 3 averaging filter, the gray value in the vicinity of the boundary line is averaged. When a 3 × 3 averaging filter is applied to this image, adjacent pixels are blended in the vicinity of the boundary, and these components are mixed in the vicinity of the boundary line of the image instead of R, G, B as single color components. Become a color. At a position apart from the boundary line to some extent, there is no effect of such blending, and monochromatic R, G and B remain.

  In this way, it is defined how to select a gray value from an image having three different exposure values according to the color component value in an image in which the color is blended in the vicinity of the boundary line by the averaging filter. For pixels that are separated from the boundary line and are represented by single colors R, G, and B, the gray values of the images of −1 EV, 0 EV, and +1 EV are selected, respectively. For pixels with blended components, the distribution of shade values of three images with different exposure values is determined according to the component. For example, in a pixel having RGB components such as (R, G, B) = (0.2, 0.7, 0.1), (−1EV image pixel gray value) × 0.2 + (0 EV image) The gray value of the pixel) × 0.7 + (the gray value of the pixel of the image of +1 EV) × 0.1 is the gray value of the pixel. In this way, in the vicinity of the boundaries of the regions α, β, and γ, it is smoothed by the averaging filter and the gray value component is blended, so that the gray value changes dramatically at the boundary line of the region. The image is synthesized as an image in which an unnatural pseudo contour is not seen, and an image in which no information is lost due to overexposure or underexposure is obtained as a hydramic range image.

  The process of generating the above high dynamic range image is shown as a flow in FIG. The processing flow is shown on the left side of FIG. 4, and the image generated by these processes is shown on the right side. This processing is performed using a general-purpose computer provided with a program necessary for processing the data of an image taken by a digital camera having an auto bracket function.

In this example, three images with different exposure values of -1EV, 0EV, and + 1EV are photographed and synthesized by the auto bracket function, but the exposure value stage has a width of 1/2 EV or 1/3 EV. It can also be changed. Also, depending on the degree of contrast and the content of the landscape, two images with different exposure values can be taken and combined instead of three. In this case, the range of the gray value is divided into two by one threshold value. Even in the case of dividing into three gradation values, depending on how to set two threshold values (upper limit value and lower limit value), the result may be divided into two gradation value ranges.
[Calculation of appropriate threshold value]
When synthesizing a high dynamic range image, it is synthesized depending on how to set thresholds (upper limit value and lower limit value in the case of three levels) that define an area for selecting a gray value in the reference exposure image. The images will change. When a landscape such as that shown in FIG. 1 is photographed, if the lower limit value that defines the area for selecting the gray value is 40 and the upper limit value is 120 in the reference exposure image (0EV), the synthesized image is uncomfortable. However, there is a difference that when the lower limit value is 120 and the upper limit value is 220, the image is synthesized as an image with less discomfort. How to set the upper limit value and the lower limit value depends on the composition of the gray value in the scenery of the subject, and changes for each image.

  Whether the synthesized image has a sense of incongruity or a little incongruity depends on the correspondence between the edge part in the original image taken and the edge part in the color-coded image to separate the gray value area You can see that When the edge portion of the image of FIG. 1 is depicted by the edge detection of Canny, the edge portion (boundary) of the color-coded image having the lower limit value 120 and the upper limit value 220 of FIG. The edge portion (boundary) of the color-coded image having the lower limit value 40 and the upper limit value 120 in FIG. The image synthesized in this way is less uncomfortable because the edge portion obtained by edge detection in the original image (reference exposure image) differs from the boundary of the color-coded image in order to divide the gray value region. It can be seen that this corresponds to a good overlap, and based on this, it is possible to calculate an appropriate threshold value using the information about the edge of the original image and the edge information of the color-coded image for separating the gray value region.

A description will be given of calculating an appropriate threshold value when image synthesis is performed using a reference exposure image (0 EV) of appropriate exposure by the camera and a total of three images with exposure values changed by 1 EV vertically. First, an original image (image with an exposure value of 0 EV) is changed to a grayscale image, and an edge is drawn by applying the Canney edge detection algorithm to the grayscale image, and the pixel at the object boundary is set to 1 with a binary density. Then, an image in which the density of other pixels is 0 (referred to as an object boundary image) is generated. An edge in this object boundary image is defined as an object boundary. Next, considering a gray value (0 to 255) as a parameter, when a certain gray value is used as a threshold, the gray value is divided into regions of the original image based on the threshold and the boundary of the region ( (Exposure boundary) is drawn, and an image (exposure boundary image) in which the pixel of the exposure boundary is 1 with a binary density and the density of other pixels is 0 is generated. Also, as shown in Table 1, the number of points is defined according to whether or not the edge of the original image is the edge (boundary) of the color-coded image. In addition, it is considered that the lower limit value as a threshold value for dividing the gray value area is in the range of the gray value 0 to 127 (low brightness area), and the upper limit value is in the range of the gray value 128 to 255 (high brightness area). The appropriate threshold value is calculated.
(A) Low lightness region While changing the gray value one by one from 127 to 0, all pixels of the original image (0EV) according to the rule of Table 1 for each gray value from each exposure boundary image and object boundary image The score is calculated, and the total value for all the pixels is used as the overlapping score. In the overlap score of each shade value calculated in the range of the shade value 127 to 0, the shade value that maximizes the overlap score is set as an appropriate threshold value (lower limit value in the case of three levels). If there is a threshold having the same aptitude, a smaller value is selected as the threshold.
(B) High Brightness Region Overlapping points are calculated for all the pixels of the original image (0EV) according to the rules in Table 1 while changing the gray value one by one from 127 to 255 as in the case of (a). In the overlap score of the respective shade values calculated in the range of the shade values 127 to 255, the shade value that maximizes the overlap score is set as an appropriate threshold value (upper limit value in the case of three levels). If there is a gray value having the same appropriateness, a larger gray value is selected as the threshold value.

  FIG. 5 is a flowchart showing a procedure for calculating an overlap score for each gray value from the exposure boundary image and the object boundary image, and obtaining an appropriate threshold value from the result. FIG. 5 shows conversion of image data at the center and processing for performing each conversion. In the flow of FIG. 4, the determination of the threshold value of the gray value is set as appropriate based on the image density, the configuration of the contrast, and the like. However, in the flow of FIG. It can be done objectively. In addition, when the overlap score calculated for each gray value is represented as a graph, for example, the shape is as shown in FIG. 6, and the exposure value that is the highest score between the gray values 0 to 127 is the lower limit value, and 128 to 255. Select the exposure value that gives the highest score between the two as the upper limit.

  A high dynamic range image is obtained from two images, either an image with proper exposure (0 EV) set by the camera and an image with a one-step exposure change up or down (+1 EV image or -1 EV image). In the case of generation, either an appropriate threshold value in the low brightness area or an appropriate threshold value in the high brightness area is calculated. The calculation algorithm at this time is the same as that shown above.

  When four or more images having different exposures are used, the process is the same until the appropriate threshold value in the low brightness area and the appropriate threshold value in the high brightness area are calculated using the 0 EV image. After that, when calculating the appropriate threshold for combining two images with different exposures (+ direction), apply the appropriate threshold calculation method in the low-lightness area for the two low exposure images. To do. Conversely, when calculating an appropriate threshold value for combining two images whose exposure is changed downward (-direction), an appropriate threshold value calculation method in a high brightness area is targeted for an image with a high exposure among the two images. Apply. As an example, when calculating an appropriate threshold value for synthesizing a + 1EV image and a + 2EV image, the appropriate threshold value calculation method in the low lightness region shown above is applied to the + 1EV image, and −1EV is applied. When calculating the appropriate threshold value for combining the image of -1EV and the image of -2EV, the method for calculating the appropriate threshold value in the high brightness area shown above for the -1EV image is applied.

  Thus, Table 1 shows the object boundary in the object boundary image and the exposure boundary in the exposure boundary image.

のように得点ルールを設定しておき、パラメータとしての各濃淡値ごとに基準とする画像における各画素につい得点ルールをもとに重なり得点を求め、所定の濃淡値範囲において重なり得点が最大となる濃淡値を濃淡値範囲における閾値として設定し、その閾値により基準とする露出値の画像から変換されたグレースケール画像を濃淡値の高低に応じて複数の領域に分け、さらに色分けした画像をもとに複数の露出値の異なる画像からハイダイナミックレンジ画像を合成するようにすることにより、領域の境界の部分において疑似輪郭が生じることはなく、違和感のないハイダイナミックレンジ画像が得られる。
［ハイダイナミックレンジ画像を生成する撮像装置］
本発明での画像合成によりハイダイナミックレンジ画像を得る機能を備えた撮像装置について説明する。この撮像装置の構成は大略的に、レンズ１、撮像素子（ＣＣＤ）２、撮影動作制御部５を含む撮像部１０と、諸種の設定を行う設定部２０と、画像合成処理を行う画像処理部３０に分けて考えられる。設定部２０では露出値の段数、撮影枚数等の設定を行うとともに、ハイダイナミックレンジ画像の合成の過程でグレースケール画像の領域分割の際に必要な濃淡値の閾値の設定を行う。この設定は撮影者の経験と被写体のコントラストとに応じて手動的に行う場合と、自動的に行う場合とを選択可能にする。撮影動作制御部５は設定部２０において設定された露出値段数、一連の撮影動作における撮影枚数の撮影条件に応じた撮影の動作を制御する。

The score rule is set as shown above, and an overlap score is obtained for each pixel in the reference image for each tone value as a parameter based on the score rule, and the overlap score is maximized in a predetermined tone value range. The gray value is set as a threshold value in the gray value range, and the grayscale image converted from the exposure value image used as a reference is divided into a plurality of areas according to the high and low values of the gray value. In addition, by synthesizing a high dynamic range image from a plurality of images having different exposure values, a pseudo contour is not generated at the boundary of the region, and a high dynamic range image without a sense of incongruity is obtained.
[Imaging device that generates high dynamic range images]
An imaging apparatus having a function of obtaining a high dynamic range image by image synthesis according to the present invention will be described. The configuration of this imaging apparatus is roughly the imaging unit 10 including the lens 1, the imaging device (CCD) 2, and the imaging operation control unit 5, the setting unit 20 for performing various settings, and the image processing unit for performing image composition processing. It can be divided into 30. The setting unit 20 sets the number of exposure values, the number of shots, and the like, and also sets the threshold value of the gray value necessary for dividing the gray scale image in the process of synthesizing the high dynamic range image. This setting makes it possible to select between manual and automatic settings according to the experience of the photographer and the contrast of the subject. The shooting operation control unit 5 controls the shooting operation according to the shooting conditions of the exposure price number set in the setting unit 20 and the number of shots in a series of shooting operations.

  The storage unit 31 stores image data of a plurality of images with different exposure values photographed continuously so that they can be processed using necessary image data at the stage of image processing. The gray scale processing unit 33 performs processing for converting an image having a reference exposure value into gray scale. The area division color classification processing unit 34 divides the grayscale image into a plurality of areas according to the level of the gray value according to the threshold set by the setting means, and colors the divided areas. The averaging filter processing unit 35 performs processing by the averaging filter to obtain an image in which the region is divided and the colors are mixed at the boundary portion. The image composition unit 36 obtains the color component of each pixel in the image obtained by the averaging filter processing unit, selects the gray values of the plurality of images having different exposure values according to the color component of each pixel, and composes the image.

  The appropriate threshold automatic setting unit 32 sets an appropriate threshold when the threshold setting is automatic, and an object boundary image in which an object boundary is depicted by performing edge detection processing on an image having an exposure value used as a reference at that time In addition, the grayscale image obtained by converting the image of the exposure value using each gray value as a parameter and the threshold value as a threshold is divided by the gray value to obtain an exposure boundary image in which the exposure boundary is rendered. Generate and obtain an overlap score for each pixel in the image of the exposure value used as a reference for each gray value based on a score rule defined in advance for the object boundary in the object boundary image and the exposure boundary in the exposure boundary image An appropriate threshold value is set such that the gray value having the maximum overlap score in the gray value range is selected as the threshold value in the range.

  The imaging apparatus includes an imaging unit 10 including an imaging operation control unit 5, a setting unit 20, and an image processing unit 30. The image processing unit 30 performs grayscale processing and area when synthesizing a hydrange range image. Performs division color coding processing, averaging filter processing, image composition, image processing for automatically setting an appropriate threshold, circuit configuration for performing arithmetic processing, image data and arithmetic data in the course of image processing Storage means for storing and holding is provided.

It is a figure which shows the image which image | photographed the scenery. (A) An image obtained by dividing the image of FIG. 1 into regions by a predetermined gray value threshold. (B) An image obtained by dividing the image of FIG. 1 with a threshold value of gray value different from the case of (a). It is a figure which shows an averaging filter. It is a flowchart which shows the process which produces | generates a high dynamic range image by this invention. It is the figure which showed the flow which sets an appropriate threshold value automatically centering on the change of image data. It is a graph which shows the example of distribution of the overlap score according to a shading value. It is a figure which shows the structure of the imaging device by this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image pick-up part 20 Setting part 30 Image processing part 1 Lens 2 Image pick-up element (CCD)
5 Shooting Operation Control Unit 31 Storage Unit 32 Appropriate Threshold Value Automatic Setting Unit 33 Grayscale Conversion Unit 34 Area Division Color Classification Processing Unit 35 Averaging Filter Processing Unit 36 Image Composition Unit

Claims (4)

A method for expanding the dynamic range of a photographic image by combining image data of a plurality of images with different exposure values obtained by changing the exposure value for the same subject,
Converting an image of a reference exposure value among a plurality of images having different exposure values into a grayscale image;
Dividing the converted grayscale image into a plurality of regions according to the level of the gray value according to a set gray value threshold;
Color-coding a plurality of regions of the divided image in the grayscale image;
Generating an image obtained by performing averaging filtering on the color-coded image;
Obtaining a color component of each pixel in the averaged image;
Selecting gray values of a plurality of images having different exposure values according to the color component of each pixel, and synthesizing the images;
A method for expanding the dynamic range of a photographic image characterized by comprising:   When the grayscale image is divided into a plurality of areas based on a gray value threshold, an object is depicted with an edge as an object boundary by applying an edge detection algorithm to the grayscale image obtained by converting the reference exposure value image A boundary of an area obtained by generating a boundary image and dividing the grayscale image according to the level of the gray value with the gray value as a parameter for the gray scale image obtained by converting the reference exposure value image as a parameter An exposure boundary image depicting the exposure boundary is generated, score rules are set for the object boundary in the object boundary image and the exposure boundary in the exposure boundary image, and the reference value is set for each gray value as a parameter. An overlap score is obtained for each pixel in the image based on the score rule, and the overlap score is determined within a predetermined gray value range. 2. The dynamic range of a photographic image according to claim 1, wherein the gray scale image is divided into a plurality of regions according to the level of the gray value, using a gray value that is large as a threshold value in the gray value range. How to enlarge.   An imaging apparatus having a function for generating an image with an expanded dynamic range by combining a plurality of images obtained by shifting exposure values by an auto bracket function, and including a shooting condition, a gray value threshold, Setting means, storage means for storing a plurality of photographed image data having different exposure values and image data converted therefrom, and converting a reference exposure value image into a grayscale image A gray-scale processing unit that divides the converted gray-scale image into a plurality of regions according to the level of the gray value based on the gray value threshold set by the setting unit, and color-divides each region A processing unit, an averaging filter processing unit that performs an averaging filter process on an image color-coded for each region generated by the region division color coding processing unit, and the averaging filter processing An image composition unit that obtains a color component of each pixel in the image obtained by the image unit and selects the gray values of the plurality of images having different exposure values according to the color component of each pixel and composites the images. An imaging device. Appropriate threshold automatic setting for automatically setting an appropriate threshold for dividing the grayscale image into a plurality of areas by the threshold of the gray value when the setting means is set to automatically set the threshold of the gray value An appropriate threshold value automatic setting unit that applies an edge detection algorithm to a grayscale image obtained by converting the reference exposure value image to generate an object boundary image in which the edge is depicted as an object boundary. An image generation unit and a boundary between regions obtained by dividing the grayscale image according to the level of the gray value by using the gray value as a parameter and the gray value as a threshold for the gray scale image obtained by converting the reference exposure value image An exposure boundary image generation unit that generates an exposure boundary image depicting a certain exposure boundary, and an object boundary in the object boundary image and the exposure boundary image A predetermined score rule table for the exposure boundary and an overlap score for each pixel in the image of the reference exposure value for each gray value as a parameter is obtained based on the score rule table to obtain a predetermined score. The imaging apparatus according to claim 3, further comprising threshold selection means for selecting a gray value having the maximum overlap score in the gray value range as a threshold in the gray value range.

Images