JP2008294785A - Image processor, imaging apparatus, image file, and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make an image focused on the whole of a screen to the three-dimensional one with a feeling of depth. <P>SOLUTION: When the image is photographed, the image is divided into a plurality of areas and pieces of distance information to subjects included in the respective areas are acquired. Blurring degrees are set by every area based on the pieces of distance information and blurring processing is performed by every area according to the blurring degrees. Thus, the image is made into the one with a blurring taste in which an area with low blurring degree in the image is floated. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus capable of correcting the degree of blurring of a contour of a subject in an image and an image processing method thereof, and more particularly to blurring processing of an image including a plurality of subjects having different distances from an imaging device.

  Imaging devices such as digital cameras and digital videos that are currently widely used can capture high-definition images by increasing the number of pixels in solid-state imaging devices such as CCD (Charge Coupled Device) and CMOS (Complimentary Metal Oxide Semiconductor) sensors. It is like that. In addition, an imaging apparatus that automatically corrects the gradation of a captured image to make the recorded image look good has already been proposed. The conventional general tone correction is, for example, to make clear images in the near view and the distant view in order to make the entire captured image clear.

  By the way, in a sight actually seen by a photographer, an object closer to the photographer is clearer and an object farther away is more blurred. That is, a stereoscopic image with a sense of depth is visible to the naked eye of the photographer. An imaging apparatus having a large lens aperture and imaging element, such as a single-lens reflex camera, can capture an image with a shallow depth of field, and thus can capture a blurred and deep image. However, in a small imaging device such as a compact digital camera, the aperture of the lens and the imaging element are small, so the captured image has a deep depth of field, and the focus is suitable for the entire range from near view to distant view. It is difficult to shoot a blurred image with a sense of depth.

  Therefore, Patent Document 1 proposes a method of blurring a part other than the person in the image when it is detected that the subject in the image is a person. Thereby, even an image in which the whole object is focused from the person to the background can be an image in which the person is raised.

Patent Document 2 proposes a method of changing gradation conversion using at least one of a focal length of a photographing lens or a photographing distance of a subject. When the focal length is long and the shooting distance is short as in macro shooting, the gradation change in the middle area is emphasized to suppress overexposure and blackout, and the gradation expression of the entire screen is made visually smooth. On the other hand, if the focal length is short and the shooting distance is long, such as landscape shooting, the gradation change in the intermediate area is reduced to convert the gradation, and the gradation change area concentrated in the intermediate area is expanded to produce a sharp image. It is converted.
JP 11-41512 A JP 2004-215100 A

  However, in the image correction method proposed in Patent Document 1, although the blurring degree can be set by the user, the blurring degree in one image is constant, so an object relatively close to a person is also relatively far away. The subject will also have the same blur, resulting in an unnatural image.

  In addition, when the gradation conversion method proposed in Patent Document 2 is applied to an image in which a near view and a distant view are mixed and the focus is adjusted to the whole, the above-described conversion method for macro photography or conversion for landscape photography One of the methods will be applied. In the conversion method for macro photography, the gradation expression of the entire screen is visually smooth, and in the conversion method for landscape photography, the whole screen is sharp. However, in any case, since the same conversion conditions are applied to the entire screen regardless of whether the foreground or the background is distant, the foreground is buried in the distant view on the screen, and a sense of depth that is actually visible to the photographer is obtained. I can't.

  Therefore, the present invention provides an image processing device that is a three-dimensional image with a sense of depth, in which an object in focus is in focus, and other objects are blurred, even in an image in which the entire focus is from near to far An object is to provide an image processing method. Another object of the present invention is to provide an imaging apparatus capable of recording distance information to a subject in association with the image in order to make the image having the entire focus into a three-dimensional image with a sense of depth as described above. To do.

  In order to achieve the above object, according to the present invention, in an image correction apparatus that corrects an image by changing a signal value of an input signal, an image including a subject is divided into regions each including one or more pixels, and the region Each of these is subjected to a blurring process by applying a blurring degree set according to the distance to the subject.

  The blurring process may be performed by a low-pass filter, and the blurring degree may be set by changing a cutoff frequency of the low-pass filter.

  Further, as the distance to the subject is longer, the blur applied to the area may be set higher.

  Further, the area where the distance to the subject is equal to or less than a predetermined distance may not be subjected to the blurring process.

  Further, the area is divided into a plurality of distance groups based on the distance to the subject, the blurring degree is set for each of the distance groups, and each of the areas has a blurring degree set for the distance group to which it belongs. The blur processing may be performed.

  Further, the blurring degree may be set higher as the distance of the area to the subject is larger than a predetermined distance.

  Further, the predetermined distance may be a distance to the subject having a specific portion.

  Further, each of the areas may be subjected to gradation correction by gradation correction characteristics corresponding to the distance to the subject.

  An electronic apparatus according to the present invention includes the image correction apparatus having the above-described configuration.

  In the electronic apparatus, a lens unit including a lens and a focus control unit, a solid-state image sensor that photoelectrically converts light incident from the lens unit into an electric signal, and an electric image including the subject acquired by the solid-state image sensor. A distance detection unit that detects a distance from a signal to the subject, a main body unit that includes an electrical signal of the image as obtained by the solid-state imaging device, and an image of the main body unit that is detected by the distance detection unit A storage unit that stores an image file including a header unit that includes information on a distance to the subject in each region, and the header unit receives an electrical signal of the image of the main body stored in the storage unit The blurring process may be performed by applying a blurring degree set in accordance with the distance to the subject stored in the screen.

  In addition, the solid-state imaging device obtains a distance detection image including a plurality of images including the subject and having different focus distances by changing the focus distance of the lens unit. Measure the strength of each high-frequency component of the regions of the plurality of images with different focal distances, detect the focal distance of the image with the highest strength of each high-frequency component of the region, detected by the distance detection unit, The focus distance of the image with the highest intensity of the high-frequency component may be used as the distance to the subject in each of the regions.

  The image correction method according to the present invention further includes an external image file storing an image file including a signal of the image including a subject, and distance information to the subject in each of a region composed of a single pixel or a plurality of pixels dividing the image. A first step of reading the image file from the memory, and a blurring process on the image including the subject according to a blurring degree according to the distance to the subject included in each of the areas read in the first step And a second step of applying.

  According to the present invention, since the degree of blurring processing is set for each area based on the distance to the object included in the area where the screen is divided, even in an image in which the entire focus is from the foreground to the distant view, the target object It can be a three-dimensional object with a sense of depth in which the other subjects are blurred.

<First Embodiment>
A first embodiment of the present invention will be described with reference to the drawings. In the following description, an imaging apparatus such as a digital camera or a digital video that performs the shooting method of the present invention will be described as an example. The imaging device may be capable of capturing a moving image as long as it can capture a still image.

(Configuration of imaging device)
First, the internal configuration of the imaging apparatus will be described with reference to the drawings. FIG. 1 is a block diagram illustrating an internal configuration of the imaging apparatus according to the first embodiment.

  The imaging apparatus of FIG. 1 includes a solid-state imaging device (image sensor) 1 such as a CCD or CMOS sensor that converts incident light into an electrical signal, and a zoom lens and a zoom lens that form an optical image of a subject on the image sensor 1. A lens unit 2 having a motor for changing the focal length of the zoom lens, that is, the optical zoom magnification, and a motor for focusing the zoom lens on the subject, and an image signal which is an analog signal output from the image sensor 1 is converted into a digital signal. An AFE (Analog Front End) 3, a microphone 4 that converts externally input sound into an electrical signal, and various image processing including blurring processing and gradation correction on an image signal that is a digital signal from the AFE 3 An image processing unit 5 to be applied, a distance detection unit 23 for detecting the distance from the image signal input from the image processing unit 5 to the subject, An audio processing unit 6 that converts an audio signal that is an analog signal from the audio signal 4 into a digital signal, and a JPEG (Joint Photographic Experts Group) compression method for the image signal from the image processing unit 5 when shooting a still image In the case of shooting a moving image, a compression processing unit 7 that performs compression coding processing such as MPEG (Moving Picture Experts Group) compression method on the image signal from the image processing unit 5 and the audio signal from the audio processing unit 6; A driver unit 8 that records the compression-encoded signal compression-encoded by the compression processing unit 7 in an external memory 22 such as an SD card, and the driver unit 8 decompresses and decodes the compression-encoded signal read from the external memory 22 A decompression processing unit 9, a video output circuit unit 10 that converts an image signal obtained by decoding by the decompression processing unit 9 into an analog signal, and a signal that is converted by the video output circuit unit 10. A video output terminal 11, a display unit 12 having an LCD or the like for displaying an image based on a signal from the video output circuit unit 10, and an audio output circuit unit 13 for converting the audio signal from the expansion processing unit 9 into an analog signal. And the audio output terminal 14 that outputs the signal converted by the audio output circuit unit 13, the speaker unit 15 that reproduces and outputs audio based on the audio signal from the audio output circuit unit 13, and the operation timing of each block coincides A timing generator (TG) 16 for outputting a timing control signal for causing the image pickup apparatus, a CPU (Central Processing Unit) 17 for controlling the drive operation of the entire imaging apparatus, and storing each program for each operation and executing the program Instructions from the user including a memory 18 for temporarily storing the data and a shutter button for taking a still image And a bus line 20 for exchanging data between the CPU 17 and each block, and a bus line 21 for exchanging data between the memory 18 and each block. . In the lens unit 2, the CPU 17 controls the focus and the diaphragm by driving the motor in accordance with the image signal detected by the image processing unit 5.

(Basic operation of the imaging device)
Next, the basic operation of the image pickup apparatus at the time of still image shooting will be described with reference to the flowchart of FIG. First, when the user turns on the power of the imaging apparatus (STEP 201), the imaging mode of the imaging apparatus, that is, the driving mode of the image sensor 1 is set to the preview mode (STEP 202). Subsequently, the camera enters a shooting mode input waiting state, and a blur mode and the like to be described later are selected. If the shooting mode is not input, the preview mode is selected (STEP 203). In the preview mode, an image signal that is an analog signal obtained by the photoelectric conversion operation of the image sensor 1 is converted into a digital signal by the AFE 3, subjected to image processing by the image processing unit 5, and compressed by the compression processing unit 7. An image signal for the current image is temporarily recorded in the external memory 22. The compressed signal is expanded by the expansion processing unit 9 via the driver unit 8, and an image having an angle of view at the zoom magnification of the lens unit 18 set at the present time is displayed on the display unit 12.

  Subsequently, the user sets the zoom magnification with the optical zoom so that the desired angle of view is obtained with respect to the subject to be imaged (STEP 204). At this time, the CPU 17 controls the lens unit 2 based on the image signal input to the image processing unit 5 to perform optimum exposure control (Automatic Exposure; AE) and focus control (Auto Focus; AF) (STEP 205). ).

  When the user determines the shooting angle of view and composition and presses the shutter button of the operation unit 19 halfway (STEP 206), the AE is adjusted (STEP 207) and the AF optimization process is performed (STEP 208). In the AF optimization process, for example, a captured image area is divided into a plurality of areas, and a part of the area is set as an AF evaluation area. Then, the luminance signal of the entire area in which the weighting of the AF evaluation area is set high by the distance information detection unit 23 while changing the focusing distance (focus position) of the lens unit 2 from the near point to the far point or from the far point to the near point. Are detected, and the focus position at which the high-frequency component is maximum is set as the focus position used for shooting. FIG. 3 shows an example of an image obtained by dividing the photographed image area into 64 regions of 8 columns and 8 rows and setting the 16 regions at the center as the AF evaluation area. FIG. 4 is a graph showing the relationship between the in-focus distance and the high-frequency component of the luminance signal in the entire area and other individual areas where the AF evaluation area of the image of FIG. The position indicated by the arrow in the graph of the entire area where the weight of the AF evaluation area indicated by the bold line is set high is the focus position used for shooting.

  When AE / AF for photographing is set, it is detected whether or not the blurring processing mode is selected for the drive mode of the image sensor 1 (step 209). When the blur processing mode is not selected, when the shutter button is fully pressed (STEP 220), a timing control signal is given from the TG 16 to the image sensor 1, the AFE 3, the image processing unit 5, and the compression processing unit 7, respectively. The operation timing of each unit is synchronized, the drive mode of the image sensor 1 is set to the still image shooting mode (STEP 221), and the image signal (raw data) output from the image sensor 1 is converted into a digital signal by the AFE 3. The data is converted and temporarily written in the frame memory in the image processing unit 5 (STEP 222). This digital signal is read from the frame memory, and various image processing such as signal conversion processing for generating a luminance signal and a color difference signal is performed in the image processing unit 5, and the signal subjected to the image processing is processed in the compression processing unit 7. After being compressed to JPEG (Joint Photographic Experts Group) format (STEP 223), the compressed image is written in the external memory 22 (STEP 213), and the photographing is completed. Thereafter, the process returns to the preview mode (STEP 202).

(Basic operation of the imaging device)
The operation during movie shooting will be described. In this imaging apparatus, when the operation unit 19 instructs to perform an imaging operation, an image signal that is an analog signal obtained by the photoelectric conversion operation of the image sensor 1 is output to the AFE 3. At this time, the image sensor 1 receives the timing control signal from the TG 16 to perform horizontal scanning and vertical scanning, and output an image signal serving as data for each pixel. In the AFE 3, when an image signal (raw data) that is an analog signal is converted into a digital signal and input to the image processing unit 5, various image processing such as signal conversion processing that generates a luminance signal and a color difference signal. Is given.

  Then, the image signal subjected to the image processing by the image processing unit 5 is given to the compression processing unit 7. At this time, an audio signal which is an analog signal obtained by inputting the sound into the microphone 4 is converted into a digital signal by the audio processing unit 6 and given to the compression processing unit 7. As a result, the compression processing unit 7 compresses and encodes the image signal and the audio signal, which are digital signals, based on the MPEG compression encoding method, gives the image signal and the audio signal to the driver unit 8, and records them in the external memory 22. At this time, the compressed signal recorded in the external memory 22 is read by the driver unit 8 and applied to the decompression processing unit 9, and decompressed to obtain an image signal. This image signal is given to the display unit 12 to display a subject image currently photographed through the image sensor 1.

  When the imaging operation is performed in this way, a timing control signal is given to the AFE 3, the image processing unit 5, the audio processing unit 6, the compression processing unit 7, and the expansion processing unit 9 by the TG 16, and one frame by the image sensor 1. An operation synchronized with each imaging operation is performed.

  When the operation unit 19 instructs to play back the moving image recorded in the external memory 22, the compressed signal recorded in the external memory 22 is read out by the driver unit 8 and given to the decompression processing unit 9. It is done. Then, the decompression processing unit 9 decompresses and decodes the image signal and the audio signal based on the MPEG compression encoding method. Then, the image signal is given to the display unit 12 to reproduce the image, and the audio signal is given to the speaker unit 15 via the audio output circuit unit 13 to reproduce the audio. Thereby, a moving image based on the compressed signal recorded in the external memory 22 is reproduced together with the sound.

  When it is instructed to reproduce an image, the decompression processing unit 9 decompresses and decodes the compressed signal recorded in the external memory 22 based on the JPEG compression encoding method, and acquires an image signal. Then, the image signal is given to the display unit 12 to reproduce the image.

(Blur processing mode)
Next, the blur processing mode will be described. In the imaging apparatus of the present invention, the image processing unit 5 includes the blur processing unit 30 that can calculate distance information of the subject in the image and perform blur processing based on the calculated distance information. Is called a blurring processing mode. The configuration of the blur processing unit 30 and the operation in the blur processing mode will be described below.

  FIG. 5 is a block diagram illustrating a configuration of the blur processing unit 30. The blur processing unit 30 temporarily stores the distance detection image converted to a digital signal by the AFE 3 when the shutter button is half-pressed and the recording image converted to a digital signal by the AFE 3 when the shutter button is fully pressed. A frame memory 31; a low-pass filter unit 34 that performs blurring on a recording image signal (digital signal) read from the frame memory 31 based on the distance to the subject in each area stored in the distance information storage unit 17a; A signal processing unit 35 that generates a luminance signal and a color difference signal from the recording image signal that has been subjected to the blurring process by the low-pass filter unit 34. The distance detection unit 23 divides the distance detection image (digital signal) stored in the frame memory 31 into predetermined regions, and corresponds to the horizontal position corresponding to the H position and the V position which are address information in the image transmitted from the TG 16. A distance from the imaging device to the subject in each area is detected in response to a synchronization signal (hereinafter referred to as Hsync) and a vertical synchronization signal (Vsync), and a distance information storage unit 17a provided in the CPU 17 receives the Hsync and Vsync signals. The distance to the subject in each area detected by the receiving distance detector 23 is stored. The distance information storage unit 17a is not limited to the one provided inside the CPU 17, and a storage device provided inside or outside the imaging device may be used.

  Next, the operation of the blur processing unit 30 configured as described above will be described. When it is detected in STEP 203 in FIG. 2 that the blurring processing mode is selected and the shutter button is half-pressed (STEP 204), a distance detection image is taken and stored in the frame memory 31. . The distance detection unit 23 divides the distance detection image stored in the frame memory 31 into predetermined regions and detects the distance from the imaging device to the subject in each region, and the distance information storage unit 17a detects the distance detection unit 23a. The distance to the subject in each region (hereinafter sometimes referred to as distance information) is stored (STEP 205).

  Here, an example of the configuration and operation of the distance detection unit 23 will be described with reference to the drawings. FIG. 6 is a block diagram showing the configuration of the distance information detection unit, and FIG. 7 is a flowchart for explaining the operation of detecting distance information. As shown in FIG. 6, the distance detection unit 23 reads distance detection images (digital signals) made up of a plurality of images taken with different focus distances from the frame memory 31 in the order of longer focus distances. And a high frequency component detector 23a for detecting the intensity of the high frequency component of the luminance signal in each region, and a high frequency in each region of the image signal at a certain focusing distance detected by the high frequency component detector 23a. A high-frequency component comparison unit 23b that compares the intensity of the component with the highest intensity of the high-frequency component in each region of the image signal at a longer focal distance.

  Next, the operation of the distance detection unit 23 will be described. First, the AF control in step 208 will be described. After the AE setting (step 207), as shown in FIG. 7, an in-focus distance (focus position) is set to the nearest point which is the shortest distance in the entire in-focus range of the lens unit 2 (step 701). Then, a distance detection image is taken at the focus position, converted into a digital signal by AFE 3, and written in the frame memory 31. The high frequency component detection unit 23a reads the digital signal of the closest point distance detection image stored in the frame memory 31 and divides it into predetermined regions, and detects the intensity of the high frequency component of the luminance signal in each region (STEP 702). . Then, an AF evaluation value is calculated from the intensity of the high-frequency component of the luminance signal in the entire area where the weighting of the AF evaluation area is set high (STEP 703).

  Subsequently, it is detected whether or not the focus position is at infinity. If the focus position is not at infinity (NO in STEP 704), the focus position is set far by a predetermined interval (one step) (STEP 705). When the Vsync signal is received (YES in STEP 706), an AF evaluation value at the focus position is calculated (STEP 703).

  When it is detected in STEP 704 that the focus position is infinite (YES in STEP 704), the focus position of the lens unit 12 is moved to the position where the AF evaluation value is maximized. An image is taken at this focus position.

  Furthermore, when it is detected in STEP 209 that the mode is the blurring processing mode, the high frequency component comparison unit 23b intensifies the high frequency component of each region of the image at each focus position from the nearest point to infinity stored in the frame memory 31. Are read from the high-frequency component detection unit 23a in order from the nearest point. First, the intensity of the high frequency component in each area of the image of the nearest point and the image having a long one-step focus position is compared, and the higher intensity and the focusing distance are stored for each area. For a certain region, let g1 be the intensity of the high-frequency component at infinity, and g2 be the intensity of the high-frequency component at a short focusing distance of one step, then g1 if g1 ≧ g2, and g2 if g1 <g2. The maximum intensity G in the focusing distance range is stored together with the focusing distance.

  Next, when there is an image with a short one-step focus distance, the intensity of the high-frequency component in each area of the image is read, and the focus distance range longer than that stored (the focus distance range read so far) ) And the maximum intensity H of the high-frequency component in each region. Then, the higher intensity and the focusing distance are stored for each region. For a certain region, when the intensity of the high-frequency component at a certain focusing distance is gn, if gn ≧ G, gn is newly updated as the value of G together with the focusing distance, and if gn <G, The value of G and the in-focus distance are maintained.

  The strength of the high frequency component increases as the distance from the imaging device to the subject and the in-focus distance are closer. When this comparison operation is completed for the images at all the focus positions, the high frequency component comparison unit 23b stores the highest intensity H of the high frequency component and the focal distance indicating the intensity for each region. The distance information from the imaging device to the subject in FIG. The distance information storage unit 17a reads the in-focus distance indicating the highest intensity in each region from the high frequency component comparison unit 23b (STEP 205).

  FIG. 8 is an example of a distance detection image and a recording image. FIG. 9 is an example in which the image of FIG. 8 is divided into predetermined areas, and the predetermined area is composed of a total of nine columns in three columns and three rows. Each region in FIG. 9 is R1, R2,..., R9 from the upper left to the right. Each region consists of a single pixel or a plurality of pixels. FIG. 10 is a graph showing the relationship between the intensity of the high-frequency component in each region R1, R2,..., R9 and the focusing distance, where the vertical axis represents the high-frequency component intensity and the horizontal axis represents the focusing distance. As described above, the intensity of the high-frequency component increases as the distance from the imaging device to the subject and the in-focus distance are closer.

  In the graphs of the regions R5 and R8 including the person, the focal distance at which the high frequency component has the maximum intensity is the same, but the maximum intensity is different. That is, since the subject position is the same in any region, the focus distance that is the maximum intensity is the same. However, the maximum intensity differs because the state of the subject included in each region is different, for example, the region includes the edge of the subject, or the subject covers the entire surface of the region.

  For the same reason, the graphs for the regions R1, R2, R3, R6, and R9, which are regions including buildings, have the same focal distance at which the high-frequency component has the maximum intensity, but the maximum intensity is different.

  When a plurality of subjects having different distances exist in one area, the maximum values of the intensity of the plurality of high frequency components appear. FIG. 11 is a graph showing the relationship between the intensity of the high-frequency component and the focus distance in an area where there are a plurality of subjects having different distances from the imaging apparatus. As described above, when two or more maximum values of the intensity of the high-frequency component appear in one area, the focal distance with the maximum intensity may be adopted as the distance to the subject in that area.

  Alternatively, a maximum value equal to or greater than a predetermined threshold value may be detected, and the shortest or the longest in focus distance may be employed. Alternatively, the focus distance may be used as the center when the local maximum values equal to or greater than the predetermined threshold are arranged in the order of the focus distance. Since there is no center when the maximum value equal to or greater than the predetermined threshold is an even number, one of the two threshold values close to the center may be adopted.

  After the distance to the subject in each area is stored in the distance information storage unit 17a, it is detected whether the composition and angle of view have changed (STEP 211). A case where the composition and the angle of view are detected will be described later. If the composition and the angle of view are not detected (No in STEP 211) and the shutter button is fully pressed (Yes in STEP 213), the image sensor 1, the AFE 3, and the image processing unit 5 are detected from the TG 16. A timing control signal is given to each of the compression processing unit 7, the operation timing of each unit is synchronized, and the driving mode of the image sensor 1 is set to the still image shooting mode (STEP 214, an analog signal output from the image sensor 1). The image signal (raw data) is converted into a digital signal by the AFE 3 and temporarily written in the frame memory in the image processing unit 5 (STEP 215), and the recording image (digital signal) stored in the frame memory 31 is stored in the low-pass filter unit 34. ) For each region divided by the distance detector 23 described above. A blurring degree is set based on the distance to the subject in each area stored in the information storage unit 17a (STEP 216), and a blurring process is performed on each area based on the set blurring degree (STEP 217). The compressed image is compressed in the JPEG format in the compression processing unit 7 (STEP 218), and then the compressed image is written in the external memory 22 (STEP 219) to complete the shooting, and then returns to the preview mode (STEP 202).

  When the compression processing unit 7 compresses the image signal that has been subjected to the blurring process in STEP 218, not only the image signal that has been subjected to the blurring process but also the image signal that has not been subjected to the blurring process is compressed, as shown in FIG. The file 80 to be written in the external memory 22 has a focusing distance indicating the maximum intensity in each area stored in the distance information storage unit 17a in STEP 210 in the header unit 82 added to the compressed image signal 81 which is not subjected to the blurring process. Information 82a may be written together. For example, the exif format can be used as a format for writing the focus distance information in such a header portion 82. It should be noted that the user can select to write only the image signal that has not been subjected to the compressed blurring process having the in-focus distance information in the header portion without writing the compressed blur signal to the external memory 22. It doesn't matter if you do. The information written in the header part 82 may be compressed.

(When it detects that the composition and angle of view have changed)
Here, when it is detected in STEP 211 that the composition and the angle of view have changed by a predetermined amount or more, the display unit 12 indicates that the angle of view has changed and that the shutter button is released and half-pressed again. Or by voice from the speaker unit 15 to detect whether the shutter button is half-pressed again (STEP 212). When it is half-pressed again (YES in STEP 212), the focus distance indicating the maximum intensity in each region is detected and stored again through AE adjustment (STEP 207) and AF control (STEP 208) (STEP 210). Here, a change in composition and angle of view is detected by a change in the AE evaluation value sequentially calculated by the CPU 17 or a subject detected by the CPU 17 from an image on the image sensor 1 that is sequentially captured and converted into a digital signal by the AFE 3. The motion vector can be detected by a sensor that detects a change in motion vector or a motion such as an acceleration sensor.

  If the pressing is not half-pressed in STEP 212 but is fully pressed in STEP 213, the processing after STEP 207 may be continued with the blurring process omitted. In addition, if the amount of change in composition and angle of view is detected between the half-press of the shutter button and the full-press of the shutter button by a change in motion vector or acceleration sensor, the change before the change based on the detected amount of change is detected. The distance information of the image is shifted so as to match the image after the change, and the blurring degree is set in each area of the image after the change based on the distance to the subject in each area (STEP 216). You may implement (STEP217). In this case, the blurring process may not be performed for an area without distance information, or the blurring process may be performed under the same conditions as the surrounding area.

(Blur processing conditions)
Each area of the image is subjected to a blurring process according to the distance to the subject in each area. The conditions for the blurring process will be described. The low-pass filter unit 34 that performs the blurring process adjusts the degree of blurring by setting a cutoff frequency. The higher the cutoff frequency, the lower the degree of blurring, and the lower the cutoff frequency, the higher the degree of blurring. When the cutoff frequency is the Nyquist frequency, the blurring process is not performed.

  In the present embodiment, the blurring process is not performed in the short distance range where the distance to the subject is from the vicinity of the imaging apparatus to the predetermined distance D1, and the blur is increased as the distance is longer in the middle distance range from the distance D1 to the distance D2 (D2> D1). Increase the degree of. In the long distance range that is a range farther than the distance D2, the highest blurring degree in the middle distance range is applied as a constant blurring degree. FIG. 13 is a graph showing the relationship between the cutoff frequency and the distance to the subject. In the short distance range, the cutoff frequency is constant at the Nyquist frequency N. In the middle distance range, the longer the distance, the lower the cutoff frequency. In the long distance range, the cutoff frequency is constant at the lowest cutoff frequency. By setting the cut-off frequency in this way, the image obtained by the blurring process has a stereoscopic effect in which the subject in the short distance range is raised from the subject in the middle distance range and the long distance range.

(Interpolation of cut-off frequency)
However, if the cut-off frequency used for the blurring process differs between adjacent areas, the boundary appears at the boundary between the areas depending on the degree of blurring. Therefore, gradation is given to the cutoff frequency used for the blurring processing of the pixels constituting the adjacent regions having different cutoff frequencies so as to make the boundary between the regions inconspicuous.

  FIG. 14 shows a schematic diagram of four regions. The centers of the areas 71 to 74 are respectively the points 71a to 74a, and the four areas of the areas 71 to 74 share the origin O, the horizontal direction of the drawing is the x axis, and the vertical direction is the y axis. Let 2a be. At this time, the coordinates of the center 71a are (a, a), the coordinates of the center 72a are (-a, a), the coordinates of the center 73a are (-a, -a), and the coordinates of the center 74a are (a, -a). is there.

For the pixel located at the center of each region, the cutoff frequency, which is the blurring degree set for each region, is used for the blurring process. For the pixels surrounded by the centers of the four regions sharing the origin O, each region is set. A value obtained by linearly interpolating the cut-off frequency with the distance from the pixel to the center of each region is used for the blurring process. Here, a pixel 71b that belongs to the region 71 shown in FIG. 14 and is surrounded by the centers 71a to 74a is considered. If the cut-off frequencies set in the regions 71 to 74 are h1, h2, h3, and h4, the cut-off frequency in the pixel 71b is Z, and the coordinates of the pixel 71b are (x, y),
Z = [{(a + x) h1 + (ax) h2} / 2a] (a + y) / 2a
+ [{(A + x) h3 + (ax) h4} / 2a] (ay) / 2a
It becomes. For pixels on a line connecting the centers of two adjacent areas, linear interpolation of the cutoff frequencies set in these two areas is used for the blurring process.

  FIG. 15 is a graph of the cut-off frequency that is changed by giving a gradation between different cut-off frequencies by linear interpolation. The five cut-off frequencies h1, h2, h5 to h7 are shown on the graph of FIG. h5 to h7 are linearly interpolated between h1 and h2. h1 is the cut-off frequency of the region 71 in FIG. 14, and h2 is the cut-off frequency of the region 72. H1 is applied to the center 71a, and h2 is applied to the center 72a. Each of h5 to h7 is a pixel 71c, a pixel 71d, and a pixel 72b (on the side of the center 71a), which are three pixels arranged on the line connecting the center 71a and the center 72a at substantially equal intervals including the center 71a and the center 72a. Applied in order). By interpolating the cut-off frequency in this way and changing the gradation for each pixel, the image after the blurring process becomes inconspicuous.

  When the regions 71 to 73 are regions located on the outer periphery of the image, the point 71e of the coordinates (a, 2a), the point 72c of the coordinates (-a, 2a), and the coordinates (-2a, 2a) in FIG. When the point 72d of the coordinate, the point 72e of the coordinate (-2a, a), and the point 73b of the coordinate (-2a, -a) are set, the point 72a, the point 71a, and the point 71a are surrounded by the point 71c, 72e, 72a, 71a. The cut-off frequency of the point where the x-coordinate on the line segment connecting the two points is the same, and the cut-off of the point where the y-coordinate on the line segment connecting the point 72a and the point 73a is the same in the portion surrounded by the points 72a, 72e, 73b In the portion surrounded by the frequency and points 72c, 72d, 72e, 72a, the cut-off frequency of the point B, that is, the center 72a is applied.

<Second Embodiment>
A second embodiment of the present invention will be described with reference to the drawings. This embodiment is the same as the first embodiment except that the configuration of the blur processing unit is different, and substantially the same parts are denoted by the same reference numerals.

  In the present embodiment, each region is divided into a plurality of groups according to the distance to the subject, and blurring processing is performed. In this case, a predetermined cutoff frequency is set and applied to each group.

  Here, as an example, a case will be described in which the distance group is divided into three types: foreground, middle, and distant views. FIG. 16 is an example in the graph of FIG. 11 of the first embodiment where the distance to the subject is less than D1, the background is D1 and less than D2, and the background is D2 and more. Here, the relationship between D1 and D2 is D1 <D2. Note that the number of distance groups is not limited to three and may be two or more.

  FIG. 17 is a block diagram of a blur processing unit according to the second embodiment. The low-pass filter unit 34 of the blur processing unit 30 of the present embodiment includes a foreground low-pass filter 34a, a middle-ground low-pass filter 34b, a far-view low-pass filter 34c, a multiplication coefficient setting unit 37, and an adder 38. The foreground low pass filter 34a, the middle view low pass filter 34b, and the far view low pass filter 34c each have a multiplier 36 connected in series in the subsequent stage. The foreground low-pass filter 34a and the multiplier 36, the middle-ground low-pass filter 34b and the multiplier 36, and the far-view low-pass filter 34c and the multiplier 36 are connected in parallel between the frame memory 31 and the signal processing unit 35, respectively. Yes. The multiplication coefficient setting unit 37 sets a multiplication coefficient used for multiplication in each multiplier 36 based on the distance to the subject in each area stored in the distance information storage unit 17a. Connected to the device 36.

  The cutoff frequency and output level of the low-pass filter 34a for near view, the low-pass filter 34b for middle scene, and the low-pass filter 34c for distant view have a relationship as shown in the graph of FIG. FIG. 18 is a graph with the horizontal axis representing the cutoff frequency and the vertical axis representing the output level, and also shows the level of the input image signal. As for the relationship between the cut-off frequency and the output level, the output level is constant at 1.0 in the low frequency range, the output level becomes lower as the frequency is higher than the predetermined frequency, and finally becomes 0. . The frequency at which the output level begins to decrease and the frequency at which the output level becomes 0 are highest in the foreground low-pass filter 34a, and lower in the order of the low-pass filter 34b for the foreground and low-pass filter 34c for the background. In this embodiment, the relationship between the cutoff frequency of the foreground low-pass filter 34a and the output level is the same as that of the input image, and therefore the foreground low-pass filter 34a may not be provided.

  The multiplication coefficient setting unit 37 sets a multiplication coefficient for each multiplier 36 in each area of the image based on the distance information of the area. FIG. 19 is a graph showing an example of the relationship between the distance to the subject and the multiplication coefficient. In FIG. 19, the foreground multiplication coefficient KN is indicated by a solid line, the foreground multiplication coefficient KM is indicated by a one-dot chain line, and the far-distance multiplication coefficient KF is indicated by a two-dot chain line. The foreground multiplication coefficient KN is constant at 1.0 in a range less than D1, and is constant at 0 in a range greater than D1. The middle scene multiplication coefficient KM is constant at 0 in a range less than D1, is constant at 1.0 in a range from D1 to less than D2, and is constant at 0 in a range from D2 or more. The far-field multiplication coefficient KF is constant at 0 in the range below D2, and constant at 1.0 in the range above D2. Thus, the sum of the multiplication coefficients KF, KM, and KF is constant at 1.0 regardless of the distance to the subject.

  Next, the blurring process of this embodiment will be described. The recording image stored in the frame memory 31 is read, and the blurring process is performed by each of the foreground low-pass filter 34a, the foreground low-pass filter 34b, and the distant view low-pass filter 34c. On the other hand, the multiplication coefficient setting unit 37 sets a multiplication coefficient for each region according to the subject distance of each region stored in the distance information storage unit 17a.

  The images subjected to the blurring process by the low-pass filters 34 a, 34 b, and 34 c are multiplied for each region by the multiplier 36 with the multiplication coefficient set by the multiplication coefficient setting unit 37 and added by the adder 38. In the image added by the adder 38, in the region where the distance to the subject is less than D1, the multiplication coefficient for the foreground low-pass filter 34a is 1.0, and the foreground low-pass filter 34b, the far-view low-pass filter Since the multiplication coefficient for 34c is 0, the blurring process is performed only by the foreground low-pass filter 34a. When the foreground low-pass filter 34a is not provided, the original image remains as it is. Similarly, the region where the distance to the subject is greater than or equal to D1 and less than D2 is blurred only by the low-pass filter 34b for the middle scene, and the region where the distance to the subject is greater than D2 is blurred by the low-pass filter 34c for far view. FIG. 20 is a schematic diagram of an image obtained by blurring an image divided into nine regions R1 to R9. The blur conditions differ between the regions R5 and R8 belonging to the foreground, the regions R4 and R7 belonging to the middle background, and the regions R1, R2, R3, R6, and R9 belonging to the far background, the far view is greatly blurred, and the middle background is slightly blurred. The close-up view is the image that remains. As a result, the image obtained by the blurring process has a stereoscopic effect in which the near view is raised from the middle view and the far view.

  By configuring in this way, distance information about each area stored in the distance information storage unit 17a can be a distance group to which the area belongs, which is rough information, instead of detailed distance information. Speed can be improved.

<Third Embodiment>
A third embodiment of the present invention will be described with reference to the drawings. The present embodiment is the same as the second embodiment except that the configuration of the blur processing unit and the relationship between the multiplication coefficient and the subject distance are different, and substantially the same parts are denoted by the same reference numerals.

  FIG. 21 is a block diagram of the blur processing unit according to the present embodiment. The blurring processing unit 30 of the present embodiment is obtained by providing an edge signal generation unit 39 in the blurring processing unit of the second embodiment of FIG. The edge signal generator 39 has a multiplier 36 connected in series at the subsequent stage. The edge signal generation unit 39 and the multiplier 36 are connected in parallel between the frame memory 31 and the signal processing unit 35, and a multiplication coefficient setting unit 37 is connected to the multiplier 36.

  FIG. 22 is a graph showing an example of the relationship between the distance to the subject and the multiplication coefficient in the present embodiment. FIG. 22 shows a multiplication coefficient KE for edge signals in addition to a multiplication coefficient KN for near view, a multiplication coefficient KM for middle scene, and a multiplication coefficient KF for distant view. In this embodiment, the foreground multiplication coefficient KN is constant at 1.0 in the range of less than D1−d1, and decreases in proportion to the distance from 1.0 to 0 in the range of D1−d1 and less than D1 + d1. , It is constant at 0 in the range of D1 + d1 or more. The multiplication factor KM for the middle scene is constant at 0 in the range of less than D1−d1, increases in proportion to the distance from 0 to 1.0 in the range of D1−d1 and less than D1 + d1, and is equal to or greater than D1 + d1 and D2−d2. The range is less than 1.0 in the range less than D2-d2 and less than D2 + d2, and decreases in proportion to the distance from 1.0 to 0, and is constant in 0 in the range greater than D2 + d2. The far-field multiplication coefficient KF is constant at 0 in the range of less than D2-d2, increases in proportion to the distance from 0 to 1.0 in the range of D2-d2 or more and less than D2 + d2, and is 1 in the range of D2 + d2 or more. .0 and constant. The relationship between d1 and d2 is D1 + d1 <D2-d2. The sum of the multiplication coefficients KF, KM, and KF is constant at 1.0 regardless of the distance to the subject. The multiplication factor KE for the edge signal is constant at 1.3 in the range less than D0, decreases in the range from 1.3 to 0 in the range from D0 to less than D1-d1, and decreases in proportion to the distance D1-d1. In the above range, it is 0 and constant. The multiplication coefficient KE is a numerical value independent of other multiplication coefficients.

The relationship between the multiplication coefficients KN, KM, KF, and KE and the distance D to the subject shown in FIG. Here, the slope of the multiplication coefficient KM when D1−d1 ≦ D <D1 + d1 is k1 (k1> 0, k1 = 1 / (2 × d1)), and the slope of the multiplication coefficient KF is D2−d2 ≦ D <D2 + d2. The slope of the multiplication coefficient KE when k2 (k2> 0, k2 = 1 / (2 × d2)) and D0 ≦ D <D1−d1 is −k0 (k0> 0, k2 = 1.3 / (D1−d1−). D0)).
KN = 1.0 (D <D1-d1)
= −k1 × {D− (D1 + d1)} (D1−d1 ≦ D <D1 + d1)
= 0 (D1 + d1 ≦ D)
KM = 0 (D1 + d1 ≦ D)
= K1 * {D- (D1-d1)} (D1-d1 ≦ D <D1 + d1)
= 1.0 (D1 + d1 ≦ D <D2-d2)
= −k2 × {D− (D2 + d2)} (D2−d2 ≦ D <D2 + d2)
= 0 (D2 + d2 ≦ D)
KF = 0 (D <D1-d1)
= K2 × {D− (D2−d2)} (D2−d2 ≦ D <D2 + d2)
= 1.0 (D2 + d2 ≦ D)
KE = 1.3 (D <D0)
= −k0 × {D− (D1−d1)} (D0 ≦ D <D1−d1)
= 0 (D1-d1 ≦ D)

  Next, the blurring process of this embodiment will be described. The recording image stored in the frame memory 31 is read and subjected to blurring processing by each of the foreground low-pass filter 34a, the middle-ground low-pass filter 34b, and the far-view low-pass filter 34c, and the edge signal generation unit 39 generates an edge signal. . On the other hand, the multiplication coefficient setting unit 37 sets a multiplication coefficient for each region according to the subject distance of each region stored in the distance information storage unit 17a.

  The image subjected to the blurring process by the low-pass filters 34a, 34b, and 34c and the edge signal generated by the edge signal generation unit 39 are multiplied for each region by the multiplier 36 with the multiplication coefficient set by the multiplication coefficient setting unit 37. And is added by the adder 38.

  In the image added by the adder 38, in the region where the distance to the subject is less than D0, the multiplication coefficient for the foreground low-pass filter is 1.0, and the multiplication coefficient for the edge signal generation unit 39 is 1. 3 is constant, and the multiplication coefficient for the low-pass filter 34b for the foreground and the low-pass filter 34c for the foreground is 0. Therefore, the blurring process is performed only by the low-pass filter 34a for the foreground and the edge is emphasized by the edge signal. It will be. In the region where the distance to the subject is greater than or equal to D0 and less than D1-d1, the multiplication coefficient for the foreground low-pass filter 34a is constant at 1.0, and the foreground low-pass filter 34b and the far-view low-pass filter 34c are constant. Since the multiplication coefficient is 0 and the multiplication coefficient for the edge signal generation unit 39 decreases in proportion to the distance from 1.3 to 0, the distance increases compared to the area where the distance to the subject is less than D0. As a result, edge enhancement is suppressed. When the foreground low-pass filter 34a is not provided, the edge of the original image is emphasized.

  In the region where the distance to the subject is equal to or greater than D1-d1, the edge is not emphasized because the multiplication coefficient for the edge signal generation unit 39 is 0. Of the regions where the distance to the subject is D1−d1 or more, the region where the distance to the subject is D1 + d1 or more and less than D2−d2 is only the low-pass filter 34b for the middle scene, and the region where the distance to the subject is D2 + d2 or more is In 34c, the blurring process is performed.

  In the region where the distance to the subject is not less than D1−d1 and less than D1 + d1, the multiplication coefficient for the low-pass filter for distant view 34c is 0, but the multiplication coefficient for the low-pass filter for near view 34a is 1.0 to 0. Since it decreases in proportion to the distance and the multiplication coefficient for the low-pass filter 34b for the foreground increases in proportion to the distance from 0 to 1.0, the image processed by the low-pass filter 34a for the foreground and the foreground The image processed by the low-pass filter 34b is mixed. Note that in the range of D1−d1 or more and less than D1 + d1, the influence of the processing in the low-pass filter 34b for the foreground is greater than the low-pass filter 34a for the foreground in proportion to the distance. Similarly, the region where the distance to the subject is greater than or equal to D2−d2 and less than D2 + d2 is a mixture of the image processed by the foreground low-pass filter 34a and the image processed by the foreground low-pass filter 34b. As a result, when the distance to the subject is close to the region near D1 or D2, the degree of blurring processing is abruptly changed to suppress an unnatural blurring image. Can do.

  In this embodiment, each area is divided into a plurality of distance groups according to the distance to the subject, and the average of the distance to the subject in the area belonging to each distance group is calculated, and the average is calculated to the subject in each area. It may be used as the distance. For example, in the graph showing the relationship between the distance and the intensity of the high frequency component shown in FIG. 16, the maximum value of the high frequency components of R5 and R8 is set for the distance indicating the maximum value of the high frequency component, that is, the region where the distance to the subject is less than D1. D11 which is the average of the distances shown, D12 which is the average of the distances indicating the maximum values of R4 and R7 for the area where the distance to the subject is greater than or equal to D1 and less than D2, and for the area where the distance to the subject is D2 or more D13, which is the average of distances indicating the maximum values of R1, R2, R3, R6, and R9, can be used as the distance to the subject in the area belonging to the corresponding distance group. In this case, the management of the distance information for each region can be simplified, so that the blurring process can be speeded up. In this case, by applying the cut-off frequency interpolation method described in the first embodiment, it is possible to make a natural blur condition by making the boundary between regions with different blur conditions inconspicuous. The distance information calculated in this way can also be applied to the first embodiment.

  In the present embodiment, the edge signal generation unit 39 may not be provided if the edge of the subject is not emphasized.

  Further, in the low-pass filter unit 34 of the present embodiment, the low-pass filter 34a for foreground is removed while leaving the corresponding multiplier 36, and the multiplication coefficient is appropriately set, so that the low-pass filter unit 34 of the first embodiment is set. It can be set as the same structure.

<Modification 1>
In the first to third embodiments of the present invention, when acquiring distance information of each area in STEP 205, a face of a person having a predetermined size or more is detected in the image, and the distance of the area including the face is detected. It is good also as what does not perform a blurring process to the area | region of the predetermined distance range containing. Thereby, it is possible to reduce the possibility of performing blurring processing on a region including a person.

  In addition, blurring processing is not performed on a region in a predetermined distance range including the distance to the subject in the region including the face, and blurring processing is performed on a region that is shorter and longer than the predetermined distance range. Good. In this case, the degree of blurring may be increased as the distance from the predetermined distance range increases. Thereby, the person in an image can be raised more naturally.

  In addition, when the image area is divided into distance groups, the area belonging to the same distance group as the area including the face may not be subjected to the blurring process, and the other areas may be subjected to the blurring process.

  Here, the face detection process will be described. The image processing unit 5 includes a face detection device 50 and can detect a person's face from the input image signal. The configuration and operation of the face detection device 50 will be described below.

  FIG. 23 shows the configuration of the face detection device 50. The face detection device 50 stores the reduced image generation means 51 that generates one or a plurality of reduced images based on the image data obtained by the AFE 3, each hierarchical image composed of the input image and the reduced image, and the memory 18. A face determination unit 52 that determines whether or not a face exists in the input image using a face detection weight table, and a detection result output unit 53 that outputs a detection result of the face determination unit 52 are provided. When a face is detected, the detection result output means 53 outputs the size and position of the detected face with reference to the input image.

  The weight table stored in the memory 18 is obtained from a large number of teacher samples (face and non-face sample images). Such a weight table can be created using, for example, a known learning method called Adaboost (Yoav Freund, Robert E. Schapire, “A decision-theoretic generalization of on-line learning and an application to boosting” ", European Conference on Computational Learning Theory, September 20, 1995.).

Adaboost is an adaptive boosting learning method. Based on a large number of teacher samples, Adaboost selects multiple weak classifiers that are effective for identification from among a plurality of weak classifier candidates. This is a learning method for realizing a highly accurate classifier by weighting and integrating. Here, a weak classifier refers to a classifier that has a higher discrimination ability than a coincidence but is not high enough to satisfy sufficient accuracy. When a weak classifier is selected, if there is a weak classifier that has already been selected, the learning is focused on the teacher sample that is misrecognized by the selected weak classifier. To select the most effective weak classifier.

  FIG. 24 shows an example of a hierarchical image obtained by the reduced image generating means 51. In this example, a plurality of hierarchical images that are generated when the reduction ratio R is set to 0.8 are shown. 24, reference numeral 60 denotes an input image, and reference numerals 61 to 65 denote reduced images. Reference numeral 68 denotes a determination area. In this example, the determination area is set to a size of 24 pixels vertically and 24 pixels horizontally. The size of the determination area is the same for the input image and each reduced image. In this example, as indicated by an arrow, a face image matching the determination area is detected by moving the determination area from left to right on the hierarchical image and performing horizontal scanning from the top to the bottom. I do. However, the scanning order is not limited to this. The reason why the plurality of reduced images 61 to 65 are generated in addition to the input image 60 is to detect faces of different sizes using one kind of weight table.

  FIG. 25 is a diagram for explaining the face detection process. The face detection process by the face determination unit 52 is performed for each hierarchical image, but since the processing method is the same, only the face detection process performed on the input image 60 will be described here.

  The face detection process performed for each hierarchical image is performed using an image corresponding to the determination region set in the image and a weight table. The face detection process includes a plurality of determination steps that sequentially shift from a rough determination to a fine determination. When a face is not detected in a certain determination step, the process does not proceed to the next determination step, and the determination area includes It is determined that no face exists. In all the determination steps, only when a face is detected, it is determined that a face exists in the determination area, and the determination area is scanned to shift to determination in the next determination area. In this way, the detected face position and size are output by the detection result output means 53. Such face detection processing is described in detail in Japanese Patent Application No. 2006-053304, which is a patent application filed by the present applicant.

  In the first to third embodiments of the present invention, the area obtained by dividing an image becomes smaller, and as the area is divided more finely, the ratio of areas including a plurality of subjects with different distances decreases. Therefore, it is possible to generate an image with good blurring.

<Modification 2> (Distance-dependent gradation correction)
In the first to third embodiments of the present invention, in addition to the blurring process, gradation correction depending on the distance to the subject may be performed on the image for each region. Hereinafter, such a gradation correction method will be described.

  In this case, the blur processing unit 30 has a function capable of calculating distance information and luminance information of a subject in the image and applying gradation correction based on the calculated distance information to the captured image. This is called a correction mode.

  FIG. 26 is a block diagram showing a configuration of the blur processing unit 30 having a distance dependent gradation correction function. In addition to the frame memory 31, the low-pass filter unit 34, and the signal processing unit 35, the blurring processing unit 30 reads the digital signal of the recording image stored in the frame memory 31 and divides it into the predetermined areas described above. A luminance information calculation unit 41 that calculates average luminance, a luminance information storage unit 42 that stores the average luminance of each region calculated by the luminance information calculation unit 41, and the subject of each region stored in the distance information storage unit 17a Of the recording image signal (digital signal) read from the frame memory 31 on the basis of the distance of the image and the average luminance of each area stored in the luminance information storage unit 42 and the gradation correction table stored in the memory 14. A gradation correction unit 43 that performs correction, and the signal processing unit 35 performs recording on which image processing has been performed on at least one of the low-pass filter unit 34 and the gradation correction unit 43. From the image signal to generate a luminance signal and color difference signals.

  Next, the operation of the blur processing unit 30 having a distance dependent gradation correction function will be described. FIG. 27 is obtained by adding a part related to the distance dependent gradation correction function to a part of the flowchart of FIG. In this case, the gradation correction mode is included in addition to the blurring processing mode as an option of the shooting mode in STEP 202 in the flowchart of FIG. When the gradation correction mode is selected, the blurring processing mode is also selected at the same time, and blurring processing similar to that in the first to third embodiments is performed up to STEP 210 in FIG. Next, it is detected whether the gradation correction mode is selected (STEP 2701). If not selected, a compressed image of the blurred image is generated (STEP 211). Processing similar to that in the embodiment is continued.

  When the gradation correction mode is selected, the luminance information calculation unit 41 reads the recording image (digital signal) stored in the frame memory 31 and is the same predetermined area as that divided by the distance detection unit 23 described above. The brightness information storage unit 42 stores the average brightness calculated by the brightness information calculation unit 41 (STEP 2702).

  The gradation correction unit 43 reads the average luminance of each area from the luminance information storage unit 42 and groups each area according to the average luminance. For example, they are grouped into three groups: high luminance with average luminance of La or higher, medium luminance with average luminance of Lb or higher and lower than La, and low luminance with average luminance of lower than Lb. Here, La and Lb are arbitrary values that can be set according to the imaging apparatus. Further, a gradation correction table set for each group of in-focus distance and average luminance stored in the memory 14 is read, and gradation correction is performed based on the in-focus distance group and average luminance group to which each region belongs. The input / output characteristics are assigned (STEP 2703).

  Subsequently, the recording image stored in the frame memory 31 is read, and gradation correction is performed by applying the gradation correction table to each region (STEP 2704). The image subjected to gradation correction is compressed in the JPEG format by the compression processing unit 6 (STEP 211).

  When the image signal that has been subjected to image correction including blurring processing and gradation correction in STEP 211 is compressed by the compression processing unit 6, not only the image signal that has undergone image correction but also the image signal that has not undergone image correction is also compressed. In addition to the in-focus distance information 82a, average luminance information for each area stored in the luminance information storage unit 42 in STEP 2702 may be written together in the header unit 82 shown in FIG.

(Tone correction table)
Next, the gradation correction table will be described. The gradation correction table of this embodiment is assigned to the three distance groups of the near view, the middle view, and the distant view and the three average brightness groups of the high luminance, the medium luminance, and the low luminance. It consists of nine input / output characteristics shown in the graph of FIG. In each graph, the horizontal axis represents the input luminance value before gradation correction, and the vertical axis represents the output luminance value after gradation correction. Based on this input / output characteristic, the luminance of each pixel constituting each region is converted.

  First, input / output characteristics assigned to an area belonging to the middle scene group will be described. FIG. 28D to FIG. 28F are input / output characteristics assigned to the area belonging to the middle scene group, and the slope k of the graph is set to 1 regardless of the luminance of the pixels in the area, for example. Constant, no particular conversion. As a result, an image having a brightness intermediate between the near view and the distant view after the correction is obtained.

  Next, input / output characteristics assigned to areas belonging to the foreground group will be described. The region belonging to the near view is corrected so as to be emphasized more than the middle view and the distant view in the image.

  FIG. 28A shows input / output characteristics assigned to a region belonging to a near-ground and high-luminance group. This is to convert the overall luminance to a low level. If the slope of the graph with the luminance of L11 or higher is k11, and the slope of the portion of L11 or lower is k13, then k13 <k <k11 (k is the group of the middle scene) The slope of the input / output characteristics assigned to the area belonging to the above) is converted so as to widen the gradation width in a portion with high luminance. Here, the relationship between the luminance L11 and La used as a reference when grouping each area of the screen with the average luminance is set to L11 <La so that the gradation width is widened over the entire high luminance range to which the target area belongs. To do.

  FIG. 28B shows the input / output characteristics assigned to the area belonging to the near-ground and medium-luminance group. Assuming that the slope of the graph in the portion where the luminance is L21 or more is k21, the slope of the portion of L22 or more and L21 or less is k22, and the slope of the portion of L22 or less is k23, k21 (k23) <k <k22. The brightness is converted to be higher in the high brightness portion, and the brightness is converted to be lower in the low brightness portion to clarify the contrast. Either of the slopes k21 and k23 may be larger or the same. Here, the relationship between the luminances L21 and L22 and La and Lb is set to L22 <Lb <La <L21 so as to increase the gradation width of the entire middle luminance range to which the target region belongs.

  FIG. 28C shows the input / output characteristics assigned to the area belonging to the foreground and low luminance group. This enhances the overall luminance, and if the inclination of the portion where the luminance is L32 or higher is k31 and the inclination of the portion of L32 or lower is k33, the gradation width of the low luminance portion as k31 <k <k33. To widen. Here, the relationship between the luminances L32 and Lb is set to Lb <L32, and conversion is performed so as to widen the gradation width of the entire low luminance range to which the target region belongs.

  Next, input / output characteristics assigned to areas belonging to a distant view group will be described. The area belonging to the distant view is corrected so that it is less conspicuous than the foreground and the foreground in the image.

  FIG. 28G shows input / output characteristics assigned to a region belonging to a distant view and a group with high luminance. In this case, assuming that the slope of the portion where the luminance is L71 or higher is k71 and the slope of the portion of L71 or lower is k73, k71 <k <k73 is converted so that the gradation width of the high luminance portion is narrowed to lower the contrast. . Further, the high luminance portion of the output luminance value is cut and darkened as a whole. Here, the relationship between the luminances L71 and La is converted so that the gradation width is narrowed over the entire high luminance range to which the target region belongs, with L71 <La.

  FIG. 28H shows input / output characteristics assigned to a region belonging to a group of a distant view and a medium luminance. This is because the luminance at the portion where the luminance is L81 or more is k81, the inclination at the portion from L82 to L81 is k82, the inclination at the portion below L82 is k83, and the luminance is k82 <k <k81 (k83). Conversion is performed such that the luminance is lowered in the high luminance portion, and the luminance is increased in the low luminance portion, and the gradation width is reduced in the middle luminance portion so that the luminance is averaged and the contrast is lowered as a whole. Either of the slopes k81 and k83 may be larger or the same. Here, the relationship between the luminances L81 and L82 and La and Lb is set to L82 <Lb <La <L81 so that the gradation width of the entire middle luminance range to which the target region belongs is reduced.

  FIG. 28I shows input / output characteristics assigned to a region belonging to a group of a distant view and a low luminance. This is because if the low luminance portion of the output luminance value is cut (offset) to make it brighter as a whole, the inclination of the graph of the portion where the luminance is L92 or higher is k91, and the inclination of the portion of L91 or lower is k93. As <k <k91, conversion is performed so that the gradation width of the high luminance part is widened, and conversion is performed so that the gradation width of the low luminance part is narrowed, thereby lowering the contrast. Here, the relationship between the luminances L92 and Lb is set to Lb <L92, and conversion is performed so as to widen the gradation width of the entire low luminance range to which the target region belongs.

  By performing distance-dependent tone correction using the tone correction table as described above, the near view has a high contrast and the contrast becomes low as the distance increases, and a stereoscopic image with a sense of depth can be obtained. In the image pickup apparatus of the present invention, the order of the blurring process and the distance-dependent gradation correction may be reversed. Further, the image pickup apparatus of the present invention may have only one of the blur processing function and the distance dependent gradation correction function.

(Input / output characteristic interpolation 1)
However, if the input / output characteristics used for tone correction differ between areas, the boundary appears in the image due to the difference in the characteristics. Therefore, gradation is given to the input / output characteristics used for the gradation correction of the pixels constituting the adjacent areas having different input / output characteristics so that the boundaries between the areas are blurred and inconspicuous. In this case as well, the same method as the linear interpolation of the cutoff frequency described with reference to FIG. 14 can be used.

  FIG. 29 is a graph of input / output characteristics that are changed by providing gradation between different input / output characteristics by linear interpolation. FIG. 29 shows five graphs p1 to p5 of input / output characteristics. p2 to p3 are linearly interpolated between p1 and p5 at equal intervals. p1 is the input / output characteristic of the region 71 of FIG. 14, and is FIG. 28 (c). p5 is the input / output characteristic of the region 72 and is shown in FIG. P1 is applied to the center 71a, and p2 is applied to the center 72a. Each of p2 to p4 is a pixel 71c, a pixel 71d, and a pixel 72b (on the side of the center 71a), which are three pixels arranged at substantially equal intervals including the center 71a and the center 72a on a line connecting the center 71a and the center 72a. Applied in order). By interpolating the input / output characteristics in this way and changing the gradation for each pixel, the image after the distance-dependent gradation correction is blurred and the boundary between the areas becomes inconspicuous.

(Input / output characteristic interpolation 2)
Note that the input / output characteristics applied to the gradation correction may be changed depending on the distance to the subject in each area without grouping the areas according to the distance range to which each area belongs. Input / output characteristics are set for a plurality of specific distance areas, and for the distance areas between specific distances, the input / output characteristics set in the specific distance area adjacent to that distance as input / output characteristics Apply the one interpolated from. The input / output characteristics are set for the distance s and the distance t. When the input / output characteristics set for the distance s is S and the input / output characteristics set for the distance t is T, the distance between the distance s and the distance t is set. The input / output characteristics U for the distance u for which the input / output characteristics are not set is obtained by linearly interpolating the input / output characteristics T and S with respect to the distance u.
U = {S (tu) + T (us)} / (ts) (1)
It can be. The input / output characteristic set to the shortest distance is applied to the distance shorter than the shortest distance for which the input / output characteristic is set, and the longest distance is applied to the distance longer than the longest distance for which the input / output characteristic is set. Apply the set input / output characteristics.

  As an example, the distance to the subject is 2 m in FIGS. 28A to 28C of the gradation correction table in FIG. 28, the distance is 6 m in FIGS. 28D to 28F, and FIG. 28 (i) are input / output characteristics applied to a region having a distance of 10 m. At this time, an input / output characteristic obtained by linearly interpolating FIGS. 28B and 28E is applied to a region where the distance to the subject is 5 m and the average luminance is medium luminance. When the above equation (1) is applied with the input / output characteristic of FIG. 28B as S and the input / output characteristic of FIG. 28E as T, the input / output characteristic U applied to this region is s = 2, t = 6, u = 5, U = (S + 3T) / 4. FIG. 30 shows a graph of the input / output characteristic U at this time. In FIG. 30, q1 is a graph of FIG. 28B, q2 is a graph of FIG. 28E, and q3 is a graph of the input / output characteristics U.

  In this way, it is not necessary to group each area according to the distance range to which it belongs. Also in this case, as described in (Input / output characteristic interpolation 1), the input / output characteristics may be linearly interpolated for the pixels surrounded by the four centers of the region sharing a certain point.

  The gradation correction table is not limited to the nine input / output characteristics for the three-level luminance range and the three-level distance range, and the distance range may be two or more levels, and further, the luminance range and the distance range. At least one of the above may be two or more stages. Further, the luminance range and the distance range may not be the same number, and the luminance range may be singular. Further, the input / output characteristics are not formed by a straight line as shown in FIG. 28 but may be formed by a curve.

  FIG. 31 shows an example of a gradation correction table composed of four input / output characteristics for a two-step luminance range of high luminance and low luminance and a two-step distance range of foreground and distant views. Also in this case, the region belonging to the foreground is corrected so as to be emphasized more than the distant view in the image, and the region belonging to the distant view is corrected so as not to stand out from the near view in the image.

  FIG. 31A shows input / output characteristics assigned to a region belonging to a group of a near-ground and high-luminance region. This converts the luminance higher and converts the luminance lower in the low luminance part to make the contrast clearer. Compared to the slope (= 1) of the straight line having the same input and output as the dashed line rising to the right shown in FIG. 31 (a), the slope of the tangent line of the curve is higher in the portion where the luminance is higher than La1 and lower than La2. Is less than 1. In a portion where the luminance is higher than La2 and lower than La1, the slope of the tangent of the curve is larger than 1, and conversion is performed so as to widen the gradation width.

  FIG. 31B shows input / output characteristics assigned to a region belonging to a foreground and low luminance group. This is to increase the overall luminance, and the gradient of the tangent is smaller than 1 in the portion where the luminance is higher than Lb1, and is larger than 1 in the portion lower than Lb1, so that the gradation width of the low luminance portion is widened. To do.

  FIG. 31C shows input / output characteristics assigned to a region belonging to a group of a distant view and a high luminance. This converts the slope of the tangent line to be smaller than 1 in the portion where the luminance is higher than Lc1, and is larger than 1 in the portion lower than Lc1, thereby reducing the contrast by reducing the gradation width of the high luminance portion. Further, the high luminance portion of the output luminance value is cut and darkened as a whole.

  FIG. 31D shows input / output characteristics assigned to a region belonging to a group of a distant view and a low luminance. In this case, the low luminance portion of the output luminance value is cut (provided with an offset) to brighten the entire image, and the tangent slope is set to be smaller than 1 as a whole so as to reduce the contrast.

  By using such a gradation correction table, an image having a sense of depth can be obtained by converting the near view to be bright and sharp and converting the distant view to be sleepy with low contrast.

  In addition, in the gradation correction table of FIG. 31, the input / output characteristics shown in FIG. 32 instead of FIG. 28C are applied to the area belonging to the distant view and high brightness group, and the area belonging to the distant view and low brightness group. However, the input / output characteristics shown in FIG. 33 may be applied instead of FIG.

  In FIG. 32, the swell of the S-shaped curve is made smaller than that in FIG. In FIG. 33, the portion where the luminance is equal to or less than Ld1 is black, and the curve bulge is made smaller than that in FIG. 31B to suppress the method of increasing the overall luminance. By using such a gradation correction table, it is possible to naturally convert an image of an object in the foreground to an image in which the object in the foreground is emphasized by enhancing contrast enhancement in the foreground and weakening in the background.

<Modification 2>
(When image processing is performed after shooting)
In the above description, the image processing method of the present invention has been described by taking the image pickup apparatus having the configuration shown in FIG. 1 as an example. However, the image processing method is not limited to the image pickup apparatus, and digital images such as liquid crystal displays and plasma televisions are used. The image processing method according to the present invention can also be used in a display device that performs processing. FIG. 34 shows a display device including an image processing apparatus (corresponding to an “image processing unit”) that performs an image processing method (blurring process and distance-dependent gradation correction) according to the present invention.

  The display device shown in FIG. 34 is the same as the imaging device shown in FIG. 1, the driver unit 8, the expansion processing unit 9, the display unit 12, the audio output circuit unit 13, the speaker unit 15, the TG 16, the CPU 17, the memory 18, and the operation unit 19. Bus lines 20 and 21 and an external memory 22. Unlike FIG. 1, the image processing device 5 includes an image processing device 5 a that processes the image signal acquired by the decompression processing unit 9 instead of the image processing unit 5. The image processing device 5a has a blur processing unit 30 shown in FIG. The external memory 22 holds an image file having a header having distance information and average luminance information for each region in an image signal that has not been subjected to image correction at the time of shooting in the imaging apparatus of the present embodiment.

  FIG. 35 is a flowchart in the case of performing blur processing and distance-dependent gradation correction image processing in the display device of this embodiment. When the user sets the display device to the image processing mode (STEP 3501) and selects an image to be displayed on the display unit 12 (STEP 3502), the signal of the selected image is decompressed by the decompression processing unit 9 from the external memory 22 via the driver unit 8. Then, the signal is converted into a signal that can be displayed on the display unit 12 by the image processing device 5a, and the selected image is displayed on the display unit 12. Next, it is detected whether or not the user selects execution of the blurring process (STEP 3503). When execution of the blurring process is selected, the focus distance information of each area of the image written in the header is read into the low-pass filter unit 34, and each area is grouped by the focus distance range to which it belongs (STEP 3504). When the group information of the distance range to which each area belongs is stored in the header, it may be read. The focus distance information and the group information of the distance range to which it belongs may be read directly from the external memory 22, or may be given after being decompressed by the decompression processing unit 8 when stored as a compressed code. The same applies to the average luminance information described later. Subsequently, the low-pass filter unit 34 sets the blurring degree of each area of the image from the focus distance information (STEP 3505). An image signal subjected to blurring processing with the set blurring degree is held (STEP 3506).

  Next, the process waits for input of a correction intensity value for changing the intensity of the blurring process. The user inputs a correction intensity value, and the value is read into the image processing unit 5a (STEP 3507). FIG. 36 shows an example of display for inputting the correction intensity value. A display unit 12 and an input button 19a of an operation unit 19 including five buttons, top, bottom, left, right, and center, are provided on the panel of the display device, and the display unit 12 is pressed by pressing the left and right input buttons 19a. The intensity of the correction intensity value can be changed with reference to the display of the correction intensity value by the slide bar on the displayed selected image.

  When the correction intensity value is input, the low-pass filter unit 34 receives the stored image signal that has been subjected to the normal blurring process and the image signal that has not been subjected to the blurring process as the input correction intensity value. The load is added, and the added image is displayed on the display unit 12. (STEP 3508).

Here, load addition at each pixel constituting the image will be described. It is assumed that the luminance of the pixel after the load addition is Y, the luminance of the pixel not subjected to the blurring process is Yin, the luminance of the pixel subjected to the normal blurring process is Yconv, and the load addition coefficient is K. At this time, Y is Y = (1−K) Yin + K · Yconv
It can be expressed as. The load addition coefficient and the correction intensity value have a linear relationship as shown in FIG. 37, and the weighted addition coefficient K_max corresponding to the strongest correction intensity may be a positive number of 1 or less.

  After that, when the user does not instruct the end of the blurring process (No in STEP 3509), the correction intensity value is changed by pressing the left and right input buttons 15a (STEP3507), and the blurring process in which the intensity is changed based on the correction intensity value is performed. The corrected image is displayed on the display unit 12. That is, it is possible to change the intensity of the blurring process while confirming the degree of the blurring process on the display unit 12. When the end of the blurring process is instructed (YES in STEP 3509), the process proceeds to detection of selection of distance dependent gradation correction (STEP 3510).

  Next, when the user selects execution of distance-dependent gradation correction (YES in STEP 3510), the focus distance information of each area of the image written in the header is read into the gradation correction unit 43, and the registration of each area. The focal distance ranges are grouped (STEP 3511). When the group information of the distance range to which each area belongs is stored in the header, it may be read. The focus distance information and the group information of the distance range to which it belongs may be read directly from the external memory 22, or may be given after being decompressed by the decompression processing unit 8 when stored as a compressed code. The same applies to the average luminance information described later.

  Subsequently, the luminance information calculation unit 41 calculates the average luminance of each area of the image from the image information, stores it in the luminance information storage unit 42, and stores each of the information stored in the luminance information storage unit 42 in the gradation correction unit 43. The average luminance information of the area is read (STEP 3512). If the average luminance information of each area is written in the header, it may be read directly by the gradation correction unit 43.

  The gradation correction unit 43 groups each region into, for example, high luminance, medium luminance, and low luminance based on the average luminance of each region directly read from the luminance information storage unit 42 or the external memory 22. Next, the gradation correction table set for each group of in-focus distance and average brightness stored in the memory 18 is read, and the gradation is determined based on the in-focus distance group and average brightness group to which each area belongs. An input / output characteristic for correction is assigned (STEP 3513), and an image signal whose gradation is corrected by the gradation correcting unit 43 according to the input / output characteristic assigned to each area is held (STEP 3514).

  Next, the input wait state for the correction intensity value for changing the intensity of the tone correction input / output characteristic is entered. The user inputs the correction intensity value with the input button 19a, and the value is read into the image processing unit 5a (STEP 3515).

  When the correction intensity value is input, the gradation correction unit 43 receives the input image signal that has been subjected to the normal gradation correction and the image signal that has not been subjected to the gradation correction. With the intensity value, the load is added as described above, and the image with the load added is displayed on the display unit 12. (STEP 3516).

  After that, when the user does not give an instruction to end the gradation correction (No in STEP 3516) and the left and right input buttons 15a are pressed to change the correction intensity value (STEP 3515), the input / output changes the intensity based on the correction intensity value. The gradation is corrected according to the characteristics, and the corrected image is displayed on the display unit 12. That is, the intensity of gradation correction can be changed while confirming the degree of gradation correction on the display unit 12. Then, the end of gradation correction is instructed (YES in STEP 3517), and the image subjected to the blurring process or gradation correction is compressed and recorded in the external memory (STEP 3518), and the process ends (STEP 3519). It may be possible to select whether or not to record an image that has been subjected to blurring processing or gradation correction.

  In the display device of the present invention, the order of the blurring process and the distance dependent gradation correction may be reversed. The display device of the present invention may have only one of the blurring processing function and the distance dependent gradation correction function. Further, the imaging device may operate as the display device of the present invention.

  The present invention can be applied to an imaging apparatus having an autofocus function and a blur function.

These are block diagrams which show the internal structure of the imaging device which concerns on 1st Embodiment. These are the flowcharts for demonstrating the basic operation | movement of the imaging device which concerns on 1st Embodiment, and the operation | movement of the blurring process mode. Is an example in which the captured image area is divided into a plurality of areas and an AF evaluation area is set. These are graphs showing the relationship between the focus evaluation distance and the intensity of the high-frequency component in the AF evaluation area of each captured image area and each area. These are block diagrams which show the structure of the blurring process part which concerns on 1st Embodiment. These are block diagrams which show the structure of a distance information detection part. These are the flowcharts for demonstrating the operation | movement which detects distance information. Is an example of a distance detection image and a recording image. Is an example in which the image of FIG. 8 is divided into nine regions. FIG. 10 is a graph showing the relationship between the intensity of the high frequency component and the focusing distance in each region of FIG. These are graphs showing the relationship between the intensity of the high-frequency component and the in-focus distance in an area where there are a plurality of subjects having different distances from the imaging device. These are graphs showing the relationship between the distance to the subject and the cutoff frequency according to the first embodiment. These are four schematic diagrams among the area | regions which divided the image. These are the graphs which carried out the linear interpolation of the cutoff frequency between the two points where the cutoff frequency was set. These are the graphs which show the relationship between the intensity | strength of the high frequency component of each area | region and the focusing distance which set the distance group. These are block diagrams which show the structure of the blurring process part which concerns on 2nd Embodiment. These are graphs showing the relationship between the cut-off frequency of the low-pass filter and the output level. These are graphs showing the relationship between the distance to the subject and the multiplication coefficient according to the second embodiment. These are the schematic diagrams of the image which performed the blurring process which concerns on 2nd Embodiment. These are block diagrams which show the structure of the blurring process part which concerns on 3rd Embodiment. These are graphs showing the relationship between the distance to the subject and the multiplication coefficient according to the third embodiment. These are block diagrams which show the structure of a face detection apparatus. Is an example of a hierarchical image obtained by the reduced image generating means. FIG. 10 is a diagram for explaining face detection processing; These are block diagrams which show the structure of the blurring process part which concerns on the modification 2. FIG. These are flowcharts for explaining the gradation correction operation. Is an example of input / output characteristics assigned to each distance group and average luminance group. Fig. 4 is a graph of input / output characteristics obtained by changing gradations by linear interpolation between different input / output characteristics. These are graphs of the input / output characteristics set to two predetermined distances and the input / output characteristics applied to the distance between them. Is another example of the input / output characteristics assigned to each distance group and average luminance group. These are another example of the input / output characteristics applied to a region belonging to a group of a distant view and a high luminance. Is another example of input / output characteristics applied to a region belonging to a group of a distant view and low luminance. These are block diagrams which show the internal structure of the display apparatus of this invention. These are flowcharts in the case of performing image correction in the display device of the present embodiment. Is an example of a display for inputting a correction intensity value. These are graphs showing the relationship between the load addition coefficient and the correction strength value.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image sensor 2 Lens part 3 AFE
4 microphone 5 image processing unit 5a image processing device 6 audio processing unit 7 compression processing unit 8 driver unit 9 expansion processing unit 10 video output circuit unit 11 video output terminal 12 display unit 13 audio output circuit unit 14 audio output terminal 15 speaker unit 16 TG
17 CPU
17a Distance information storage unit 18 Memory 19 Operation unit 19a Input button 20 Bus line 21 Bus line 22 External memory 23 Distance detection unit 23a High frequency component detection unit 23b High frequency component comparison unit 30 Blur processing unit 31 Frame memory 32 Distance detection unit 34 Low pass filter 34a Low-pass filter for near view 34b Low-pass filter for middle scene 34c Low-pass filter for far view 35 Signal processing unit 36 Multiplier 37 Multiplication coefficient setting unit 38 Adder 39 Edge signal generation unit

Claims (11)

An image processing apparatus that performs blurring processing on a captured image including a plurality of subjects,
An image processing apparatus comprising: a blur processing unit that performs blur processing on the captured image according to a blurring degree corresponding to an imaging distance that is a distance from the imaging apparatus that captured the captured image to each subject.   The image processing apparatus according to claim 1, wherein the blurring degree is higher as the imaging distance is longer.   The image processing apparatus according to claim 1, wherein the blurring degree is higher as the imaging distance is longer than a predetermined distance.   The image processing apparatus according to claim 3, wherein the predetermined distance is a distance to the subject having a specific portion. An imaging unit for acquiring a captured image including a plurality of subjects;
An imaging distance detection unit that detects an imaging distance that is a distance from the imaging unit to each subject;
A blur processing unit that performs blur processing on the captured image according to a blur level according to the imaging distance of each subject;
An imaging apparatus comprising: An imaging unit for acquiring a captured image including a plurality of subjects;
An area dividing unit for dividing the captured image into a plurality of divided areas;
An imaging distance detection unit that detects an imaging distance that is a distance from the imaging unit to a subject included in each of the divided regions;
A blur processing unit that performs a blur process for each of the divided regions according to the blur level corresponding to each imaging distance;
An imaging apparatus comprising: A main body unit including an electrical signal of the captured image as acquired by the imaging unit, and a header unit having information on distances to subjects included in the respective divided areas detected by the imaging distance detection unit. A storage unit for storing image files,
The blur process is performed by applying a blurring degree set according to a distance to the subject stored in the header unit to the electrical signal of the captured image of the main body unit stored in the storage unit. Imaging device. A distance group generation unit that divides the divided region into a plurality of distance groups based on the imaging distance,
8. The blur processing unit according to claim 6, wherein the blur processing unit performs blur processing for each of the distance groups to which the divided region belongs according to a blur level corresponding to each of the distance groups. Imaging device. An imaging unit for acquiring a captured image including a plurality of subjects;
An imaging distance detection unit that detects an imaging distance that is a distance from the imaging unit to each subject;
A recording unit that records the captured image and information related to the imaging distance of each subject in association with each other;
An imaging apparatus comprising: A captured image including a plurality of subjects;
An image file comprising information relating to an imaging distance, which is a distance from an imaging device that has captured the captured image to each subject, and is associated with each other. An imaging distance detection step for detecting an imaging distance that is a distance to the subject;
An imaging step of acquiring a captured image including a plurality of the subjects;
A gradation correction step of performing a blurring process on the captured image acquired in the imaging step according to a blurring degree according to the imaging distance of each subject detected in the imaging distance detection step;
An image processing method comprising:

Images