JP2006050155A - Image sharpening apparatus and method, electronic apparatus using image sharpening apparatus and sharpening program, and recording medium for recording the program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image sharpening apparatus and method or the like capable of generating an image the whole sharpness of which is uniform by eliminating the effect of lens aberration or the like. <P>SOLUTION: A device such as an electronic apparatus for obtaining image data through a lens is provided with the image sharpening apparatus 1 including: a distance calculation section 3 for calculating a distance between a reference pixel being a processing reference for a plurality of pixels arranged in a planar direction and an object pixel being an object of sharpening; a sharpening degree calculation section 4 for calculating a sharpening degree of the object pixel on the basis of the calculated distance between the reference pixel and the object pixel; and an image sharpening section 5 for applying sharpening processing to image data on the basis of the calculated sharpening degree. The apparatus 1 applies sharpening processing in response to a degree of blur even to image data whose degree of blur changes depending on the distance from a particular part of the lens and can produce an image the whole sharpness of which is uniform. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image sharpening device and method for sharpening an image, an electronic device using the image sharpening device, and peripheral devices thereof.

  In recent years, along with the development of small, low power consumption image sensors, it is possible to incorporate cameras in various wireless communication terminals such as personal digital assistants and mobile phones. Since these built-in cameras are small in size, they have a lower quality lens than general digital cameras, and images with a blurred outline are often generated compared to general digital cameras. Processing (sharpening) that emphasizes the change and sharpens the image is necessary.

Conventionally, image sharpening processing has been performed uniformly on the entire image. In recent years, in such sharpening processing, light reception information of a target pixel to be sharpened is obtained around the target pixel. A sharpening processing system that performs a sharpening process while changing the sharpening degree of an image by comparing and referring to the received light information of the pixels is known (for example, see Patent Document 1).
Japanese Patent Laid-Open No. 2003-256831

  However, as shown in the schematic diagram of FIG. 8, when the camera 02 condenses on the imaging unit 04 by the lens 03, the refractive power of the light becomes stronger toward the periphery of the lens 03 and the light may not converge at one point (spherical surface (Aberration) and a difference in refractive index depending on the wavelength causes a phenomenon in which light does not converge at one point on the optical axis 03a (chromatic aberration), and the image is blurred when an image is formed on the imaging unit 04. Since the influence of the aberration increases as the light passes through the peripheral portion of the lens 03, even if the entire image is subjected to uniform sharpening using the system described in Patent Document 1, the peripheral portion of the image is increased. There was a problem that it was not possible to compensate for the large blur difference. In particular, since aberrations often occur in a low-quality lens, an image obtained by an imaging device such as a camera-equipped mobile phone, a camera-equipped PHS, or a camera-equipped personal digital assistant has an aberration at the periphery of the image as compared with a normal digital camera. The above-mentioned problem becomes more serious.

  The present invention has been made in view of such problems, and eliminates blurring of an image whose degree changes depending on a distance from a specific position, such as lens aberration, and makes the entire image uniform. An object is to provide an image sharpening apparatus and method, an electronic apparatus, a sharpening program, and a recording medium on which the program is recorded.

  The present invention relates to an image sharpening apparatus and method that can generate an image with uniform overall sharpness, a peripheral device, and the like. Specifically, the above-described problem according to the present invention is related to reference pixels in a plurality of pixels of an image sensor. Based on the value of the distance to the target pixel as a target to be sharpened, based on the sharpening degree, a sharpening degree calculating means for calculating a sharpening degree for the image data obtained from the target pixel, This is achieved by an image sharpening device comprising sharpening processing means for sharpening image data.

  Then, by calculating the sharpening degree based on the distance value between the reference pixel and the target pixel and applying the sharpening process, an image whose degree of blur changes depending on the distance from the reference pixel is automatically In particular, it is possible to perform a sharpening process in which the degree is changed in accordance with the degree of blur.

  In the image sharpening device for sharpening image data obtained from an image sensor through a lens, the above-described problem of the present invention is a distance between a reference pixel and a target pixel to be sharpened in a plurality of pixels of the image sensor. A distance calculation means for calculating a value of the reference, a memory for storing a distance value between the reference pixel and the target pixel and a sharpening degree relationship with respect to the target pixel, a distance obtained by the distance calculation means, and the memory And a sharpening degree calculating means for calculating a sharpening degree with reference to the above, and a sharpening process is performed on the image data received by the target pixel based on the sharpening degree calculated by the sharpening degree calculating means. Effectively achieved by an image sharpening device comprising sharpening processing means.

  Then, the distance between the reference pixel and the target pixel is calculated by the distance calculation means, and processing for image sharpening is performed on the basis of the calculated distance value and the sharpening degree stored in the memory. Even for image data obtained through a lens whose degree of aberration varies depending on the distance from the part, a reference pixel is provided corresponding to the reference part of the lens, and the distance from the reference pixel is accurately calculated. The sharpening process can be performed with varying degrees according to the distance.

  In addition, the above-described problem of the present invention is more effectively achieved by an image sharpening device in which the reference pixel is a pixel that receives light transmitted through the optical axis of the lens.

  Then, by providing the reference pixel on the optical axis of the lens, it is possible to sharpen the image obtained through the lens that has a greater influence of aberration toward the peripheral portion, corresponding to the influence of aberration.

  Further, the above-mentioned problem of the present invention is that the sharpening degree calculation means calculates the sharpening degree using a function depending on the distance between the reference pixel and the target pixel, and the function Is more effectively achieved by an image sharpening device which is a function of increasing the sharpening degree with distance.

  By using a function that depends on the distance between the reference pixel and the target pixel for sharpening, the sharpening process can be performed accurately and quantitatively, and a function that increases with distance is applied, and aberrations are applied. Sharpening processing can be performed with a function corresponding to the influence of.

  Further, the above-described problem of the present invention is that the sharpening processing means performs an image sharpening device that performs sharpening processing on at least one of luminance information and color difference information forming the image data, and the image data. This can also be achieved more effectively by an image sharpening device that performs sharpening processing on the luminance information and color difference information to be formed at different degrees.

  Then, sharpening by the sharpening processing means is applied to the image data formed by the luminance information and color difference information, so that this is a compression coding method that is often used when recording and transmitting information with a digital camera such as JPEG. The invention can be applied. For image data that is compression-encoded by a method such as JPEG, which reduces the color difference information more than luminance information at the time of compression encoding, only luminance information, only color difference information, or luminance information and color difference information are separately provided. By performing the sharpening process with the degree of the above, it is possible to compensate for the change in the texture of the image caused by the change in the ratio between the luminance information and the color difference information by the compression encoding, and to generate a high quality image.

  In addition, in the image sharpening method for sharpening image data obtained from an image sensor through a lens, the above-described problem of the present invention is a distance between a reference pixel and a target pixel to be sharpened in a plurality of pixels of the image sensor. A distance calculation step for calculating a value of the distance; a distance calculated by the distance calculation step; a memory storing a relationship between a distance value between the reference pixel and the target pixel and a sharpening degree with respect to the target pixel; Then, a sharpening degree calculating step for calculating a sharpening degree, and a sharpening process for performing sharpening processing on the image data received by the target pixel based on the sharpening degree calculated by the sharpening degree calculating step. And a sharpening process step.

  Then, it is possible to perform a sharpening process in which the degree is changed for each pixel according to the distance from the reference pixel.

  In the image sharpening step, the above-mentioned problem of the present invention is an image sharpening method in which sharpening processing is performed on at least one of luminance information and color difference information forming the image data, and the image data Is effectively achieved by an image sharpening method in which the brightness information and the color difference information forming the image are sharpened at different degrees.

  The present invention can be applied to a compression encoding method such as JPEG by performing sharpening in the sharpening processing step on image data formed with luminance information and color difference information.

  In addition, the above-described problems of the present invention are achieved by an electronic device typified by various wireless communication terminals including any one of the above-described image sharpening devices in an electronic device capable of capturing image data. The object is achieved by a program for causing a computer to execute one of the above-described image sharpening methods, or a recording medium on which the program is recorded.

  The above-described image sharpening device is provided in an electronic device that can capture image data, such as a camera-equipped mobile phone, thereby supplementing the effects of aberrations in various electronic devices including lenses with large aberrations. Image sharpening processing can be performed.

  According to the present invention, the image data whose degree of blur changes depending on the distance from a specific portion of the lens is also subjected to a sharpening process according to the degree of blur, so that the overall sharpness is uniform. Can be generated, and an image of good quality can be obtained.

  Hereinafter, preferred embodiments of an image sharpening method, a sharpening device, and an electronic apparatus according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram of an imaging apparatus 01 that uses the image sharpening apparatus according to the present embodiment. The main body of the imaging device 01 is a main body of various portable terminals incorporating various imaging devices such as a camera-equipped mobile phone, a camera-equipped PHS (Personal Handy-Phone System), and a camera-equipped personal digital assistant. The imaging device 01 includes a camera unit 02 and captures an image obtained through the lens 03 on the imaging unit 04.

  FIG. 2 is an enlarged view of the light receiving surface of the imaging unit 04. FIG. The imaging unit 04 is a variety of imaging devices such as a CCD (change coupled device) imaging device, and a rectangular pixel 04b formed by a light receiving element such as a photodiode is integrated on a light receiving unit 04a provided on the light receiving surface side. The intensity of light received by each pixel 04b is detected as the magnitude of the electric signal. In the light receiving unit 04a, the pixels 04b are arranged in a matrix in the planar direction (m rows and n columns in FIG. 2), and an arbitrary coordinate 04c is obtained by taking the x coordinate in the horizontal direction and the y coordinate in the vertical direction. (X, y). Then, a pixel 04b that receives light transmitted through the central portion of the lens 03 located on the optical axis of the lens 03 is formed as a reference pixel 04d at a substantially central portion of the light receiving portion 04a. In the present embodiment, the coordinates (X, Y) of the reference pixel 04d are used.

  A filter is attached to each surface of the pixel 04b, and each pixel 04b detects light of a color corresponding to the filter. The filter has three essential colors, red (R), green (G), and blue (B) as essential elements, and includes colors that improve the hue (for example, yellow, cyan, etc.). Also good. However, in this embodiment, it is assumed that there are only three types of filters, red (R), green (G), and blue (B), for the sake of simplicity.

  As shown in FIG. 1, the signal of the image data acquired by each pixel 04b in the imaging unit 04 is converted from an analog signal to a digital signal by the AD conversion unit 05 for each of red (R), green (G), and blue (B). After being converted and further transmitted to the color system conversion unit 06, RGB-YUV conversion is performed and converted into luminance information and color difference information of the YUV color system, and then input to the sharpening device 1.

  Further, the image data output from the sharpening device 1 is subjected to compression encoding processing such as JPEG in the compression unit 07 and is stored in the storage unit 08 or sent to the transmission unit 09 to be transmitted to another device by wireless communication or the like. Sent to.

  FIG. 3 is a block diagram showing the configuration of the image sharpening apparatus 1 as an embodiment of the present invention. The sharpening device 1 includes an image data acquisition unit 2 that acquires various types of image data, and a distance between a target pixel 04e and a reference pixel 04d that are objects to be sharpened from the image data acquired by the image data acquisition unit 2. Using the distance calculation unit 3 to be calculated and the value of the distance between the reference pixel 04d and the target pixel 04e calculated by the distance calculation unit 3, the sharpness that calculates and sets the sharpness (degree) of the target pixel 04e The image sharpening unit 4, the image sharpening unit 5 that sharpens the image data based on the sharpening degree calculated by the sharpening degree calculation unit 4, and the image sharpening unit 5 sharpens the image data. And an image data output unit 6 for storing images and outputting them to other processes. The image sharpening degree calculation unit 4 has a built-in memory 4a, and stores information necessary for calculating the sharpening degree for the target pixel, such as a gain correction value described later.

  The image data acquisition unit 2 is output from the imaging unit 04 and supplied with image data processed by the AD conversion unit 05 and the color system conversion unit 06, respectively.

  The distance calculation unit 3 is supplied with image data from the image data acquisition unit 2 and specifies position information in the vertical and horizontal directions of each pixel 04b forming the image data, and based on the position information. In addition, the reference pixel 04d is specified, and the distance between the reference pixel 04d and the target pixel 04e is calculated for each of the pixels 04b when each of the pixels 04b is the target pixel 04e to be sharpened.

  As shown in FIG. 2, when the coordinates of the target pixel 04e are (i, j), the distance between the reference pixel 04d and the target pixel 04e is sharpened by using the horizontal offset value and the vertical offset value. The distance between the target pixel 04e to be performed and the corrected reference pixel 04d is calculated. Specifically, the Euclidean distance (actual linear distance between the pixels 04) is calculated using the equation (1) shown below, or the four neighboring distances (moving only in the vertical or horizontal direction) are calculated using the equation (2). (Distance expressed by how many pixels need to be moved from the reference pixel 04d to the target pixel 04e) under the possible conditions.

Length = (i−CENTER_X + OFFSET_X) × (i−CENTER_X + OFFSET_X) +
(J−CENTER_Y + OFFSET_Y) × (j−CENTER_Y + OFFSET_Y) (1)
Length = | (i−CENTER_X + OFFSET_X) | + | (j−CENTER_Y + OFFSET_Y) |
... (2)

  Here, i represents the i-th in the horizontal direction of the sharpening target image (that is, the x coordinate of the target pixel 04e in the present embodiment), and j is the j-th in the vertical direction of the sharpening target image (that is, the present embodiment). CENTER_X indicates the horizontal center of the light receiving unit 04a in the imaging unit, CENTER_Y indicates the vertical center of the light receiving unit 04a in the imaging unit, OFFSET_X is Indicates the offset value in the horizontal direction for adjustment based on the image obtained through the center of the lens. OFFSET_Y is adjusted based on the image obtained through the optical axis at the center of the lens. The offset value in the vertical direction for performing adjustment when there is a deviation is shown.

  Expression (1) is an expression that is often used to derive an accurate distance in a two-dimensional space, and is excellent in that the distance can be calculated with high accuracy. Expression (2) is an approximate distance with a small number of operations. It is excellent in that it can be calculated.

  Next, the sharpening degree calculation unit 4 uses the distance value calculated by the distance calculation unit 3 (referred to as Length) to calculate the sharpening degree value (referred to as α) by a sharpening degree function. To do. Here, the sharpening degree refers to the degree of sharpening necessary for each target pixel 04e, and the sharpening degree function depends on the imaging unit of the camera, lens characteristics, and other device characteristics. This is a function set in advance so as to satisfy the sharpening degree for all the target pixels 04e.

  FIG. 4 is a diagram illustrating an example of the sharpening degree function. In FIG. 4, the horizontal axis indicates the distance value (hereinafter referred to as “Length”) between the reference pixel 04d and the target pixel 04e obtained from the distance calculation unit 3, and the vertical axis indicates the sharpening degree value ( α).

  In the function shown in FIG. 4, if the value of Length is 0 to X0, it becomes section 0. If X0 to X1, it becomes section 1. Similarly, if X1 to X2, it becomes section 2.. The sharpening degree function exhibits a straight line that rises to the right in each section, and the slope value increases as the value of the section increases (that is, the distance between the reference pixel 04d and the target pixel 04e increases). That is, the sharpening degree function is a function that depends on the value of the distance between the reference pixel 04d and the target pixel 04e.

Then, the sharpening degree value α is obtained based on the following equation (note that the value of the sharpening degree value α obtained by this equation is hereinafter referred to as “α value”).
SC_GAIN = SC_SLOPE × (Length−SC_RANGE) (3)
SC_OFFSET (j) = SC_OFFSET (j−1) × SC_SLOPE (j−1) (SC_RANGE (j) −SC_RANGE (j−1)) (4)

  Here, SC_GAIN indicates the gain of the corresponding section of Length, SC_SLOPE indicates the slope of the straight line of the corresponding section of Length, SC_RANGE indicates the start point of the corresponding section of Length, and SC_SLOPE (j) is each section Indicates the slope (k0, k1,..., Kn), SC_RANGE (j) indicates the start point (0, X1,..., X5) of each section, and n corresponds to the value of Length. Indicates the section number, j indicates the section number, and the gain correction value is a fixed value that is adjusted so that the α value is between 0 and αMax, and changes according to the performance characteristics of the camera. Indicates the maximum degree of sharpening, and SC_OFFSET indicates the value of SC_GAIN at the start point of each section.

  The function shown in FIG. 4 is an example of the sharpening degree function, and the number of sections, the starting point of each section, and the slope of the function in each section are the imaging unit of the camera, the characteristics of the lens, and other device characteristics. It can be freely set and adjusted as appropriate according to the situation.

  In addition, since it takes time to perform the above calculation for each target pixel 04e, a lookup table of Length values and α values is created in advance, and α can be obtained by referring to this lookup table. It is also possible to shorten the processing speed by obtaining the value. In this case, the lookup table is stored in the memory 4a.

  Then, the image sharpening unit 5 uses this α value obtained by the sharpening degree calculation unit 4 and this image when the image data of the target pixel 04e before the sharpening process is set to f (i, j). Pixel data h (i, j) of the target pixel 04e sharpened from the data f is obtained by either of the following two formulas (6) or (7).

  Where (i, j) is the i th column and j th row of the image, b (i, j) is a low pass convolution operator, and c (i, j) is a high pass convolution operator. , Β, μ are fixed values that are sharpening coefficients, αMax is the maximum value of α indicated by the sharpening degree calculation unit 4, and f (i, j) is the sharpening of the target pixel 04e. It is the previous image data, and h (i, j) is the output image data after the target pixel 04e is sharpened.

  That is, according to the calculations shown in Equation (6) and Equation (7), the original image is easily sharpened as the α value increases, and the sharpening characteristic of the output image h (i, j) with respect to the original image is strong. It shows that it becomes.

  The sharpening correction coefficient of β and μ is a value adjusted in accordance with the size of the convolution operator. For example, for a 3 × 3 convolution operator, β is 1/9 and μ is 1/8. It becomes.

  FIG. 5 is a plan view showing a representative example of an operator used in image processing by the sharpening processing unit 5 indicated by b and c in the above formula. The first operator 7 and the second operator 8 are high-pass convolution operators and form a Laplacian filter. When calculation is performed by applying these operators, the first operator 7 performs second order differentiation on the target pixel 04e with respect to the surrounding eight pixels, and the second operator 8 performs second order differentiation on the target pixel 04e with respect to the upper, lower, left, and right four pixels. By performing differentiation, these perform filtering processing for extracting the high-frequency component of the target pixel 04e, and emphasize the contour portion. On the other hand, the third operator 9 is a low-pass convolution operator. When calculation is performed by applying this operator, the density values of the image data for the target pixel 04e and the surrounding eight pixels are averaged, and this performs a filtering process for extracting the low frequency component of the target pixel 04e. Smooth the image. Note that the first to third operators 7, 8, and 9 shown in FIG. 5 form a convolution filter for 9 pixels of 3 × 3 pixels, but the present invention is not limited to this. For example, 5 × 5 pixels and 7 Other sizes of convolution operators such as x7 pixels may be used in the sharpening processing unit 5.

  Then, the image data output unit 6 outputs the image sharpened by the image sharpening unit 5 and supplies it to the compression unit 07 as described above.

  FIG. 6 is a flowchart showing image sharpening processing, and “S” in the figure indicates each step of the flow.

  First, in step S1, input image data is received from the image data acquisition unit 2, and i and j are set to 0 and initialized. Here, i indicates the pixel order in the horizontal direction, and j indicates the pixel order in the vertical direction. That is, as described above, (i, j) as the position information on the plane is the object to be sharpened. The coordinates of the target pixel are indicated.

  When the target pixel 04e to be sharpened is determined, the distance between the reference pixel 04d and the target pixel 04e is calculated by the distance calculation step S2 using the distance calculation unit 3. Next, based on the value of the distance between the reference pixel 04d and the target pixel 04e calculated in the distance calculation step S2, the sharpening degree of the target pixel 04e is calculated in the sharpening degree calculation step S3 using the sharpening degree calculation unit 4. Then, in the sharpening process step S4 using the image sharpening unit 5, the image data of the target pixel 04e is subjected to the sharpening process based on the calculated sharpening degree to obtain the target pixel 04e. Sharpen image data. In step S4, the target pixel 04e is filtered using an operator as shown in FIG. Specifically, the screen noise is first removed by smoothing using the low-pass convolution operator 9, and then the high-pass convolution operators 7 and 8 are used to emphasize the change in density value that forms the peripheral portion of the image, thereby sharpening. Make it. However, it is not necessary to stick to the above-mentioned method for the order and the number of times the operators are used. For example, first, sharpening processing is performed using the high-pass convolution operators 7 and 8, and then smoothing processing is performed using the low-pass convolution operator. Also good.

  In step S5, it is confirmed whether i has advanced to the maximum pixel (Max_Horizontal) in the horizontal direction of the image. If i is smaller than the maximum horizontal pixel (Max_Horizontal), it is determined that i has not advanced to the maximum horizontal pixel (Max_Horizontal) in the horizontal direction of the image, and i is increased by 1 (+1) in step S7. Increment) and return to step S2 to repeat the process.

  If i is equal to or greater than the maximum horizontal pixel (Max_Horizontal), it is checked in step S6 whether j has advanced to the maximum vertical pixel (Max_Vertical) in the image. If j is smaller than Max_Vertical, it is determined that j has not advanced to the maximum pixel (Max_Vertical) in the vertical direction of the image, j is incremented by 1 (+1 increment) in step S7, and i is set to 0. Returning to step S2, the process is repeated. If j is larger than the maximum pixel (Max_Vertical) in the vertical direction, the image sharpening process ends.

  Here, in the above flowchart, the order of the target pixels 04e to be sharpened is determined by the raster scan method, but all the pixels 04b of the light receiving unit 04a are finally sharpened at least once. If it becomes 04e, any order may be used for selecting the target pixel 04e to be sharpened.

  The above processing is realized mainly by software processing. However, the present invention is not necessarily limited to software processing, and can be realized by a logic circuit using hardware.

  FIG. 7 is a block diagram showing a configuration when the image sharpening device according to the present invention, which is another embodiment of the present invention, is realized by hardware logic.

  This image sharpening device 11 is also incorporated in the image pickup device 01 as shown in FIG. 1, as in the case of the image sharpening device 1 according to the embodiment shown in FIG. For the sake of simplicity, the same components as those of the above-described one embodiment will be described below using the same reference numerals.

  The image sharpening device 11 shown in FIG. 7 includes a target pixel 04e that is a target to be sharpened, and a pixel that receives light transmitted through the center of the lens 03 and positioned on the optical axis of the lens 03 as a reference pixel 04d. The distance calculation circuit 12 that calculates the distance of the target pixel 04e, and the sharpness degree calculation circuit 13 that sets the sharpness degree of the target pixel 04e based on the distance value between the reference pixel 04d and the target pixel 04e calculated by the distance calculation circuit 12 And a sharpening processing circuit 14 that performs a sharpening process on the image data based on the sharpening degree calculated by the sharpening degree calculation circuit 13.

  In the sharpening device 11, the distance calculation circuit 12 calculates the distance between the target pixel 04e and the reference pixel 04d for the input image data obtained through a lens such as a camera, and the sharpening degree calculation circuit 13 calculates the reference pixel 04d. Based on the value of the distance to the target pixel 04e, the sharpening degree is set for each of the target pixels 04e. The sharpening processing circuit 14 performs a sharpening process on the image data of the target pixel 04e based on the calculated sharpening degree value. In this case as well, the filtering process using various convolution operators is performed as in step S4 to sharpen the target pixel 04e. The sharpening device 11 outputs the image data that has been processed after performing the sharpening process on all the target pixels 04e, and supplies the image data to the compression encoding unit 07.

  As described above, as shown in one embodiment and other embodiments of the present invention, the image sharpening devices 1 and 11 according to the present invention apply the distance calculation unit 3 (distance to the target pixel 04e to be sharpened. The calculation circuit 12) calculates the distance from the reference pixel 04d, sets the sharpening degree by the sharpening degree calculation unit 4 (sharpening degree calculation circuit 13) based on the calculated distance, and then performs the sharpening processing unit. 5 (sharpening processing circuit 14), the distance from the reference pixel 04d is obtained even if the image data is obtained through a lens in which the sharpness loss degree differs greatly between the central portion and the peripheral portion of the image. Depending on the process, the degree of sharpening can be changed. Accordingly, even for image data obtained through a lens whose degree of aberration changes depending on the distance from the reference portion, such as a lens having a greater influence of aberration toward the peripheral portion, for example, the reference portion ( In the present embodiment, the pixel on the optical axis) is provided with a reference pixel 04d, and a sharpening process is performed with varying degrees according to the distance from the reference pixel 04d. High images can be generated.

  It should be noted that the present invention is applied not only to the lens problem but also to the case where the sharpness differs between the central portion and the peripheral portion of the image as a result of performing processing such as shading on the image data. Since an image with uniform sharpness can be generated, an image with high quality can be obtained similarly.

  In addition, the invention according to the present embodiment generally has a small size and an aberration in order to achieve excellent functions and effects even when the sharpness of the image obtained from the lens central portion and the image obtained from the lens peripheral portion is greatly different. Even when the invention according to the present embodiment is applied to a portable information terminal such as a mobile phone device with a camera or a portable information terminal, in which a large, low-quality lens is often used, a remarkable effect is exhibited. High quality images can be obtained.

  Furthermore, as shown in the above-described embodiments, the present invention applies the sharpening by the sharpening processing means to the image data formed by the luminance information and the color difference information, so that the information can be obtained with a digital camera such as JPEG. Applicable to compression encoding methods often used during recording and transmission, and to compensate for changes in image quality caused by changes in the ratio of luminance information and color difference information due to compression encoding, and to generate high-quality images Is possible.

  In the present embodiment, the sharpening device 1 is provided immediately before the compression coding unit 07 and performs the sharpening process on the luminance information and the color difference information before the compression coding process. Without being limited thereto, it is also possible to provide the sharpening device 1 after the compression coding unit 07 and perform the sharpening processing on the luminance information and color difference information that have been compression-coded.

  Further, the sharpening device 1 according to the present invention can be applied to image data other than luminance information and color difference information without being limited to the color space. The present invention can be applied to part or all, or part or all of the XYZ signal and YIQ signal from which the RGB signal is converted.

  Moreover, both the above-mentioned embodiments are illustrations of preferred embodiments of the present invention, and the scope of the present invention is not limited thereto.

  Furthermore, in the present embodiment, the light receiving unit 04a in which the rectangular pixels 04b are arranged in a matrix shape is formed on the surface of the imaging unit 04. However, the shape of the pixels 04b is not limited to a rectangular shape, and the arrangement is in a matrix shape. For example, a substantially wedge-shaped pixel may be radially arranged, or a regular hexagonal pixel may be regularly arranged. In such a case, it can be considered that it is often more efficient to determine the order of the target pixels 04e to be sharpened without using the raster scan method described above.

  In the above best mode, the imaging device 01 is built in the main body of a portable terminal such as a mobile phone, PHS, or portable information terminal. However, the present invention is not limited to this. For example, a general digital camera, The present invention can be applied to any device as long as it is an electronic device equipped with an imaging device, such as a personal computer provided with an imaging device.

  Furthermore, the type, number, connection method, and the like of each circuit unit constituting the image sharpening device are not limited to the above-described embodiments.

  The present invention also includes a computer program for causing various computers to execute the above-described image sharpening method. The computer program may be obtained and installed by the computer main body through any route.

It is a block diagram which shows the structure of the imaging device in which the image sharpening apparatus which concerns on one embodiment and other embodiment of this invention is used. It is an enlarged view of the light-receiving surface of the image pick-up element used for the said imaging device. It is a block diagram which shows the structure of the sharpening apparatus of the image which concerns on one embodiment of this invention. It is an example of the sharpening degree function used for the sharpening apparatus of an image same as the above. It is a top view which shows the example of an operator of the image process used for the sharpening apparatus of an image same as the above. It is a flowchart which shows the sharpening process of the image in the sharpening apparatus of an image same as the above. It is a block diagram which shows the structure of the sharpening apparatus of the image implement | achieved by the hardware logic based on other embodiment of this invention. It is a schematic diagram which shows the lens aberration of the camera as a problem of a prior art.

Explanation of symbols

01 Imaging device
02 Camera section
03 Lens section
03a Optical axis
04 Imaging unit
04b pixel
04d Reference pixel
04e Target pixel 1 Sharpening device 2 Image data acquisition unit 3 Distance calculation unit 4 Sharpening degree calculation unit 4a Memory 5 Image sharpening unit 6 Image data output unit 11 Sharpening device 12 Distance calculation circuit (distance calculation means)
13 Sharpening degree calculation circuit (sharpening degree calculation means)
14 Sharpening processing circuit (sharpening processing means)

Claims (14)

Sharpening degree calculation means for calculating a sharpening degree for image data obtained from the target pixel based on a distance value between a reference pixel and a target pixel as a target to be sharpened in a plurality of pixels of the image sensor; ,
An image sharpening apparatus comprising: a sharpening processing unit that sharpens the image data based on the sharpening degree. In an image sharpening device that sharpens image data obtained from an image sensor through a lens,
Distance calculating means for calculating a distance value between a reference pixel and a target pixel to be sharpened in a plurality of pixels of the image sensor;
A memory for storing a relationship between a distance value between the reference pixel and the target pixel and a sharpening degree of the target pixel;
A sharpening degree calculating means for calculating a sharpening degree with reference to the distance obtained by the distance calculating means and the memory;
A sharpening processing unit for performing a sharpening process on the image data received by the target pixel based on the sharpening degree calculated by the sharpening degree calculating unit. .   The image sharpening device according to claim 1, wherein the reference pixel is a pixel that receives light transmitted through an optical axis of the lens. The said sharpening degree calculation means calculates the said sharpening degree using the function depending on the distance of the said reference pixel and the said object pixel, The Claim 1 characterized by the above-mentioned. Image sharpening device.   The image sharpening apparatus according to claim 4, wherein the function is a function whose sharpening degree increases with distance.   6. The image according to claim 1, wherein the sharpening processing unit performs a sharpening process on at least one of luminance information and color difference information forming the image data. Sharpening device.   6. The image according to claim 1, wherein the sharpening processing unit performs a sharpening process on the luminance information and the color difference information forming the image data at different degrees. Sharpening device. In a method for sharpening an image that sharpens image data obtained from an image sensor through a lens,
A distance calculating step of calculating a distance value between a reference pixel and a target pixel to be sharpened in a plurality of pixels of the image sensor;
A sharpness for calculating a sharpening degree by referring to a distance calculated by the distance calculating step, a value of a distance between the reference pixel and the target pixel, and a memory storing a relationship of the sharpening degree with respect to the target pixel. A conversion degree calculating step;
A sharpening processing step of performing a sharpening process on the image data received by the target pixel based on the sharpening degree calculated by the sharpening degree calculating step. .   The image sharpening method according to claim 8, wherein in the sharpening processing step, sharpening processing is performed on at least one of luminance information and color difference information forming the image data.   9. The image sharpening method according to claim 8, wherein, in the sharpening processing step, the brightness information and the color difference information forming the image data are subjected to sharpening processing at different degrees. In electronic devices that can capture image data,
An electronic apparatus comprising the image sharpening device according to claim 1.   The electronic device according to claim 11, wherein the electronic device is a wireless communication terminal with a camera or a portable information terminal with a camera.   A program for causing a computer to execute the image sharpening method according to any one of claims 8 to 10. A computer-readable recording medium having the program according to claim 13 recorded thereon.

Images