JP2007053659A - Measurement imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a measurement imaging device capable of performing highly precise smear correction and acquiring a measurement image with high accuracy (high image quality). <P>SOLUTION: The measurement imaging device includes an OB unit 25 (optical shielding unit) as optical shielding region, in which a structural pixel of an imaging sensor 2 is masked; a smear reference used as a reference formed based on the pixel output information of the OB unit 25 for smear correction time; an image memory 9 for storing defective pixel information of the OB unit 25 and dark-time output characteristic information of each structural pixel; and an image processing unit 5 for performing smear reference correction for removing an influence of the defective pixel and dark-time output characteristics effecting on the smear reference from the smear reference, based on the defective pixel information stored in the image memory 9 and the dark-time output characteristic information. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a measurement imaging apparatus for measuring two-dimensional luminance and color using a planar light source such as a display and a backlight or a predetermined set of point light sources as a measurement target. The present invention relates to a measurement imaging apparatus capable of performing smear correction on generated smear.

  Conventionally, a CCD image sensor has been mounted as an image sensor for taking higher-quality digital images, and flat light sources such as displays (FPDs) and backlights, and predetermined point light sources are assembled in a flat shape. There is an imaging device for measurement that measures a two-dimensional luminance and color with a target as a measurement object. In general, a problem that smear, which is a kind of image noise, occurs in imaging using a CCD image sensor has been known for a long time.

For example, as shown in FIG. 13 (highlight image) 710 in FIG. 13, the smear is a vertical direction (in a position where the subject 702 exists) when a high-brightness subject 702 such as a light exists in the imaging area 701. This is a phenomenon in which streak of light (smear 703) appears in the vertical direction. When strong light is incident on the CCD, light leaks into the vertical transfer section of the CCD, and this light is photoelectrically converted to generate unnecessary charges. It happens. When the smear is generated, as shown in the diagram of reference numeral 720 (the state on the imaging surface and the output waveform of the smear), for example, a position corresponding to the smear 703 is present in a certain CCD pixel line 704 in the horizontal direction. An output signal 705 having a steep mountain-shaped output waveform with a high output value appears. Note that a highlight portion 706 in the highlight image indicated by reference numeral 710 is called blooming, and is a phenomenon that occurs when strong light is incident and excess charge generated in the pixel overflows to surrounding pixels. The image looks like a ooze.
Japanese Patent Publication No. 56-35067

  By the way, as a measurement object of the measurement imaging apparatus, specifically, for example, a display image of a liquid crystal display (for example, a color image of a single color or a plurality of colors) or a display unit of an instrument panel of an automobile or the like (for example, an indicator unit made of LEDs). Although the purpose is to evaluate whether the brightness and color as designed are obtained, strict accuracy is required (simply a beautiful image such as a photograph taken with a normal digital camera). The purpose is different from getting.) For this reason as well, when smear occurs in photographing using the measurement imaging apparatus, it is necessary to correct the smear on the photographed image with higher accuracy.

  Patent Document 1 discloses a basic concept of performing smear correction by detecting a smear component and subtracting it. Various methods have been proposed for smear correction. However, the conventional method has not been a correction technique that meets the required accuracy of measurement applications.

  The present invention has been made in view of the above circumstances, and provides a measurement imaging apparatus capable of performing high-precision smear correction and thus obtaining a high-precision (high-quality) measurement image. With the goal.

  An imaging device for measurement according to claim 1 of the present invention includes an imaging sensor that captures a predetermined measurement object using a CCD, and is configured to be capable of correcting smear generated in a captured image by the imaging sensor. A light-shielding unit as an optical light-shielding region in which pixels constituting the imaging sensor are masked, and a smear reference used as reference data for smear correction created based on pixel output information of the light-shielding unit. A storage unit that stores defective pixel information in the light-shielding unit and dark output characteristic information of each component pixel, and a defective pixel based on the defective pixel information and dark output characteristic information stored in the storage unit. An image processing unit that performs smear reference correction that excludes, from the smear reference, influence on the smear reference due to dark output characteristics. I am characterized in.

  According to the above configuration, the light shielding unit is an optical light shielding region in which the constituent pixels of the image sensor are masked, and the smear reference used as reference data at the time of smear correction created based on the pixel output information of the light shielding unit, and the light shielding. The defective pixel information in the unit and dark output characteristic information of each constituent pixel are stored in the storage unit. Then, based on the defective pixel information and dark output characteristic information stored in the storage unit, the image processing unit performs smear reference correction that excludes the influence of the defective pixel and dark output characteristic on the smear reference from the smear reference. Done.

  According to a second aspect of the present invention, in the measurement imaging apparatus according to the first aspect, the image processing unit corrects a variation in the dark output characteristic in the smear reference as the smear reference correction. According to this configuration, the variation in the dark output characteristic in the smear reference is corrected by the image processing unit as the smear reference correction.

  According to a third aspect of the present invention, there is provided the measurement imaging apparatus according to the first or second aspect, wherein the defective pixel information includes address information of a defective pixel, and the image processing unit includes address information of the defective pixel. Based on this, smear reference correction is performed in which the pixel value of the defective pixel is interpolated using the pixel values of the peripheral pixels of the defective pixel to exclude the influence of the defective pixel from the smear reference. According to this configuration, the image processing unit interpolates the pixel value of the defective pixel using the pixel values of the peripheral pixels of the defective pixel on the basis of the address information of the defective pixel, and thereby affects the influence of the defective pixel from the smear reference. The smear reference correction to be excluded is performed.

  According to a fourth aspect of the present invention, there is provided the measurement imaging apparatus according to any one of the first to third aspects, wherein the light shielding unit includes a predetermined number of pixel lines perpendicular to the vertical transfer CCD unit in the imaging sensor, and the image processing unit includes: A pixel line obtained by performing the smear reference correction by subtracting the variation in the dark output characteristics from the output data of the pixel line as a smear reference and interpolating the pixel value of the defective pixel, and obtaining the smear reference correction Is subtracted from the output data of each effective pixel line parallel to the pixel line in the imaging effective area of the imaging sensor, and smear correction is performed on the captured image.

  According to this configuration, the light shielding unit includes a predetermined number of pixel lines perpendicular to the vertical transfer CCD unit in the image sensor, and the image processing unit causes variations in dark output characteristics with respect to output data of the pixel line as a smear reference. The smear reference correction is performed by the process of subtracting and the process of interpolating the pixel value of the defective pixel, and the output data of the pixel line obtained by the smear reference correction is the pixel line of the light shielding unit in the imaging effective area of the imaging sensor. By subtracting from the output data of each parallel effective pixel line, smear correction is performed on the captured image.

  A measurement imaging apparatus according to a fifth aspect of the present invention is the measurement imaging apparatus according to any one of the first to fourth aspects, wherein the measurement imaging apparatus is a two-dimensional color luminance meter that measures two-dimensional color and luminance. According to this configuration, the measurement imaging apparatus is a two-dimensional color luminance meter that measures two-dimensional color and luminance.

  According to the measurement imaging apparatus according to claim 1, since the smear reference correction that excludes the influence on the smear reference due to the defective pixel and the dark output characteristic from the smear reference is performed, the accuracy of the smear reference itself can be improved. Smear correction using this can also be performed with high accuracy, and as a result, a measurement image with high accuracy (high image quality) can be obtained.

  According to the measurement imaging apparatus of the second aspect, as the smear reference correction, since the variation in the dark output characteristics in the smear reference is corrected, the pixel line output data (smear reference) in the light-shielding portion usually has It is possible to obtain a smear reference from which the influence of output variations between pixels is eliminated, and to obtain a highly accurate smear reference.

  According to the measurement imaging apparatus of the third aspect, since the pixel value of the defective pixel is interpolated using the pixel values of the peripheral pixels of the defective pixel, the pixel values of the peripheral pixels are averaged, for example, By an easy method (calculation) such as interpolation of pixel values, the influence of defective pixels on smear reference can be excluded.

  According to the measurement imaging apparatus of the fourth aspect, the process of subtracting the variation in the dark output characteristics and the process of interpolating the pixel value of the defective pixel from the output data (smear reference) of the pixel line in the light shielding unit are performed. Therefore, the accuracy of the smear reference itself can be increased, and the smear correction is performed by subtracting the output data of the pixel line whose accuracy has been improved by this correction from each effective pixel line of the imaging effective region. Smear correction can be performed with high accuracy.

  According to the measurement imaging apparatus of the fifth aspect, since the measurement imaging apparatus is a two-dimensional color luminance meter, smear correction on the measurement image (captured image) obtained in the measurement of saturation and luminance is highly accurate. Can be done.

  FIG. 1 is a schematic block diagram illustrating an example of a measurement imaging apparatus 1 according to the present embodiment. As shown in FIG. 1, the measurement imaging apparatus 1 includes an imaging sensor 2, an A / D conversion unit 3, a timing generation unit 4, an image processing unit 5, a control unit 6, a monitor unit 7, an operation unit 8, an image memory 9, and An external interface unit 10 is provided.

  The imaging sensor 2 is a solid-state imaging device that captures subject light (subject brightness), that is, a device that photoelectrically converts incident subject light into an image signal corresponding to the amount of light, and outputs a photodiode (PD). A CCD image sensor as a so-called two-dimensional sensor in which photoelectric conversion elements such as) are arranged in a matrix. In front of the image sensor 2, an optical lens system (not shown) for imaging subject light and a rotating disk (not shown) in which a plurality of color filters (for example, R, G, B) are arranged. ing. An image signal of each color is obtained by rotating the rotating disk, selecting a color filter, and forming an image of the subject light on the image sensor 2 through each color filter. The CCD image sensor may be a so-called Bayer type color area sensor in which RGB color filters are attached in a checkered pattern on the surface of the two-dimensionally arranged CCD. In this case, a rotating disk on which color filters are arranged is not necessary.

  Specifically, for example, as shown in FIG. 2, the imaging sensor 2 performs vertical transfer of a photoelectric conversion unit 21 (light receiving unit) formed of a photodiode array and a signal charge photoelectrically converted by the photoelectric conversion unit 21 and read and transferred. The vertical transfer unit 22 (vertical transfer path, vertical CCD) and the signal charge transferred by the vertical transfer unit 22 are horizontally applied to a signal output unit (amplifier unit for detecting charges) disposed in the subsequent stage. A horizontal transfer unit 23 (horizontal transfer path, horizontal CCD) for transfer is provided. Here, the imaging sensor 2 has the PD 212 of the photoelectric conversion unit 21 and the vertical CCD of the vertical transfer unit 22 alternately arranged for each column, and each end of the vertical CCD is connected to each element of the horizontal CCD. Is a so-called interline transfer type CCD image sensor having a structure arranged in a comb shape.

  Specifically, a CCD that performs each transfer of signal charges is provided with a number of electrodes on an oxide film on the surface of a silicon substrate, and a potential well (quantum well; Well) is created and can be used to hold charges. By appropriately controlling the voltage applied to each electrode, the charge of each element is transferred simultaneously to the adjacent elements. Thereby, the charge for each pixel 24 held by each element can be sequentially taken out in a bucket relay manner.

  The imaging sensor 2 includes an OB (optical black) unit 25. The OB unit 25 is optically shielded from light so that photoelectric conversion is not performed by covering (masking) with a light-shielding film made of a metal thin film such as aluminum (Al) on the imaging sensor surface 20 of the imaging sensor 2. It is an area. In other words, it can be said that the OB unit 25 is a CCD having a structure in which light is not irradiated. The light shielding film is not limited to a metal thin film, and may be shielded by using, for example, a film-shaped light shielding member, or using the light shielding member and a metal thin film in combination. When outputting the image data, it is possible to remove noise components such as dark current by outputting the data of the photoelectric conversion unit 21 with the image data of the OB unit 25 as a reference. The OB portion 25 is provided on one end side in the horizontal transfer direction on the imaging sensor surface 20 (see FIG. 3). Specifically, in the end side portion along the horizontal transfer unit 23 on the imaging sensor surface 20, the photoelectric conversion unit 21 and one end of the vertical transfer unit 22 (PD 212 and vertical transfer unit 221) are covered in a band shape. Provided (see FIG. 2). Similarly to the OB unit 25, each transfer CCD, that is, the vertical transfer unit 22 and the horizontal transfer unit 23 is also shielded by a light shielding film made of a metal thin film such as aluminum so that photoelectric conversion is not performed. However, in actuality, the light is not completely shielded from light and a smear phenomenon or the like may occur. The light shielding or smear phenomenon will be described in detail later.

  A portion other than the OB portion 25 of the imaging sensor surface 20 is an imaging effective area 26 (imaging area). Here, a pixel corresponding to the pixel 24 in the imaging effective area 26 is referred to as an effective pixel 261, and a pixel corresponding to the pixel 24 in the OB portion 25 is referred to as an OB pixel 251. In addition, each pixel line arranged in the horizontal transfer direction (direction perpendicular to the vertical transfer unit 22) in the OB unit 25 is referred to as an OB line. According to this, in FIG. 3, for example, as shown in the enlarged view indicated by reference numeral 220 for the region indicated by reference numeral 210, the OB unit 25 is composed of four OB lines indicated by arrows of reference numeral 221. Although it is shown to be composed of two OB lines indicated by the respective arrows of reference numeral 222, both illustrate the concept and are not limited to this, and correspond to each measurement imaging device 1. An OB unit 25 having a suitable number of OB lines is set. Further, the arrangement position of the OB portion 25 may not be the lower end side in FIG. 3, for example, the upper end side.

  The A / D converter 3 converts the analog image signal output from the image sensor 2 into, for example, a 12-bit digital image signal (digital image data). The A / D conversion unit 3 converts an analog image signal into a digital image signal based on an A / D conversion clock signal input from a timing generation unit 4 described later.

  The timing generator (timing generator) 4 generates a predetermined timing pulse (timing signal), and controls the photographing operation by the image sensor 2 and the processing operation of the A / D converter 3 by this timing pulse. For example, the timing generation unit 4 generates timing pulses such as a pixel drive signal and vertical / horizontal synchronization signals based on the imaging control signal from the control unit 6 and outputs the timing pulses to the imaging sensor 2. The charge is accumulated in conjunction with (photographing the subject light into an image signal), and image data is captured from the image sensor 2. In addition, an A / D conversion clock (timing pulse) is generated and output to the A / D conversion unit 3, and the captured image data is sequentially passed through the A / D conversion unit 3 to the image processing unit 5. Output.

  The control unit 6 includes a ROM that stores each control program, a RAM that temporarily stores various data, a central processing unit (CPU) that reads and executes the control program from the ROM, and the like. It is responsible for overall operation control. The control unit 6 calculates control parameters and the like required by each unit of the device based on various signals from each unit of the device such as the imaging sensor 2 and the operation unit 8, and controls the operation of each unit by transmitting this.

  The monitor unit 7 displays an image captured by the image sensor 2, that is, an image processed by the image processing unit 5, an image stored in the image memory 9 (for example, an image subjected to smear correction), or the like. A liquid crystal display (LCD) comprising a liquid crystal display element. The operation unit 8 is used to input operation instructions to the measurement imaging apparatus 1 by a user. For example, various operation switches (operation buttons) such as a power switch, an imaging switch, a mode setting switch for setting each imaging mode, and a menu selection switch. ).

  The external interface unit 10 transmits / receives data to / from an external device 100 such as a PC. For example, a wired (network such as a LAN) such as a USB (Universal Serial Bus), an IrDA (Infrared Data Association), or a wireless A data transmission / reception device according to a communication standard. Note that an arrow indicated by a double line between the functional blocks in FIG. 1 indicates a path through which image data is transmitted, and an arrow indicated by a single line indicates a path through which a control signal is transmitted.

  The image processing unit 5 performs various types of image processing (digital signal processing) on the digital image signal transmitted from the image sensor 2 and obtained by the A / D conversion processing by the A / D conversion unit 3. A smear correction process and a smear reference correction process are performed. Specifically, the image processing unit 5 is composed of a field-programmable gate array (FPGA), which is a device capable of freely combining arithmetic logic by a program, for example, and capable of high-speed arithmetic processing. However, it is not limited to an FPGA as long as it can perform image data arithmetic processing, and may be an arithmetic processing device such as a CPU described later.

  Here, the smear generated in the interline transfer type CCD image sensor and the correction method when the smear occurs will be described in detail. First, a main factor (mechanism) in which smear occurs will be described using a schematic diagram in which a part of the imaging surface shown in FIG. 6 is enlarged. As described above, smear occurs when light leaking into the vertical transfer unit is photoelectrically converted to generate unnecessary charges, and the unnecessary generated charges are mainly the following (1) to (3). Is mentioned.

(1): Charge generated by light transmitted through the light shielding film of the transfer unit In this case, the light beam 224 that has passed through the light shielding film 223 of the vertical transfer unit 22 (vertical CCD; here expressed as VCCD) is photoelectrically converted in the VCCD. A charge is generated. This is because even though the light shielding film 223 is made of metal, its thickness is thin (since it is an Al thin film), so that strong light is transmitted. Note that since the VCCD is a kind of photoelectric conversion element, when it receives light, it is photoelectrically converted to generate electric charge. The case (1) is the largest cause of smear.
(2): Charge generated by sneak light from the edge of the transfer portion light shielding film In this case, the light beam 225 obliquely incident from the outside of the VCCD light shielding film 223 reaches the VCCD and is photoelectrically converted in the VCCD. A charge is generated.
(3): Charges generated in a place other than the PD part of the regular photoelectric conversion region, for example, the energy quantum transfer barrier part (insensitive area) indicated by reference numeral 2261. In this case, depending on the incident angle of the light ray 226, In the highlight irradiation part, the generated charge outside the quantum well located outside the light shielding film 223 of the VCCD increases, and this generated charge (illegal charge) is generated by straying into the VCCD.

  However, the effects of the above (2) and (3) are, for example, those having a CCD structure in which all except the regular photoelectric conversion part (PD part) are shielded by the light shielding film 227 as shown in FIG. Almost negligible. In FIG. 7, a microlens 229 is disposed in the opening 228 above each PD formed in the light shielding film 227, and the opening 228 is narrowed by a configuration in which the microlens 229 efficiently collects light. Thus, the light shielding area is made larger. FIG. 6 schematically shows a pixel and a transfer unit in a CCD, and in practice, a partition is configured in terms of energy by a potential distribution.

  The charges generated by the above factors are highlighted when the effective charge transferred from the PD to the VCCD (charge obtained by the regular photoelectric conversion unit; captured image data) is vertically transferred in the VCCD. Is superimposed on the effective charge in the VCCD of the portion (superimposed when passing through the highlight irradiation portion). The amount of unwanted charge, ie, smear component, superimposed on the effective charge is proportional to the light intensity of the highlight irradiation part and the exposure time during which the effective charge during vertical transfer is exposed to the highlight. The exposure time is determined by the vertical transfer rate (clock rate) and the area of the highlight irradiation part (the length of the highlight irradiation part in the VCCD direction). As described above, smear is caused by the cases of (1) to (3) above. The charge generated in the high luminance portion of the subject has already been subjected to photoelectric conversion by the normal photoelectric conversion unit, and the VCCD It is generated by mixing with the signal charge being transferred.

  When smear occurs as described above, the smear correction process is performed by the image processing unit 5. Here, the image information of the OB unit 25 when the smear appears is used for the smear correction. That is, if a captured image at the time of appearance of a certain smear is obtained by the imaging sensor 2, the captured image is composed of an image of the imaging effective area 26 on the imaging sensor surface 20 and an image of the OB unit 25, and the smear component is These are included in the imaging effective area 26 and the OB unit 25 in the same manner. The image desired to be obtained by smear correction is an image of the effective imaging area 26 from which the smear component has been subtracted, but as an image having a smear component to be subtracted from the effective imaging area 26, the OB portion that does not include subject image information 25 images are used. Note that data serving as a reference (for subtracting the smear component) upon smear correction, here, output data of the OB unit 25 (an OB line output 311 described later) is hereinafter referred to as a smear reference as appropriate.

  FIG. 4 is a graph illustrating output characteristics of the OB unit 25 when smear occurs. The graph indicated by reference numeral 310 in the figure is the output characteristics (OB line output 311) of the OB line as actual data obtained when smear appears, and the graphs indicated by reference numerals 320 and 330 are the OB line output for explanation. 311 is separated into a smear component (smear component 321) and an OB line dark output characteristic (OB line dark output 331). The horizontal axis of each graph indicates the pixel horizontal address in the OB line direction, and the vertical axis indicates the output charge. Each graph shows an example and is not limited to this.

  The OB line output 311 is an output for one line in the OB unit 25, that is, one OB among a plurality of OB lines constituting the OB unit 25, for example, four OB lines of the OB unit 25 shown in FIG. The output of the line (each OB pixel constituting this OB line) is shown and is output data as the smear reference. The smear component 321 is a smear component (data for one pixel line) included in the OB line output 311. The peak position of the smear component 321 is not limited to this, and is a position corresponding to a location (pixel horizontal address) where smear occurs. Further, the number (waveform) of the peaks 3211 of the smear component 321 is not limited to this, and corresponds to the case where smear occurs at a plurality of locations.

  The OB line dark output 331 indicates the output of the OB unit 25 when smear does not occur. Actually, the output characteristic of each OB pixel of the OB unit 25 has variation (inter-pixel variation), and therefore, for example, the output is represented by the variation characteristic 332. In addition, defective pixels may exist in each OB pixel, and as a result, the output becomes a large value or a small value that does not fall within the range of the variation, specifically, for example, an abnormally high output such as so-called “whiteout”. A defective pixel output 333 (a white image with no gradation) or a defective pixel output 334 having no output such as so-called “blackout” (or blackout) (even a dark output, for example, zero output) is output. Sometimes. The black spots are caused by the reason that the gate of the OB pixel is broken or dust is attached on the PD. In addition, there may be a plurality of defective pixel outputs 333 and 334, respectively, or only one of them may exist. In some cases, the defective pixel outputs 333 and 334 do not exist.

  When the smear correction is performed, ideally, only the smear component 321 may be removed from the captured image, that is, the image of the effective imaging area 26. However, the OB unit 25 actually includes the smear component 321 and the smear component 321. Since the OB line output 311 that is synthesized (superimposed) with the OB line dark output 331 is output, the above-described variation characteristics 332 and defective pixels are obtained from the actual OB line output 311 so as to approach the ideal smear component 321. Smear correction may be performed on the captured image using data obtained by subtracting error components (OB line dark output 331) such as outputs 333 and 334. On the other hand, if the OB line output 311 is used as it is for smear correction and an operation for excluding the smear component from the image of the imaging effective area 26 is performed, the image of the imaging effective area 26 is affected by the error component. It will be.

  From the above, the image processing unit 5 performs smear reference correction by performing arithmetic processing (subtraction processing) for removing (excluding) the OB line dark output 331 from the OB line output 311. Specifically, this smear reference correction is a process of subtracting the variation characteristic 332 from the OB line output 311, and interpolation with respect to the defective pixel outputs 333 and 334 (such that the defective pixel outputs 333 and 334 become appropriate values). Correction). The arithmetic processing for removing the OB line dark output 331 (the above error component) from the OB line output 311 is called error component removal processing.

For example, as shown in FIG. 5, if the pixel 2B is a defective pixel as shown in FIG. 5, the pixel value of an actual pixel (adjacent pixel) adjacent to the pixel 2B is used. A pixel value of the pixel 2B is calculated based on an interpolation formula such as an average calculation shown in A) to (C). However, each symbol in the equation indicates the pixel value of the corresponding pixel. The symbol “/” indicates division.
(A) 2B = (2A + 2C) / 2
(B) 2B = (1B + 2A + 2C + 3B) / 4
(C) 2B = (1A + 1B + 1C + 2A + 2C + 3A + 3B + 3C) / 8

  The image processing unit 5 performs the pixel interpolation processing based on the defective pixel address information of the OB unit 25 corresponding to the pixel 2B stored in the correction data memory 93, and the defective pixel obtained by this interpolation processing is processed. The pixel value is stored in the defective pixel address in the reference memory 92 described later. Similarly, with respect to the correction of variation, an arithmetic process for excluding variations between pixels is performed based on OB dark data (variation characteristics 332) stored in a correction data memory 93 described later. The obtained image data is stored in the reference memory 92.

  In addition, when the OB unit 25 includes a plurality of OB lines as shown in the present embodiment, the image processing unit 5 is averaged by performing a predetermined averaging process on the plurality of OB lines. Data for one line (average OB line data) is calculated, and this data is redefined as smear reference data (referred to as representative smear reference) representing the OB unit 25, that is, a plurality of OB lines, and will be described later. To store. The averaging process may be an arithmetic average or a geometric average. Further, the arithmetic processing for obtaining a representative OB line from a plurality of OB lines may not be the above average processing, and any arithmetic processing can be adopted. After calculating the representative smear reference, the image processing unit 5 subtracts the representative smear reference data for each line, that is, for example, for each horizontal pixel line parallel to the horizontal transfer unit 23 in FIG. The smear correction is performed by performing the calculation.

  The image memory 9 is composed of a memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The image memory 9 is RAW image data before image processing by the image processing unit 5, or the image processing unit 5 or the control unit 6. This is a so-called buffer for storing image data or the like during or after various processes. The image memory 9 includes a frame memory 91, a reference memory 92, and a correction data memory 93. The frame memory 91 stores all pixel data including a captured image (frame image) obtained by the imaging sensor 2, that is, image data (referred to as effective pixel data) of the imaging effective area 26 and image data (OB pixel data) of the OB unit 25. Store. The frame memory 91 also stores captured images that have been smear corrected.

  The reference memory 92 stores reference data for smear correction (the above smear reference). However, this smear reference is OB line output data (correction) obtained by performing the above error component removal processing on the image (each OB line output 311) of the OB unit 25 in all pixel data stored in the frame memory 91. (Referred to as OB line output), or the representative smear reference data obtained by averaging a plurality of corrected OB line outputs.

  The correction data memory 93 stores data (correction data) for correcting the smear reference. Here, the correction data is the address information of the defective pixel described above, and dark output data (referred to as OB dark data) of each pixel constituting the OB unit 25. This OB dark data specifically corresponds to the variation characteristic 332. The defective pixel address information and OB dark data are given as default values. This predetermined value is given, for example, as output data obtained by measuring in advance in a state where light is shielded for each image sensor 2 (each pixel) (a state where there is no smear effect). The defective pixel address information and OB dark data are not for one line of data shown in the OB line dark output 331 but for all pixel data (all OB lines) in the OB section 25. . The defective pixel address information is not limited to the OB portion 25 but may be pixel address information for all pixel data.

  The frame memory 91 is configured to store all pixel data of a captured image. For example, the image data of the OB unit 25 used as a smear reference is separated from all pixel data, and the remaining imaging is performed. Only the image data of the effective area 26 may be stored in the frame memory 91, and the image data of the OB unit 25 may be stored in the reference memory 92 (even in this configuration, the OB unit 25 is similar to the above. The smear reference correction process or the subsequent smear correction process is performed on the image data). In this case, when the image data is read from the image sensor 2, the image data of the OB unit 25 and the image data of the imaging effective area 26 are separated based on the pixel address, and the frame memory 91 and the reference memory 92 are respectively separated. And may be stored. In this case, the image memory of the OB unit 25 may be separately provided in the reference memory 92. The memories 91 to 93 are connected via a predetermined BUS (image BUS).

  FIG. 8 is a flowchart illustrating an example of an operation related to smear correction in the measurement imaging apparatus 1 according to the present embodiment. First, a captured image (frame image) is acquired by the imaging sensor 2 (assuming that smear has occurred in the captured image). Specifically, charge accumulation by photoelectric conversion in the PD is started (for example, the reset operation of the previous accumulated charge is performed at the start of accumulation) (step S1), and then the accumulated charge in the PD is transferred to the vertical transfer unit. The image pickup operation is terminated by transferring the image to the image sensor 22 (step S2), and the vertical transfer unit 22 performs the vertical transfer of the electric charge and the horizontal transfer unit 23 performs the horizontal transfer alternately so that the image pickup sensor 2 is serially processed. All pixel data (captured image) is read out from A, and after A / D conversion processing is performed by the A / D converter 3, it is stored in the frame memory 91 (step S3). Then, the smear reference is corrected based on the image of the OB unit 25 of the photographed image stored in the frame memory 91 (step S4), and the smear reference (representative smear reference) is used to correct the smear reference. Is corrected (step S5). Note that an image obtained by the smear correction (referred to as a smear corrected image) is transferred by the control unit 6 to the external device 100 via the external interface unit 10 (external output) so that the external device 100 performs a predetermined process. (Step S6).

  FIG. 9 is a flowchart showing an example of the operation relating to smear reference correction in step S4. In step S <b> 4, the image processing unit 5 performs processing for removing the OB line dark output 331 from each OB line output 311 of the OB unit 25. That is, the variation characteristic 332 (OB dark data) corresponding to each OB line output 311 is subtracted from each OB line output 311 to perform OB dark correction processing (step S41), and the OB unit 25 ( Interpolation processing of defective pixels in each OB line output 311) is performed (step S42). Then, a representative smear reference is created by, for example, averaging the plurality of corrected OB line outputs that have been subjected to OB dark correction and defective pixel interpolation error component removal processing (step S43), and this representative smear reference data Is stored in the reference memory 92 (step S44).

  FIG. 10 is a flowchart showing an example of the operation related to OB dark correction in step S41. In step S41, the image processing unit 5 causes the image data (OB image data; each OB line output) from the frame memory 91 to the OB unit 25 to vary from the correction data memory 93 to each OB line (when OB is dark). Data) is read (step S411), and the corresponding variation characteristic is subtracted from each OB line output (step S412). Then, if each OB line output (corrected OB line output) from which the variation characteristic is subtracted is a smear reference, that is, the data of each OB line of the OB unit 25 read from the frame memory 91 is a smear reference, an error component corresponding thereto The data is stored in the reference memory 92 as data constituting the corrected smear reference that has been corrected by the removal process (step S413).

  FIG. 11 is a flowchart showing an example of the operation related to the OB portion defective pixel interpolation in step S42. In step S42, the image processing unit 5 reads the image data (each OB line output) of the OB unit 25 from the frame memory 91 and the address information of the defective pixel in the OB unit 25 from the correction data memory 93 ( In step S421), based on the defective pixel address information, the pixel value of the pixel having the defective pixel address is interpolated from the pixel values of the adjacent pixels by an average operation or the like (step S422). Then, the pixel value (or corrected OB line output) of the defective pixel obtained by this interpolation is stored in the reference memory 92 as data constituting the corrected smear reference (step S423).

  FIG. 12 is a flowchart showing an example of the operation relating to smear correction in step S5. In step S5, the image processing unit 5 reads the image data of the imaging effective area 26 stored in the frame memory 91 for each pixel line (one effective pixel line) (step S51). After the representative smear reference created in step S4 is subtracted from the image data for one pixel line (step S52), the image data for one pixel line after the subtraction of the representative smear reference is stored in the frame memory 91. (Step S53). If the image processing unit 5 determines that the subtraction processing for all the pixel lines of the effective pixels has been completed (YES in step S54), the flow ends, and the subtraction processing for all the lines of effective pixels ends. If it is determined that it is not (NO in step S54), the process returns to step S51 and is repeated until the end of all lines. The smear correction execution timing is not limited to that shown in the above flow. For example, when the image is transferred from the frame memory 91 to the external device 100, the smear correction is performed while transferring the pixel in units of one pixel line (representing the representative smear reference). The timing at which the subtraction process is performed may be used.

  As shown in the flowchart of FIG. 12, in this embodiment, smear correction of effective pixel data is performed by subtracting a smear reference (representative smear reference) from effective pixel data in the imaging effective area 26. However, strictly speaking, the dark output characteristics of the CCD vary slightly from pixel to pixel, and the dark output characteristics of each pixel in the imaging effective area 26 are defined by the dark output of the pixels of the OB unit 25. It cannot be used, but it can be used as an approximate value. Therefore, by applying a smear reference (representative smear reference) obtained from the image data of the OB unit 25 as smear correction, the dark correction of the imaging effective area 26 can be performed at the same time.

  As described above, according to the measurement imaging apparatus 1 of the present embodiment, the OB unit 25 is an optical light shielding region in which the constituent pixels of the imaging sensor 2 are masked, and based on the pixel output information of the OB unit 25. The generated smear reference used as the reference data for smear correction, the defective pixel information in the OB unit 25, and the dark output characteristic information of each component pixel are stored in the image memory 9. Then, the image processing unit 5 removes the influence of the defective pixel and the dark output characteristic on the smear reference from the smear reference based on the defective pixel information and the dark output characteristic information stored in the image memory 9. Correction is performed. According to this, since the smear reference correction is performed to exclude the influence on the smear reference due to the defective pixel and the dark output characteristic from the smear reference, the accuracy of the smear reference itself can be improved, and the smear correction using this is also high. Therefore, it is possible to obtain a measurement image with high accuracy (high image quality).

  In addition, since the image processing unit 5 corrects the variation in the dark output characteristic in the smear reference as the smear reference correction, the output data (smear reference) of the OB line (pixel line) in the OB unit 25 is usually included. Thus, it is possible to obtain a smear reference from which the influence of the output variation (variation characteristic 332) between each pixel is removed, and it is possible to obtain a highly accurate smear reference.

  Further, the image processing unit 5 interpolates the pixel value of the defective pixel using the pixel value of the peripheral pixel (adjacent pixel) of the defective pixel on the basis of the address information of the defective pixel. Since the smear reference correction is excluded, the influence of the defective pixel on the smear reference is excluded by an easy method (calculation), for example, by averaging the pixel values of the peripheral pixels and interpolating the pixel values of the defective pixels. be able to.

  The OB unit 25 includes a predetermined number of OB lines perpendicular to the vertical transfer unit 22 in the image sensor 2, and the image processing unit 5 subtracts the variation in dark output characteristics from the output data of the OB line as a smear reference. The smear reference correction is performed by the process of performing and the process of interpolating the pixel value of the defective pixel, and the output data of the OB line obtained by the smear reference correction is the OB of the OB unit 25 in the imaging effective area 26 of the imaging sensor 2. By subtracting from the output data of each effective pixel line parallel to the line, smear correction is performed on the captured image. As described above, since the process of subtracting the variation in the dark output characteristic from the output data (smear reference) of the OB line in the OB unit 25 and the process of interpolating the pixel value of the defective pixel are performed, the accuracy of the smear reference itself is determined. In addition, since the smear correction is performed by subtracting the output data of the OB line whose accuracy is improved by this correction from each effective pixel line of the imaging effective area 26, the smear correction is performed with high accuracy. It can be.

  Furthermore, since the measurement imaging apparatus 1 is a two-dimensional color luminance meter that measures two-dimensional color and luminance, smear correction on a measurement image (captured image) obtained in the measurement of saturation and luminance is highly accurate. Can be done.

In addition, this invention can take the following aspects.
(A) In the above-described embodiment, smear reference correction processing and smear correction processing for a captured image are performed in the measurement imaging device 1 (image processing unit 5 and control unit 6). It is good also as a structure performed in the predetermined process part outside the imaging device 1 for a measurement. Specifically, after all the captured image data including the image data of the OB unit 25 is read from the imaging sensor 2, the image data is not buffered in the image memory 9, and the external device is connected via the external interface unit 10. 100 (see FIG. 1), and the external device 100 may perform various correction processes similar to those in the above-described embodiment by performing arithmetic processing using, for example, predetermined software.

  (B) In the above embodiment, for example, as shown in the flowchart of FIG. 12, correction is performed for each line, that is, as a calculation method for subtracting one representative smear reference (OB line) from one effective pixel line. However, instead of such a line-to-line calculation, a composite image composed of representative smear references that are repeatedly arranged for the number of lines corresponding to all effective pixel lines in the effective imaging area 26, for example, calculation between surfaces. The calculation method may be such that image (surface) data in which representative smear references are repeatedly arranged is subtracted at once from the image (surface) data including all effective pixel lines.

  (C) Instead of the variation characteristic 332 shown in FIG. 4, the variation characteristic 332 and the variation characteristic having the opposite negative and positive characteristics are stored in the correction data memory 93 or the like, and when correcting the variation relative to the smear reference, The inverse characteristic variation characteristic may be added to or multiplied by the OB line output 311.

It is a schematic block diagram showing an example of a measurement imaging device according to the present embodiment. It is an enlarged view explaining the example of 1 structure of the light-receiving surface of the image sensor used for the said imaging device for a measurement. It is a schematic diagram explaining an example of the whole structure of the light-receiving surface of the said imaging sensor. It is a graph explaining the output characteristic of the OB part when smear occurs. It is a schematic diagram explaining the interpolation process of the defective pixel of a smear reference. It is a perspective view explaining the main factors (mechanism) which a smear generate | occur | produces. It is a perspective view explaining CCD structure light-shielded by the light shielding film altogether except a regular photoelectric conversion part (PD part). It is a flowchart which shows an example of the operation | movement regarding a smear correction | amendment in the imaging device for a measurement which concerns on this embodiment. It is a flowchart which shows an example of the operation | movement regarding smear reference correction | amendment in step S4 of the flowchart shown in FIG. 10 is a flowchart showing an example of an operation related to OB dark correction in step S41 of the flowchart shown in FIG. It is a flowchart which shows an example of the operation | movement regarding OB part defect pixel interpolation in step S42 of the flowchart shown in FIG. It is a flowchart which shows an example of the operation | movement regarding the smear correction | amendment in step S5 of the flowchart shown in FIG. It is a schematic diagram explaining generation | occurrence | production of the smear in the past.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device for measurement 2 Imaging sensor 5 Image processing part 9 Image memory (storage part)
91 Frame memory 92 Reference memory 93 Correction data memory 22 Vertical transfer unit (vertical transfer CCD unit)
223, 227 Light shielding film 25 OB part (light shielding part)
26 Imaging effective area 311 OB line output 321 Smear component 332 Dispersion characteristics (variation)
333, 334 Defective pixel output (defective pixel information)

Claims (5)

A measurement imaging apparatus including an imaging sensor that captures a predetermined measurement object using a CCD, and configured to be capable of correcting smear generated in an image captured by the imaging sensor,
A light-shielding portion as an optical light-shielding region formed by masking the constituent pixels of the imaging sensor;
A storage unit that stores smear reference used as reference data at the time of smear correction created based on pixel output information of the light shielding unit, and defective pixel information in the light shielding unit and dark output characteristic information of each constituent pixel;
An image processing unit that performs smear reference correction that excludes the influence of the defective pixel and the dark output characteristic on the smear reference from the smear reference based on the defective pixel information and the dark output characteristic information stored in the storage unit; An imaging device for measurement, comprising:   The measurement image pickup apparatus according to claim 1, wherein the image processing unit corrects a variation in the dark output characteristic in the smear reference as the smear reference correction. The defective pixel information includes address information of defective pixels,
The image processing unit, based on the address information of the defective pixel, interpolates the pixel value of the defective pixel using the pixel values of the peripheral pixels of the defective pixel, and thereby removes the influence of the defective pixel from the smear reference. The measurement imaging apparatus according to claim 1, wherein reference correction is performed. The light shielding portion is composed of a predetermined number of pixel lines perpendicular to the vertical transfer CCD portion in the image sensor,
The image processing unit performs the smear reference correction by a process of subtracting a variation in the dark output characteristics from the output data of the pixel line as a smear reference and a process of interpolating a pixel value of a defective pixel, and the smear reference correction The smear correction is performed on the captured image by subtracting the output data of the pixel line obtained by the above from output data of each effective pixel line parallel to the pixel line in the imaging effective area of the imaging sensor. The measurement imaging apparatus according to any one of 1 to 3.   The measurement imaging apparatus according to claim 1, wherein the measurement imaging apparatus is a two-dimensional color luminance meter that measures a two-dimensional color and luminance.

Images