JP2010212814A - Calibration device, calibration method, calibration program, computer-readable recording medium, and image input device - Google Patents

Abstract

A calibration device that unifies variations in pixel values output from a plurality of image input devices.
In a calibration device, Target_Min and Target_Min are assigned as pixel values for the dark light intensity and sensor saturation pixel value of the image input device 100a, and the dark light intensity sensor pixel value and the bright light intensity sensor of the image input device 100a are assigned. Based on the pixel value, sensor saturation pixel value, Target_Min, and Target_Min, the calculation unit 11 that calculates Target_Mid as the pixel value for the bright light intensity of the image input device 100a, and the dark light intensity as the pixel value after calibration And a mapping unit 13 for assigning Target_Min and Target_Mid to the bright light intensity. Therefore, the calibration device 50 can unify variations in pixel values output from the image input devices 100a to 100c.
[Selection] Figure 1

Description

  The present invention relates to a calibration device, a calibration method, a calibration program, and a calibration program for unifying variations in pixel values output from a plurality of image sensors by calibrating pixel values output from an image sensor. The present invention relates to a computer-readable recording medium on which the image is recorded and an image input apparatus including the calibration device.

  In recent years, small information terminals such as mobile phones and PDCs are equipped with an image sensor that reads an image on a panel surface. And an image sensor displays an image with the input of an image by mounting the drive circuit and optical sensor element which drive a liquid crystal in the same display panel. Further, the image sensor provides the user with an image display unit as a touch panel by processing the input image.

  Here, an optical sensor element such as a CCD (Charge Coupled Device) is used for image input (imaging). However, since the output characteristics of the optical sensor element are different for each individual, the output variation also occurs in the image sensor accordingly. The variation causes a problem that when a plurality of image sensors capture the same image, images of different quality are captured. Therefore, in order to solve the problem, it is necessary to correct and unify variations in pixel values output from a plurality of image sensors.

  Patent Documents 1 and 2 disclose calibration methods for optical sensor elements.

JP 61-53868 (Released on March 17, 1986) JP 2004-93894 A (publication date March 25, 2004)

  However, the calibration methods described in Patent Documents 1 and 2 are methods for calibrating output characteristics for each optical sensor element in order to capture a high-quality image, and are pixels output from a plurality of image sensors. It is not related to a method for unifying the dispersion of values.

  Therefore, in this method, it is impossible to unify the dispersion of pixel values output from a plurality of image sensors, and uniform image quality can be obtained when the same image is input between image sensors having different sensitivities. I can't.

  Further, the output variation for each image sensor is actually adjusted manually by a professional quality control staff at the time of product shipment. However, since image sensors vary greatly from one production lot to another, this method requires a great deal of time and effort on the operator.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a calibration apparatus, a calibration method, a calibration program, and a calibration program for unifying variations in pixel values output from a plurality of image sensors. Another object of the present invention is to provide a computer-readable recording medium in which a recording program is recorded, and an image input device including such a calibration device.

  The calibration device according to the present invention is a calibration device that unifies pixel values output from a plurality of image sensors to the same pixel value for the same light intensity in order to solve the above-described problem, Among the plurality of image sensors, an image sensor to be calibrated is a target image sensor, a representatively selected image sensor is a reference image sensor, a light intensity under dark conditions is a first light intensity, and the first light. When two light intensities higher than the intensities are set as the second and third light intensities in order of increasing light intensity, the first assigned pixel value is used as the pixel value for the first and third light intensities of the reference image sensor. , And a first assigning means for assigning a third assigned pixel value larger than the first assigned pixel value, respectively, and the reference image for the first to third light intensities. Based on the first to third reference pixel values, which are pixel values of the sensor, and the first and third assigned pixel values, a pseudo pixel value is calculated as a pixel value for the second light intensity of the reference image sensor. Calculating means; and second assigning means for assigning the first assigned pixel value and the pseudo control pixel value to the first and second light intensities as pixel values after calibration of the target image sensor, respectively. It is characterized by providing.

  The calibration method according to the present invention is a calibration method for unifying pixel values output from a plurality of image sensors to the same pixel value for the same light intensity in order to solve the above-described problem. Among the plurality of image sensors, the image sensor to be calibrated is the target image sensor, the representatively selected image sensor is the reference image sensor, the light intensity under dark conditions is the first light intensity, Assuming that two light intensities higher than one light intensity are the second and third light intensities in order of increasing light intensity, the first assignment is made as a pixel value for the first and third light intensities of the reference image sensor. A first assigning step for assigning a pixel value and a third assigned pixel value larger than the first assigned pixel value, and the first to third light intensities, Based on the first to third reference pixel values, which are pixel values of the quasi-image sensor, and the first and third assigned pixel values, a pseudo pixel value as a pixel value for the second light intensity of the reference image sensor is obtained. A calculation step for calculating, and a second allocation for allocating the first allocated pixel value and the pseudo-controlled pixel value to the first and second light intensities as pixel values after calibration of the target image sensor, respectively. And a step.

  According to the above configuration, in the calibration device according to the present invention, the calculation means includes the first to third reference pixel values that are the pixel values of the reference image sensor for the first to third light intensities, and the first assignment means. Based on the first and third assigned pixel values assigned by, the pseudo pixel value is calculated as the pixel value for the second light intensity of the reference image sensor.

  That is, by determining the reference image sensor, the pseudo-pixel value is calculated based on the first to third reference pixel values and the first and third assigned pixel values, so regardless of the target image sensor, The pseudo pixel value can be acquired based only on the reference image sensor.

  Then, the second assigning unit assigns the first assigned pixel value and the pseudo control pixel value to the first and second light intensities as the pixel values after the calibration of the target image sensor.

  Accordingly, the first assigned pixel value is assigned by the first assigning means, and the pseudo pixel value is calculated when the reference image sensor is determined. Therefore, when the target image sensor is calibrated, the characteristics of each target image sensor are changed. There is no need to consider. That is, there is no need to individually set / change parameters for each target image sensor.

  As described above, in the calibration apparatus and the calibration method according to the present invention, the target image sensor is calibrated based on the reference image sensor that is typically selected, so that time and labor required for calibration are greatly reduced. However, variations in pixel values output from a plurality of image sensors can be unified.

  The calibration device according to the present invention is a calibration device that unifies pixel values output from a plurality of image sensors to the same pixel value for the same light intensity in order to solve the above-described problem, Among the plurality of image sensors, an image sensor to be calibrated is a target image sensor, a representatively selected image sensor is a reference image sensor, a light intensity under dark conditions is a first light intensity, and the first light. When the two light intensities higher than the intensities are the second and third light intensities in order of increasing light intensity, the first to third pixel values of the reference image sensor with respect to the first to third light intensities. Based on a reference pixel value and first and second target pixel values that are pixel values of the target image sensor for the first and second light intensities, Calculating means for calculating a pseudo-pixel value as a pixel value of the target image sensor; a pixel value after calibration of the target image sensor; a first assigned pixel value with respect to the first and third light intensities; and Assigning means for setting a third assigned pixel value larger than the first assigned pixel value, assigning the first target pixel value to the first assigned pixel value, and assigning the pseudo control pixel value to the third assigned pixel value. It is characterized by providing these.

  According to the above configuration, in the calibration apparatus according to the present invention, the calculation means includes the first to third reference pixel values that are the pixel values of the reference image sensor for the first to third light intensities, and the first and third reference pixel values. Based on the first and second target pixel values that are the pixel values of the target image sensor for the two light intensities, the pixel value of the target image sensor for the third light intensity is calculated as a pseudo pixel value.

  Therefore, since the pseudo-pixel value is calculated as a value obtained by taking in the first and second target pixel values of the target image sensor, the variation of the first and second target pixel values existing for each target image sensor is included. Calculated. That is, a pseudo pixel value including output characteristics for each target image sensor is calculated.

  Furthermore, in the calibration device according to the present invention, the assigning means uses the first target pixel value as the first assigned pixel value and the pseudo-pixel value as the third pixel value after the calibration of the target image sensor. Assign to each assigned pixel value. Therefore, variations in pixel values output from a plurality of image sensors can be unified.

  In addition, in the calibration apparatus according to the present invention, since a reference image sensor is defined as a representative image sensor, it is not necessary to individually change parameters for each target image sensor, and the time and labor required for calibration are greatly reduced. Can be reduced.

  As described above, in the calibration apparatus according to the present invention, since the target image sensor is calibrated based on the reference image sensor that is typically selected, a plurality of times are reduced while greatly reducing the time and labor required for calibration. Variation of pixel values output from the image sensor can be unified.

  In the calibration apparatus according to the present invention, it is preferable that the calculation means calculates the pseudo-pixel value according to equation (A).

y = {x2 * (Z1-y1) + z2 * (y1-x1)} / (z1-x1) (A)
However,
y: pseudo pixel value of the reference image sensor x1: first reference pixel value of the reference image sensor y1: second reference pixel value of the reference image sensor z1: third reference pixel value of the reference image sensor x2: first of the reference image sensor Assigned pixel value z2: The third assigned pixel value of the reference image sensor.

  That is, in the calibration device according to the present invention, the pseudo pixel value is calculated using the first to third reference pixel values related to the reference image sensor and the first and third assigned pixel values of the reference image sensor. ).

And the basic equation for deriving equation (A) is:
(Z1-y1) :( y1-x1) = (z2-y) :( y-x2)
Indicated by.

  That is, since the parameter of the target image sensor is not included in the formula (A), it is not necessary to consider the characteristics of each target image sensor when any image sensor is the target light panel.

  Therefore, it is not necessary to individually set and change parameters for each target image sensor, and it is possible to unify variations in pixel values output from a plurality of image sensors while greatly reducing the time and labor required for calibration. it can.

  In the calibration device according to the present invention, it is preferable that the calculation means calculates the pseudo-pixel value according to equation (B).

z = {x2 * (y1-Z1) + y2 * (z1-x1)} / (y1-x1) (B)
However,
z: pseudo control pixel value of target image sensor x1: first reference pixel value of reference image sensor y1: second reference pixel value of reference image sensor z1: third reference pixel value of reference image sensor x2: first of target image sensor Target pixel value y2: The second target pixel value of the target image sensor.

  That is, in the calibration device according to the present invention, the pseudo pixel value is expressed by the formula (B) using the first to third reference pixel values related to the reference image sensor and the first and second target pixel values of the target image sensor. ).

And the basic equation for deriving equation (B) is:
(Z1-y1) :( y1-x1) = (z-y2) :( y2-x2)
Indicated by.

  Therefore, the calibration device according to the present invention can calculate a pseudo pixel value including variation of the first and second target pixel values existing for each target image sensor, and based on the pseudo pixel value, The target image sensor can be calibrated.

  In the calibration apparatus according to the present invention, in the average output characteristic curve showing the relationship between the intensity of light incident on the image sensor and the pixel value output from the image sensor with respect to the intensity of the light, When the light intensity that is the boundary between the light intensity region in which the pixel value changes correlating with the change and the light intensity region higher than that is the saturated light intensity, the third light intensity is the reference image. The saturation light intensity of the sensor is preferable.

  By setting the saturation light intensity of the reference image sensor to the third light intensity, the darkest light intensity (first light intensity) to the brightest light intensity (third light intensity) in the linear region of the average output characteristic curve of the reference image sensor. Can be set as the calibration target light intensity range. Therefore, calibration in a wider light intensity range is possible, and a more suitable calibration device can be provided to the user.

  In addition, when the reference image sensor selected representatively is the image sensor having the highest sensitivity among the plurality of image sensors, all the image sensors except the reference image sensor, that is, the reference image sensor, are not sensitive. Calibration can be accurately performed on the image sensor. In addition, the reference image sensor itself can be a calibration target. As a result, the yield can be remarkably improved when manufacturing the image sensor.

  However, it may be difficult in practice to extract the image sensor having the highest sensitivity from all the image sensors. Therefore, in such a case, it is preferable to extract an image sensor with higher sensitivity and use the image sensor as a reference image sensor. Thereby, the yield can be improved when manufacturing the image sensor.

  In the calibration device according to the present invention, it is preferable that the first and third assigned pixel values are arbitrarily determined.

  Since the first and third assigned pixel values can be arbitrarily determined, the range can be determined according to the use state of the image sensor, and the user can perform flexible calibration according to the use state and purpose of use. Yes.

  The calibration apparatus may be realized by a computer. In this case, a calibration program that causes the calibration apparatus to be realized by a computer by operating the computer as each of the above-described means, and a program recorded therein Computer-readable recording media are also within the scope of the present invention.

  The calibration apparatus according to the present invention can also be applied to an image input apparatus.

  As described above, the calibration device according to the present invention provides the first assigned pixel value and the third assigned pixel value larger than the first assigned pixel value as the pixel values for the first and third light intensities of the reference image sensor. A first allocating unit for allocating allocated pixel values; first to third reference pixel values that are pixel values of the reference image sensor for the first to third light intensities; and the first and third allocated pixel values. And calculating means for calculating a pseudo pixel value as a pixel value for the second light intensity of the reference image sensor, and the first and second light intensities as pixel values after calibration of the target image sensor. On the other hand, a second assigning means for assigning the first assigned pixel value and the pseudo control pixel value, respectively.

  In addition, as described above, the calibration apparatus according to the present invention includes the first and third reference pixel values that are pixel values of the reference image sensor with respect to the first to third light intensities, and the first and second. Calculation means for calculating a pseudo pixel value as a pixel value of the target image sensor for the third light intensity based on the first and second target pixel values which are pixel values of the target image sensor for light intensity; As a pixel value after calibration of the target image sensor, a first assigned pixel value and a third assigned pixel value larger than the first assigned pixel value are set for the first and third light intensities, Allocating means for allocating the first target pixel value to the first allocated pixel value and the pseudo control pixel value to the third allocated pixel value, respectively.

  In the calibration method according to the present invention, as described above, the pixel value for the first and third light intensities of the reference image sensor is larger than the first assigned pixel value and the first assigned pixel value. A first assigning step for assigning third assigned pixel values, first to third reference pixel values which are pixel values of the reference image sensor for the first to third light intensities, and the first and third assignments. Calculating a pseudo pixel value as a pixel value for the second light intensity of the reference image sensor based on a pixel value; and the first and second pixel values after calibration of the target image sensor And a second assigning step for assigning the first assigned pixel value and the pseudo control pixel value to the light intensity, respectively.

  Therefore, the calibration device and the calibration method according to the present invention can easily unify variations in pixel values output from a plurality of image sensors.

It is a block diagram which shows the principal part structure of the calibration apparatus which concerns on this Embodiment. It is a figure which shows typically the cross section of a liquid crystal panel with a built-in sensor. FIG. 3A is a schematic diagram showing a state in which a position touched by the user is detected by detecting a reflected image with a sensor built-in liquid crystal panel included in the image input apparatus. FIG. 3B is a schematic diagram illustrating a state in which a position touched by the user is detected by detecting a shadow image with a sensor built-in liquid crystal panel included in the image input apparatus. It is a block diagram which shows the structure of the liquid crystal panel with a sensor with which an image input device is provided, and the structure of its peripheral circuit. It is a block diagram which shows the structure of a liquid crystal panel with a built-in sensor, and the structure of a peripheral circuit. It is a figure which shows the average output characteristic curve which showed the relationship between the light intensity which injects into a some photosensor element, and the average pixel value output from the said photosensor element in a general image input device. It is a figure which shows each average output characteristic curve in case a some image input device exists. It is a figure for demonstrating a bright light intensity sensor pixel value and a dark light intensity sensor pixel value, (a) is a figure for demonstrating the bright light intensity sensor pixel value of an image input device, (b) is an image input device. It is a figure for demonstrating the dark light intensity sensor pixel value of. It is a figure for demonstrating the pseudo sensor saturated pixel value of an image input device. It is a figure which shows a mode that a (pseudo) sensor saturation pixel value is each assigned to Target_Max, and a dark light intensity sensor pixel value is assigned to Target_Min. It is a figure which shows the relationship of ad shown by Formula (1). It is a flow for demonstrating the operation | movement of a calibration. It is a figure which shows the relationship of ad shown by Formula (4). It is a figure which shows a mode that the bright light intensity sensor pixel value of the image input device made into the object of calibration is each assigned to Target_Mid, and a dark light intensity sensor pixel value is assigned to Target_Min. It is a flow for demonstrating operation | movement of the other calibration apparatus which concerns on this invention.

Hereinafter, the present embodiment will be described with reference to the drawings. In describing the calibration device 50 according to the present embodiment, the structure of the image input device 100 including the calibration device 50 will be described first, and then the calibration device 50 will be described.
(About the image input device 100)
Examples of the image input device 100 including the calibration device 50 include a display with a built-in optical sensor having both functions of image display and image input. Hereinafter, the display with a built-in optical sensor will be described with reference to FIGS.

  The image input device 100 includes a sensor built-in liquid crystal panel 301 capable of detecting an image of an object in addition to displaying data. Here, the image detection of the object is, for example, detection of a position pointed (touched) by the user with a finger or a pen, or reading (scanning) of an image of a printed material or the like. The device used for display is not limited to a liquid crystal panel, and may be an organic EL (Electro Luminescence) panel or the like.

  The structure of the sensor built-in liquid crystal panel 301 will be described with reference to FIG. FIG. 2 is a diagram schematically showing a cross section of the sensor built-in liquid crystal panel 301. The sensor built-in liquid crystal panel 301 described here is an example, and any structure can be used as long as the display surface and the reading surface are shared.

  As shown in the figure, the sensor built-in liquid crystal panel 301 includes an active matrix substrate 51A disposed on the back surface side and a counter substrate 51B disposed on the front surface side, and has a structure in which a liquid crystal layer 52 is sandwiched between these substrates. is doing. The active matrix substrate 51A is provided with a pixel electrode 56, a data signal line 57, an optical sensor circuit 32 (not shown), an alignment film 58, a polarizing plate 59, and the like. The counter substrate 51B is provided with color filters 53r (red), 53g (green), 53b (blue), a light shielding film 54, a counter electrode 55, an alignment film 58, a polarizing plate 59, and the like. In addition, a backlight 307 is provided on the back surface of the sensor built-in liquid crystal panel 301.

  When the sensor built-in liquid crystal panel 301 is formed of CG silicon liquid crystal (continuous grain boundary crystal silicon liquid crystal), the photosensor element 6 of the photosensor circuit 32 has high blue light receiving sensitivity. Therefore, the photosensor element 6 is provided in the vicinity of the pixel electrode 56 provided with the blue color filter 53b. However, the optical sensor element 6 may be arranged on sub-pixels of other colors, or may be arranged on all sub-pixels so that the imaging resolution is higher than the display resolution.

  Next, with reference to FIGS. 3A and 3B, two types of methods for detecting the position where the user touches the sensor built-in liquid crystal panel 301 with a finger or a pen will be described.

  FIG. 3A is a schematic diagram illustrating a state in which a position touched by the user is detected by detecting a reflected image. When the light 63 is emitted from the backlight 307, the optical sensor circuit 32 including the optical sensor element 6 detects the light 63 reflected by the object 64 such as a finger. Thereby, the reflected image of the target object 64 can be detected. Thus, the sensor built-in liquid crystal panel 301 can detect the touched position by detecting the reflected image.

  FIG. 3B is a schematic diagram illustrating a state in which a position touched by the user is detected by detecting a shadow image. As shown in FIG. 3B, the optical sensor circuit 32 including the optical sensor element 6 detects external light 61 transmitted through the counter substrate 51B and the like. However, when there is an object 62 such as a pen, the incident of the external light 61 is hindered, so that the amount of light detected by the optical sensor circuit 32 is reduced. Thereby, a shadow image of the object 62 can be detected. Thus, the sensor built-in liquid crystal panel 301 can also detect a touched position by detecting a shadow image.

  As described above, the optical sensor element 6 may detect reflected light (shadow image) of the light emitted from the backlight 307, or may detect a shadow image caused by external light. Further, the two types of detection methods may be used in combination to detect both a shadow image and a reflected image at the same time.

  Next, the configuration of the main part of the image input apparatus 100 will be described with reference to FIG. FIG. 4 is a block diagram illustrating a main configuration of the image input apparatus 100. As illustrated, the image input device 100 includes one or more display / light sensor units 300, a circuit control unit 600, a data processing unit 700, a main control unit 800, a storage unit 901, a primary storage unit 902, an operation unit 903, An external communication unit 907, an audio output unit 908, and an audio input unit 909 are provided. Here, the image input apparatus 100 will be described as including two display / light sensor units 300 (first display / light sensor unit 300A and second display / light sensor unit 300B). When the first display / light sensor unit 300A and the second display / light sensor unit 300B are not distinguished, they are referred to as the display / light sensor unit 300.

  The display / light sensor unit 300 is a so-called liquid crystal display device with a built-in light sensor. The display / light sensor unit 300 includes a sensor built-in liquid crystal panel 301, a backlight 307, and a peripheral circuit 309 for driving them.

  The sensor built-in liquid crystal panel 301 includes a plurality of pixel circuits 31 and photosensor circuits 32 arranged in a matrix. The detailed configuration of the sensor built-in liquid crystal panel 301 will be described later.

  The peripheral circuit 309 includes a liquid crystal panel drive circuit 304, an optical sensor drive circuit 305, a signal conversion circuit 306, and a backlight drive circuit 308.

  The liquid crystal panel driving circuit 304 outputs a control signal (G) and a data signal (S) in accordance with the timing control signal (TC1) and the data signal (D) from the display control unit 601 of the circuit control unit 600, and the pixel circuit 31. It is a circuit which drives. Details of the driving method of the pixel circuit 31 will be described later.

  The optical sensor driving circuit 305 is a circuit that drives the optical sensor circuit 32 by applying a voltage to the signal line (R) in accordance with a timing control signal (TC2) from the sensor control unit 602 of the circuit control unit 600. Details of the driving method of the optical sensor circuit 32 will be described later.

  The signal conversion circuit 306 is a circuit that converts the sensor output signal (SS) output from the optical sensor circuit 32 into a digital signal (DS) and transmits the converted signal to the sensor control unit 602.

  The backlight 307 includes a plurality of white LEDs (Light Emitting Diodes) and is disposed on the back surface of the sensor built-in liquid crystal panel 301. When a power supply voltage is applied from the backlight drive circuit 308, the backlight 307 is turned on and irradiates the sensor built-in liquid crystal panel 301 with light. Note that the backlight 307 is not limited to white LEDs, and may include LEDs of other colors. The backlight 307 may include, for example, a cold cathode fluorescent lamp (CCFL) instead of the LED.

  The backlight driving circuit 308 applies a power supply voltage to the backlight 307 when the control signal (BK) from the backlight control unit 603 of the circuit control unit 600 is at a high level, and conversely, the backlight control unit 603. When the control signal from is at a low level, no power supply voltage is applied to the backlight 307.

  Next, the circuit control unit 600 will be described. The circuit control unit 600 has a function as a device driver that controls the peripheral circuit 309 of the display / light sensor unit 300. The circuit control unit 600 includes a display control unit 601, a sensor control unit 602, a backlight control unit 603, and a display data storage unit 604.

  The display control unit 601 receives display data from the display data processing unit 701 of the data processing unit 700, and performs timing control on the liquid crystal panel driving circuit 304 of the display / light sensor unit 300 in accordance with an instruction from the display data processing unit 701. A signal (TC1) and a data signal (D) are transmitted, and the received display data is displayed on the sensor built-in liquid crystal panel 301.

  The display control unit 601 temporarily stores the display data received from the display data processing unit 701 in the display data storage unit 604. Then, a data signal (D) is generated based on the primary stored display data. The display data storage unit 604 is, for example, a video random access memory (VRAM).

  The sensor control unit 602 transmits a timing control signal (TC2) to the optical sensor driving circuit 305 of the display / optical sensor unit 300 in accordance with an instruction from the sensor data processing unit 703 of the data processing unit 700, and the sensor built-in liquid crystal panel 301. Run the scan with.

  In addition, the sensor control unit 602 receives a digital signal (DS) from the signal conversion circuit 306. Then, image data is generated based on the digital signal (DS) corresponding to the sensor output signal (SS) output from all the optical sensor circuits 32 included in the sensor built-in liquid crystal panel 301. That is, the image data read in the entire reading area of the sensor built-in liquid crystal panel 301 is generated. Then, the generated image data is transmitted to the sensor data processing unit 703.

  The backlight control unit 603 transmits a control signal (BK) to the backlight drive circuit 308 of the display / light sensor unit 300 in accordance with instructions from the display data processing unit 701 and the sensor data processing unit 703 to drive the backlight 307. Let

  When the image input apparatus 100 includes a plurality of display / light sensor units 300, the display control unit 601 instructs the display / light sensor unit 300 to display the display data from the data processing unit 700. When received, the liquid crystal panel drive circuit 304 of the display / light sensor unit 300 is controlled according to the instruction. When the sensor control unit 602 receives an instruction from the data processing unit 700 as to which display / light sensor unit 300 is to scan the object, The sensor drive circuit 305 is controlled and a digital signal (DS) is received from the signal conversion circuit 306 of the display / light sensor unit 300 according to the instruction.

  Next, the data processing unit 700 will be described. The data processing unit 700 has a function as middleware that gives an instruction to the circuit control unit 600 based on a command received from the main control unit 800.

  The data processing unit 700 includes a display data processing unit 701 and a sensor data processing unit 703. When the data processing unit 700 receives a command from the main control unit 800, at least one of the display data processing unit 701 and the sensor data processing unit 703 operates according to the value of each field included in the received command. .

  The display data processing unit 701 receives display data from the main control unit 800, and gives instructions to the display control unit 601 and the backlight control unit 603 according to the command received by the data processing unit 700, and displays the received display data. The image is displayed on the sensor built-in liquid crystal panel 301. The operation of the display data processing unit 701 according to the command will be described later.

  The sensor data processing unit 703 gives an instruction to the sensor control unit 602 and the backlight control unit 603 according to the command received by the data processing unit 700.

  The sensor data processing unit 703 receives image data from the sensor control unit 602 and stores the image data in the image data buffer 704 as it is.

  Next, the main control unit 800 executes an application program. The main control unit 800 reads the program stored in the storage unit 901 into a primary storage unit 902 configured by, for example, a RAM (Random Access Memory) and executes the program.

  An application program executed by the main control unit 800 causes the data processing unit 700 to display display data on the sensor built-in liquid crystal panel 301 or to scan an object on the sensor built-in liquid crystal panel 301. Send commands and display data. When “data type” is designated as a command, at least one of whole image data, partial image data, and coordinate data is received from the data processing unit 700 as a response to the command.

  The circuit control unit 600, the data processing unit 700, and the main control unit 800 can be configured by a CPU (Central Processing Unit), a memory, and the like, respectively. The data processing unit 700 may be configured by a circuit such as an ASIC (application specific integrate circuit).

  Next, the storage unit 901 stores programs and data executed by the main control unit 800 as shown in the figure. The program executed by the main control unit 800 may be separated into an application-specific program and a general-purpose program that can be shared by each application.

  Next, the operation unit 903 receives an input operation of the user of the image input apparatus 100. The operation unit 903 includes input devices such as a switch, a remote controller, a mouse, and a keyboard, for example. Then, the operation unit 903 generates a control signal corresponding to the input operation of the user of the image input apparatus 100 and transmits the generated control signal to the main control unit 800.

In addition, the image input device 100 includes an external communication unit 907 for communicating with an external device by wireless / wired communication, an audio output unit 908 such as a speaker for outputting audio, a microphone for inputting an audio signal, and the like. A voice input unit 909 or the like may be provided as appropriate.
(Configuration of sensor built-in liquid crystal panel)
Next, the configuration of the sensor built-in liquid crystal panel 301 and the configuration of the peripheral circuit 309 of the sensor built-in liquid crystal panel 301 will be described with reference to FIG. FIG. 5 is a block diagram showing the main part of the display / light sensor unit 300, particularly the configuration of the sensor built-in liquid crystal panel 301 and the configuration of the peripheral circuit 309.

  The sensor built-in liquid crystal panel 301 includes a pixel circuit 31 for setting light transmittance (brightness) and an optical sensor circuit 32 that outputs a voltage corresponding to the intensity of light received by the sensor. The pixel circuit 31 generically uses an R pixel circuit 31r, a G pixel circuit 31g, and a B pixel circuit 31b corresponding to the red, green, and blue color filters, respectively.

  The pixel circuits 31 are arranged on the sensor built-in liquid crystal panel 301 in the column direction (vertical direction) and 3n in the row direction (horizontal direction). A set of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b is continuously arranged in the row direction (lateral direction). This set forms one pixel.

  In order to set the light transmittance of the pixel circuit 31, first, the high level voltage (TFT 33 is turned on) to the scanning signal line Gi connected to the gate terminal of the TFT (Thin Film Transistor) 33 included in the pixel circuit 31. Voltage). Thereafter, a predetermined voltage is applied to the data signal line SRj connected to the source terminal of the TFT 33 of the R pixel circuit 31r. Similarly, the light transmittance is also set for the G pixel circuit 31g and the B pixel circuit 31b. Then, by setting these light transmittances, an image is displayed on the sensor built-in liquid crystal panel 301.

  Next, the photosensor circuit 32 is arranged for each pixel. One pixel may be arranged in the vicinity of each of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b.

  In order for the optical sensor circuit 32 to output a voltage corresponding to the light intensity, first, the sensor readout line RWi connected to one electrode of the capacitor 35 and the anode terminal of the photodiode 36 are connected. A predetermined voltage is applied to the sensor reset line RSi. In this state, when light is incident on the photodiode 36, a current corresponding to the amount of incident light flows through the photodiode 36. Then, according to the current, the voltage at the connection point (hereinafter referred to as connection node V) between the other electrode of the capacitor 35 and the cathode terminal of the photodiode 36 decreases. When the power supply voltage VDD is applied to the voltage application line SDj connected to the drain terminal of the sensor preamplifier 37, the voltage at the connection node V is amplified and output from the source terminal of the sensor preamplifier 37 to the sensing data output line SPj. Based on the output voltage, the amount of light received by the optical sensor circuit 32 can be calculated.

  Next, the liquid crystal panel drive circuit 304, the optical sensor drive circuit 305, and the sensor output amplifier 44, which are peripheral circuits of the sensor built-in liquid crystal panel 301, will be described.

  The liquid crystal panel drive circuit 304 is a circuit for driving the pixel circuit 31, and includes a scanning signal line drive circuit 3041 and a data signal line drive circuit 3042.

  The scanning signal line driving circuit 3041 sequentially selects one scanning signal line from the scanning signal lines G1 to Gm for each line time based on the timing control signal TC1 received from the display control unit 601, and A high level voltage is applied to the selected scanning signal line, and a low level voltage is applied to the other scanning signal lines.

  Based on the display data D (DR, DG, and DB) received from the display controller 601, the data signal line driver circuit 3042 generates a predetermined voltage corresponding to the display data for one row for each line time. The data signal lines SR1 to SRn, SG1 to SGn, and SB1 to SBn are applied (line sequential method). Note that the data signal line driver circuit 3042 may be driven by a dot sequential method.

  The optical sensor driving circuit 305 is a circuit for driving the optical sensor circuit 32. Based on the timing control signal TC2 received from the sensor control unit 602, the optical sensor driving circuit 305 selects a predetermined sensor readout signal line from the sensor readout signal lines RW1 to RWm for each line time. A read voltage is applied, and a voltage other than a predetermined read voltage is applied to the other sensor read signal lines. Similarly, based on the timing control signal TC2, a predetermined reset voltage is applied to the sensor reset signal line selected from the sensor reset signal lines RS1 to RSm for each line time, and the others. A voltage other than a predetermined reset voltage is applied to the sensor reset signal line.

The sensing data output signal lines SP1 to SPn are grouped into p groups (p is an integer of 1 to n), and the sensing data output signal lines belonging to each group are connected via a switch 47 that is sequentially turned on in time division. And connected to the sensor output amplifier 44. The sensor output amplifier 44 amplifies the voltage from the group of sensing data output signal lines connected by the switch 47 and outputs the amplified voltage to the signal conversion circuit 306 as sensor output signals SS (SS1 to SSp).
(Regarding the calibration device 50)
The structure of the image input device 100 has been described above with reference to FIGS. Next, the calibration device 50 will be described with reference to FIGS. 1 and 6 to 12. In the present embodiment, some terms are used to describe the configuration and operation of the calibration device 50. Therefore, for convenience of explanation, terms are first explained, and then the details of the calibration device 50 are explained.
<Average output characteristic curve>
The average output characteristic curve will be described with reference to FIG. FIG. 6 shows an average output characteristic curve showing the relationship between the light intensity incident on a plurality of photosensor elements (imaging elements) and the average pixel value output from the photosensor elements in a general image input device. FIG.

  The average output characteristic curve has a linear region where the pixel value changes with a correlation with a change in light intensity, and a saturation region which is a light intensity region higher than that. In the saturation region, the pixel value eventually converges to a constant value as the light intensity increases.

  More specific description will be given with reference to FIG. The average output characteristic curve is a light intensity range (linear range) in which the pixel value changes with a correlation with a change in light intensity in a range from light intensity (dark light intensity) B to light intensity W under dark conditions. Have Further, in the light intensity region that is larger than the light intensity W, the pixel value does not change with a correlation with the change in the light intensity, and eventually has a region (saturation region) where the pixel value converges. That is, the average output characteristic curve is divided into a linear region and a saturation region with the light intensity W as a boundary.

  In the present embodiment, the light intensity W that is the boundary between the linear region and the saturated region is referred to as saturated light intensity. That is, the saturation light intensity can be said to be the limit light intensity at which the average output characteristic curve can maintain linearity. In FIG. 6, Vwm represents the pixel value with respect to the saturation light intensity, and Vsm represents the convergence value of the pixel value in the saturation region.

  Here, the saturated light intensity W is calculated as follows. Generally, a so-called coefficient of determination or R-square value is used as a linearity evaluation index. The R square value is a value obtained by squaring the Pearson product moment correlation coefficient R, and the Pearson product moment correlation coefficient is calculated by the following [Equation 1].

However, in [Equation 1]
x, y: set of random variables x _i , y _i : sample values of x and y x _m , y _m : average value of sample values n: number of samples

  In [Equation 1], the R-square value is 1 when the distribution of x and y is completely a straight line. On the other hand, the R-square value approaches 0 as it deviates from the straight line. Since the R-square value has such characteristics, it is used as an index of linearity. If the value is about 0.99, it is determined that the linearity is good.

The saturated light intensity W is calculated using the above characteristic of the R square value. Specifically, the output of a light source such as a fluorescent lamp or LED illumination, or the output when the distance between the light source and the image input device is changed and the light intensity of the image input device is increased from the low side to the high side. The saturated light intensity is determined from the graph in which the pixel values are plotted. That is, a pixel value having an R square value of 0.99 is determined, and the light intensity at that time is set as the saturated light intensity. The R-square value is not necessarily 0.99, and may be selected as appropriate.
<Reference image input device>
Next, the reference image input device will be described. First, in order to describe the reference image input device, each average output characteristic curve when there are a plurality of image input devices 100 is shown in FIG. FIG. 7 shows an example in which there are three image input devices 100 (image input devices 100a to 100c), and the image input devices 100a to 100c have average output characteristic curves 101a to 101c, respectively.

  In this embodiment, a typical image input device is referred to as a reference image input device (reference image sensor). Here, “representative” means that an image input device arbitrarily extracted from a plurality of image input devices serves as a reference (representative) when performing calibration, and is based on some criteria in particular. Thus, a representative image input device is not determined. Therefore, in the present embodiment, any one of the image input devices 100a to 100c may be used as the reference image input device.

  Therefore, for convenience of explanation, in the present embodiment, the image input device 100a is representatively selected from the image input devices 100a to 100c, and the image input device 100a is assumed to be a reference image input device. Of course, there is no problem even if the image input devices 100b and 100c are the reference image input devices.

  As shown in FIG. 7, the following characteristics are observed in the image input device 100a arbitrarily extracted as a typical image input device. Specifically, when the saturated light intensity acquired based on the average output characteristic curves 101a to 101c is set to Wa to Wc in that order, the image input apparatus 100a has the saturated light intensity Wa related to the image input apparatus 100a. Lowest. When the pixel values corresponding to the saturated light intensities Wa to Wc are Vwa to Vwc, respectively, the pixel values Vwa to Vwc are higher in the order of the saturated light intensities Wa to Wc. Therefore, the gradient of the linear region related to the average output characteristic curves 101a to 101c increases in the order of the image input devices 100a to 100c corresponding to the saturated light intensities Wa to Wc. This indicates that the sensitivity of the image sensor is higher in the order of the image input devices 100a to 100c.

Here, the number of the image input devices 100 is not necessarily limited to three, and may be two or four or more. However, in the present embodiment, the following description will be given on the assumption that the number of image input devices is three (image input devices 100a to 100c).
<Bright light intensity sensor pixel value, dark light intensity sensor pixel value>
Next, the bright light intensity sensor pixel value and the dark light intensity sensor pixel value will be described with reference to FIG. FIG. 8A is a diagram for explaining the bright light intensity sensor pixel value of the image input device 100, and FIG. 8B is a diagram for explaining the dark light intensity sensor pixel value of the image input device 100. It is.

  As shown in FIG. 8A, a box capable of irradiating light with a certain brightness is put on the image input apparatus 100. In this embodiment, the average value of the sensor pixel values output in this state is referred to as the bright light intensity sensor pixel value of the image input device 100. The light intensity at this time is referred to as bright light intensity (second light intensity).

  Similarly, as shown in FIG. 8B, the image input apparatus 100 is covered with a box capable of realizing the darkness that is the same condition as the dark room. In this embodiment, the average value of the sensor pixel values output in this state is referred to as the dark light intensity sensor pixel value of the image input device 100. The light intensity at this time is referred to as dark light intensity (first light intensity).

The average value referred to here represents an average value of pixel values output by a plurality of photosensor elements (a plurality of image sensors constituting the image sensor) according to the image input apparatus 100.
<Pseudo sensor saturation pixel value>
Next, the pseudo sensor saturation pixel value (pseudo control pixel value) will be described with reference to FIG. FIG. 9 is a diagram for explaining the pseudo sensor saturation pixel value of the image input apparatus 100.

  As described above, the reference image input device refers to an image input device typically selected from a plurality of image input devices. In this embodiment, the saturation light intensity of the reference image input device is calculated, and the pixel value of the other image input device corresponding to the saturation light intensity is referred to as a pseudo sensor saturation pixel value.

Referring to FIG. 9 as an example, the saturation light intensity of the image input device 100a that is the reference image input device among the image input devices 100a to 100c is Wa. Therefore, the pixel values of the other image input devices (image input devices 100b and 100c) with respect to the saturated light intensity Wa become the pseudo sensor saturated pixel values. Note that the method for calculating the pseudo sensor saturation pixel value follows formulas (1) to (3) described later, and details thereof will be described later.
<Mapping (Target_Max, Target_Min)>
Finally, mapping will be described with reference to FIG. FIG. 10 is a diagram illustrating a state in which a (pseudo) sensor saturation pixel value is assigned to Target_Max (third assigned pixel value) and a dark light intensity sensor pixel value is assigned to Target_Min (first assigned pixel value).

  As shown in FIG. 10A, the average output characteristic curves 101a to 101c of the image input devices 100a to 100c have different curve shapes. Therefore, in this state, the same pixel value cannot be obtained when the image input devices 100a to 100c are irradiated with light having the same light intensity. Therefore, Target_Max and Target_Min are set in order to unify the pixel values output from the image input devices 100a to 100c. Then, the calibration device 50 assigns the pseudo sensor saturation pixel value to Target_Max and assigns the dark light intensity sensor pixel value to Target_Min.

In brief, as shown in FIG. 10B, for an arbitrary image input device, the pseudo sensor saturation pixel value is assigned to a predetermined Target_Max, and the dark light intensity sensor pixel value is assigned to a predetermined Target_Min. It is done. Therefore, even when any one of the image input devices 100a to 100c is irradiated with light, the respective pseudo sensor saturation pixel values and dark light intensity sensor pixel values become Target_Max and Target_Min. As a result, when the image input devices 100a to 100c are irradiated with light having the same light intensity, the same pixel value can be obtained. Details thereof will be described later. Further, although there is a mapping in a method different from the mapping described above, the details will be described in [Other Embodiments] described later.
(Main components of the calibration device 50)
Next, the calibration device 50 will be described with reference to FIG. FIG. 1 is a block diagram showing a main configuration of the calibration apparatus 50. As shown in FIG. The calibration device 50 may be included in the signal conversion circuit 306 described above or may be provided in the image input device 100 as a new configuration. If the configuration described below is provided, the configuration can be arbitrarily selected.

  In the following description, the calibration device 50 is described as being provided in each of the image input devices 100a to 100c. Further, the description will be made assuming that the reference image input device is the image input device 100a (see FIG. 7) and the calibration target is the image input device 100b (target image sensor).

  The calibration device 50 includes a calculation unit 11 (calculation unit), a setting unit 12, and a mapping unit 13 (allocation unit). In FIG. 1, the recording unit 10 is disposed outside the calibration device 50, but the recording unit 10 is configured inside or outside the calibration device 50. May be.

  The recording unit 10 includes a bright light intensity sensor pixel value (second reference pixel value), a dark light intensity sensor pixel value (first reference pixel value), and a saturated light intensity related to the image input device 100a that is a reference image input device. The sensor saturation pixel value (third reference pixel value) in Wa (FIG. 7) is recorded. In addition, the recording unit 10 records the bright light intensity sensor pixel value (second target pixel value) and the dark light intensity sensor pixel value (first target pixel value) of the image input device 100b to be calibrated. Yes. The bright light intensity sensor pixel value and the dark light intensity sensor pixel value can be acquired for each image input device by the method described with reference to FIG.

  The calculation unit 11 calculates a pseudo sensor saturation pixel value of the image input device 100b. Specifically, the calculation unit 11 relates to the bright light intensity sensor pixel value, the dark light intensity sensor pixel value, and the sensor saturation pixel value related to the image input apparatus 100a which is the reference image input apparatus, and the image input apparatus 100b. The bright light intensity sensor pixel value and the dark light intensity sensor pixel value are read from the recording unit 10, and the pseudo sensor saturation pixel value of the image input device 100b is calculated based on these values.

  The pseudo sensor saturation pixel value is calculated according to the equation (3) derived from the equations (1) and (2). Here, for easy understanding of the equation (1), the relations a to d shown in the equation (1) are shown in FIG. As shown in FIG. 11 and Expression (1), the pseudo sensor saturation pixel values of the image input device 100b are respectively the image input device 100a that is the reference image input device and the image input device 100b that is the object of calibration. Is calculated based on a ratio calculation using a difference between the bright light intensity sensor pixel value, the dark light intensity sensor pixel value, and the (pseudo) sensor saturation pixel value.

a: b = c: d (1)
However,
a: Sensor saturation pixel value−bright light intensity sensor pixel value in the reference image input device (image input device 100a) b: Bright light intensity sensor pixel value−dark light intensity sensor pixel value in the reference image input device (image input device 100a) c: Pseudo sensor saturation pixel value−bright light intensity sensor pixel value in the image input apparatus 100b d: Bright light intensity sensor pixel value−dark light intensity sensor pixel value in the image input apparatus 100b (x1-y1): (y1-z1) = (X2-y2) :( y2-z2) (2)
x2 = {x1.y2- (x1-y1) .z2-y2z1} / (y1-z1) (3)
However,
x1: Sensor saturation pixel value in the reference image input device (image input device 100a) y1: Bright light intensity sensor pixel value in the reference image input device (image input device 100a) z1: In the reference image input device (image input device 100a) , Dark light intensity sensor pixel value x2: pseudo sensor saturation pixel value in the image input apparatus 100b y2: bright light intensity sensor pixel value in the image input apparatus 100b z2: dark light intensity sensor pixel value in the image input apparatus 100b

  That is, in the present embodiment, the calculation unit 11 calculates the pseudo sensor saturation pixel value of the image input device 100b so that the relational expression of a: b = c: d in Expression (1) holds.

  Equation (3) is shown in a form in which the pseudo sensor saturation pixel value (x2) is calculated for each image input device. However, in practice, the saturation pseudo sensor pixel value (x2) is calculated for each photosensor element of the image input device, and the average value thereof becomes the pseudo sensor saturation pixel value of the image input device. Therefore, when there is no variation in the pixel value output from the sensor element, Equation (3) may be used as it is as a calculation formula for the pseudo sensor saturation pixel value of the image input device, and the pixel value output from the sensor element If there is variation, the pseudo sensor saturation pixel value may be calculated for each sensor element by Equation (3), and the average value may be used as the pseudo sensor saturation pixel value of the image input device.

  Here, in reality, since the pixel value output from the sensor element varies, the pseudo sensor saturation pixel value is calculated by Equation (3) for each sensor element, and the average value is calculated as the pseudo value of the image input device. In many cases, the sensor saturated pixel value is used. In that case, by using the expression (3), variations in the bright light intensity sensor pixel value and the dark light intensity sensor pixel value existing in the image input device 100b (more specifically, a plurality of light sources related to the image input device 100b). Since the pseudo sensor saturation pixel value is calculated while including the variation of the pixel value output from the sensor element), calibration in accordance with the actual situation is performed.

  The setting unit 12 stores Target_Max and Target_Min as target values used by the mapping unit 13 for mapping as setting values. Specifically, Target_Max and Target_Min are input to the setting unit 12 via an input device (not shown) or the like, and the setting unit 12 stores these input values as setting values. Then, the setting unit 12 outputs Target_Max and Target_Min to the mapping unit 13 in response to reading from the mapping unit 13. Note that Target_Max and Target_Min may be recorded in the recording unit 10, and which structure is used may be appropriately selected.

  The mapping unit 13 acquires the pseudo sensor saturation pixel value of the image input device 100b from the calculation unit 11, and reads the dark light intensity sensor pixel value of the image input device 100b from the recording unit 10. Further, the mapping unit 13 reads Target_Max and Target_Min from the setting unit 12. Then, the mapping unit 13 assigns (maps) the pseudo sensor saturation pixel value of the image input device 100b to Target_Max and the dark light intensity sensor pixel value of the image input device 100b to Target_Min.

  Note that “assign” means assigning Target_Max to 255 and Target_Min to 0, for example, when the pixel value is represented by 0 to 255. By setting in this way, the pseudo sensor saturation pixel value of the image input device 100b is calibrated to 255, and the dark light intensity sensor pixel value of the image input device 100b is calibrated to 0. The pixel value to be assigned is not limited to 0 to 255, and can be selected as appropriate.

  The operation of the calibration process based on the above configuration will be described with reference to FIG. FIG. 12 is a flowchart for explaining the calibration operation.

  First, in step S10, the calibration device 50 acquires average output characteristic curves of a plurality of image input devices, and calculates a saturated light intensity at which the R-square value is 0.99 for each average output characteristic curve. In the present embodiment, saturated light intensities Wa, Wb, and Wc at which the square value is 0.99 are calculated for the average output characteristic curves 101a to 101c of the image input devices 100a to 100c. In addition, about the method by which the calibration apparatus 50 calculates R square value, you may think that the calculating part which is not shown in figure calculates with a general method.

  Next, in step S12, the calibration device 50 determines a representative image input device. In the present embodiment, the image input device 100a is representatively selected from the image input devices 100a to 100c, and the image input device 100a is described as the reference image input device. In the present embodiment, for convenience of explanation, the image input device 100a having the lowest saturation light intensity is used as the reference image input device. The image input device 100a corresponding to Wa is determined as the reference image input device. The determination may be considered that the determination unit of the calibration device 50 (not shown) determines the comparison by comparing the magnitudes of Wa, Wb, and Wc.

  In the present embodiment, for convenience of explanation, the image input device 100a having the lowest saturation illuminance is determined as the representative reference image input device, but the representative reference image input device may be selected by another method. Good.

  Subsequently, in step S14, the recording unit 10 of the calibration device 50 records the sensor saturation pixel value with respect to the saturation light intensity of the reference image input device. In the present embodiment, since the reference image input device is the image input device 100a, the recording unit 10 records the sensor saturated pixel value with respect to the saturated light intensity Wa of the image input device 100a.

  In step S <b> 16, Target_Max and Target_Min used for mapping are set in the setting unit 12 of the calibration device 50. Note that Target_Max and Target_Min may be set in the recording unit 10, and which one to use may be appropriately selected. Further, Target_Max and Target_Min may be configured to be input by the user via an input device or the like (not shown).

  Next, step S20 to step S26 will be described. Steps S20 to S26 may be performed before or after Steps S10 to S16 or simultaneously. Further, the order of steps S20 to S26 may be arbitrarily determined, and is not limited to this order.

  In step S20 and step S22, the recording unit 10 records the bright light intensity sensor pixel value and the dark light intensity sensor pixel value of the reference image input device. Therefore, in the present embodiment, since the reference image input device is the image input device 100a, the recording unit 10 records the bright light intensity sensor pixel value and the dark light intensity sensor pixel value of the image input device 100a.

  In step S24 and step S26, the recording unit 10 records the bright light intensity sensor pixel value and the dark light intensity sensor pixel value of the image input device that is the object of calibration. Accordingly, in the present embodiment, the recording unit 10 records the bright light intensity sensor pixel value and the dark light intensity sensor pixel value of the image input device 100b. Then, when the process up to step S26 is completed, the process proceeds to step S30.

  In step S <b> 30, the calculation unit 11 calculates a pseudo sensor saturation pixel value of the image input device to be calibrated. In the present embodiment, the calculation unit 11 calculates the pseudo sensor saturation pixel value of the image input device 100b with respect to the saturation light intensity Wa based on the pixel values recorded in steps S14, 16, 20-26. The pseudo sensor saturated pixel value is calculated according to the above formulas (1) to (3).

  In step S32, the mapping unit 13 acquires the pseudo sensor saturation pixel value of the image input device 100b from the calculation unit 11, and also records the Target_Max, Target_Min, and the dark light intensity sensor pixel value of the image input device 100b. Read from. In addition, the mapping unit 13 assigns the pseudo sensor saturation pixel value of the image input device 100b to Target_Max, and assigns the dark light intensity sensor pixel value of the image input device 100b to Target_Min.

  Therefore, for example, when the minimum value of the range that can be taken by the pixel value after calibration of the image input apparatus 100b is set to 0 and the maximum value of the range that can be taken is set to 255, Target_Min is set to 0, and Target_Max is set to 255, respectively. In addition, the range which can be taken is not restricted to 0-255, You may select suitably.

  As described above, the calibration of the image input apparatus 100b that is the object of calibration is performed through steps S10 to S32. In the above description, the image input device 100b has been described. However, the same calibration result can be obtained for other image input devices through the same steps.

  Next, effects obtained by calibration by the calibration device 50 will be described.

  According to the above configuration, in the calibration device 50, the calculation unit 11 includes the bright light intensity sensor pixel value, the dark light intensity sensor pixel value, and the sensor saturation pixel value related to the image input device 100a that is the reference image input device. Then, the bright light intensity sensor pixel value and the dark light intensity sensor pixel value related to the image input device 100b are read from the recording unit 10, and the pseudo sensor saturated pixel value of the image input device 100b is calculated based on these values.

  Therefore, the pseudo sensor saturation pixel value is calculated as a value obtained by taking in the bright light intensity sensor pixel value and the dark light intensity sensor pixel value of the image input device 100b. The bright light intensity sensor pixel value and the dark light intensity sensor pixel value are values calculated as an average value of a plurality of sensor elements. Therefore, a pseudo sensor saturation pixel value including a variation in pixel values output by the plurality of sensor elements is acquired.

  Further, in the calibration device 50, the mapping unit 13 uses the dark light intensity sensor pixel value related to the image input device 100b as Target_Min as the pixel value after calibration of the image input device 100b, and the pseudo sensor saturation of the image input device 100b. Pixel values are mapped to Target_Max, respectively. The same calibration can be executed for the image input devices 100a and 100c. Therefore, variations in pixel values output from a plurality of image sensors can be unified.

  In addition, since the calibration apparatus 50 defines the image input apparatus 100a as a representative image input apparatus, it is not necessary to individually change parameters for each of the image input apparatuses 100a to 100c, and time and labor required for calibration are reduced. It can be greatly reduced. This makes it possible to unify variations in pixel values output from a plurality of image sensors while greatly reducing the time and labor required for calibration.

  In the calibration device 50, calibration is performed from the darkest light intensity (dark light intensity) to the brightest light intensity (saturated light intensity) in the linear region of the average output characteristic curve of the reference image input device (image input device 100a). It is possible to set the target light intensity range. Therefore, calibration in a wider light intensity range is possible, and a more suitable calibration device can be provided to the user.

  The calibration device 50 preferably calibrates the light intensity region corresponding to the linear region of the reference image input device. This is because the mapping unit 13 assigns Target_Min and Target_Max as possible ranges of the pixel values after calibration of the image input apparatus 100b. In other words, in order to accurately calibrate the gray area existing between the sensor saturation pixel value (white) and the dark light intensity sensor pixel value (black), the pixel value has a correlation with the change in light intensity. This is because calibration is particularly desired in a changing light intensity range.

  Therefore, it is preferable that the reference image input device is an image input device with the highest sensitivity. If the image input device with the lowest sensitivity is used as the reference image input device, the saturation light intensity of the reference image input device is included in the saturation region in the image input device with the highest sensitivity. As a result, the most sensitive image input device (more precisely, all image input devices having higher sensitivity than the reference image input device) is not accurately calibrated in the gray region.

  For this reason, the calibration device 50 uses the most sensitive image input device as the reference image input device, and the darkest light intensity (dark light intensity) to the brightest light intensity (dark light intensity) in the linear region of the average output characteristic curve. It is particularly preferable that the light intensity range up to (saturated light intensity) is a target light intensity range for calibration.

  Further, when the image input device having the highest sensitivity is an image input device, the image input device has a sensitivity lower than that of the image input device 100a, that is, the image input device 100b excluding the image input device 100a, and the image input device 100c. In this case, the image input apparatus 100a itself can be accurately calibrated, so that the yield can be significantly improved when the image input apparatus is manufactured. Moreover, since the possible pixel values after calibration are Target_Min and Target_Max, it is not necessary to individually change parameters for each image input device, and the time and labor required for calibration can be greatly reduced. it can.

  However, it may be difficult in practice to extract the image sensor having the highest sensitivity from all the image sensors. Therefore, in such a case, it is preferable to extract an image sensor with higher sensitivity and use the image sensor as a reference image sensor. Thereby, the yield can be improved when manufacturing the image sensor.

  Further, the calibration apparatus 50 can arbitrarily determine Target_Max and Target_Min.

  Therefore, since the range can be determined according to the usage status of the image input apparatus, the user can perform flexible calibration according to the usage status, the usage purpose, and the like.

  When each image input device includes the calibration device 50, it is not necessary to individually change the parameters (Target_Max and Target_Min) for each image input device, so that the time and labor required for calibration are greatly increased. Can be reduced.

  Note that the bright light intensity sensor pixel value, the dark light intensity sensor pixel value, and the sensor saturation pixel value related to the image input apparatus 100a that is the reference image input apparatus are recorded in the recording unit 10 at the time of manufacturing the image input apparatus 100b or at the time of factory shipment. Can be stored in advance. In addition, the bright light intensity sensor pixel value and the dark light intensity sensor pixel value related to the image input device 100b may be stored in the recording unit 10 after each pixel value is measured. Thereby, the calculation unit 11 can calculate the pseudo sensor saturation pixel value by reading each pixel value from the recording unit 10 when performing calibration.

  As described above, in the calibration device 50 according to the present embodiment, the time and labor required for calibration to unify the dispersion of pixel values output from a plurality of image input devices are greatly reduced, and image input is performed. It is possible to significantly improve the yield when manufacturing the device.

Although the present embodiment has been described by taking a liquid crystal panel with a built-in optical sensor as an example, the present invention is not limited to a liquid crystal panel with a built-in optical sensor, but can be applied to a general image sensor such as a CCD.
[Other Embodiments]
Here, a calibration device 60 according to another embodiment of the present invention will be described with reference to FIGS. The description of the same contents as those described with reference to FIG. FIG. 13 is a diagram illustrating the relationship of a to d represented by Expression (4). FIG. 14 is a diagram illustrating how the bright light intensity sensor pixel value and the dark light intensity sensor pixel value of the image input device 100b are assigned to Target_Mid, respectively. FIG. 15 is a flowchart for explaining the calibration operation according to the calibration device 60.

  Based on FIG. 13, the calibration device 60 acquires the sensor saturation pixel value, the bright light intensity sensor pixel value, and the dark light intensity sensor pixel value related to the reference image input device 100a, and sets Target_Max to the sensor saturation pixel value. , Target_Min is assigned to each dark light intensity sensor pixel value. Note that Target_Max and Target_Min are given based on the information processing capability of the reference image input device 100a. For example, when the reference image input apparatus 100a handles pixel values of 0 to 255, Target_Min is assigned to 0 and 255 is assigned to Target_Max. Alternatively, Target_Min may be assigned to 10 when pixel values 0 to 10 are to be used for other purposes.

  Then, Target_Mid (a pseudo pixel value) is calculated based on the following equations (4) and (5).

y = {x2 * (Z1-y1) + z2 * (y1-x1)} / (z1-x1) (4)
However,
y: Target_Mid of the reference image input device 100a
x1: Dark light intensity sensor pixel value of the reference image input device 100a y1: Bright light intensity sensor pixel value of the reference image input device 100a z1: Sensor saturation pixel value of the reference image input device 100a x2: Target_Min of the reference image input device 100a
z2: Target_Max of the reference image input device 100a
And

  Equation (4) is calculated based on the following equation (5).

a: b = c: d (5)
However,
a: Sensor saturation pixel value-bright light intensity sensor pixel value in the image input apparatus 100a b: Bright light intensity sensor pixel value-dark light intensity sensor pixel value in the image input apparatus 100a c: Target_Max-Target_Mid in the image input apparatus 100a
d: Target_Mid-Target_Min in the image input device 100b
And

  Subsequently, as shown in FIG. 14, the bright light intensity sensor pixel value of the image input device 100b is assigned to Target_Mid, and the dark light intensity sensor pixel value is assigned to Target_Min.

  That is, by setting the image input device 100a as the reference image input device, Target_Mid is calculated based on the sensor saturation pixel value, bright light intensity sensor pixel value, and dark light intensity sensor pixel value, and Target_Max and Target_Min. Target_Mid is acquired regardless of the characteristics of the image input apparatus 100b to be calibrated.

  This operation will be described with reference to FIG.

  The recording unit 10 records Target_Mid of the image input device 100a which is a reference image input device. This value is obtained based only on the image input device 100a. The recording unit 10 also records the bright light intensity sensor pixel value and the dark light intensity sensor pixel value of the image input device 100b.

  Alternatively, Target_Mid may be calculated by the calculation unit 11. In this case, the recording unit 10 records the sensor saturation pixel value, the bright light intensity sensor pixel value, and the dark light intensity sensor pixel value related to the image input device 100a, and further records Target_Max and Target_Min. .

  Then, the calculation unit 11 reads out the sensor saturation pixel value, the bright light intensity sensor pixel value, the dark light intensity sensor pixel value, and Target_Max and Target_Min from the recording unit 10 according to the image input device 100a, and the above (4), ( 5) Calculate Target_Mid based on the formula. That is, the calculation unit 11 calculates Target_Mid after assigning the sensor saturation pixel value related to the image input device 100a to Target_Max and assigning the dark light intensity sensor pixel value related to the image input device 100a to Target_Min.

  An allocating unit (first allocating unit) (not shown) may allocate the sensor saturation pixel value related to the image input device 100a to Target_Max and the dark light intensity sensor pixel value related to the image input device 100a to Target_Min.

  The setting unit 12 stores Target_Min as a target value used by the mapping unit 13 for mapping as a setting value. Note that Target_Min may have a configuration recorded in the recording unit 10, and which configuration is adopted may be appropriately selected.

  The mapping unit 13 (second allocating unit) reads Target_Mid, the bright light intensity sensor pixel value and the dark light intensity sensor pixel value of the image input device 100b from the recording unit 10, and reads Target_Min from the setting unit 12, respectively. Then, the mapping unit 13 assigns (maps) the bright light intensity sensor pixel value of the image input device 100b to Target_Mid and the dark light intensity sensor pixel value of Target_Min.

  With the above operation, the bright light intensity sensor pixel value and the dark light intensity sensor pixel value of the image input device 100b are calibrated to Target_Mid and Target_Min, respectively. Even if the calibration target is the image input device 100c, the same calibration result is obtained.

  The operation of the calibration process based on the above configuration will be described with reference to FIG. FIG. 15 is a flowchart for explaining the calibration operation according to the calibration device 60.

  First, in steps S40 to S44, the average values of the sensor values of the image input device 100a in the bright light intensity, the dark light intensity, and the saturated light intensity are the bright light intensity sensor pixel value, the dark light intensity sensor pixel value, and the sensor saturated pixel. Get as value. Note that the order of the flow from step S40 to step S44 is not limited to the order shown in FIG.

  Subsequently, in step S46, the calculation unit 11 (or an assigning unit (not shown in FIG. 1)) sets the sensor saturation pixel value related to the image input device 100a to Target_Max, and the dark light intensity sensor pixel value related to the image input device 100a. Assign to Target_Min.

  In step S48, the calculation unit 11 reads out the sensor saturation pixel value, the bright light intensity sensor pixel value, the dark light intensity sensor pixel value, and Target_Max and Target_Min from the recording unit 10 according to the image input device 100a. Calculate Target_Mid based on equations (4) and (5).

  Next, in steps S50 and 52, the average values of the sensor values in the bright light intensity and the dark light intensity of the image input device 100b are acquired. Note that the order of the steps S50 and S52 is not limited to the order shown in FIG. Further, the operations in steps S50 and S52 may be performed prior to or simultaneously with the operations from step S40 to step S48.

  Finally, the mapping unit 13 reads the Target_Mid and the bright light intensity sensor pixel value and the dark light intensity sensor pixel value of the image input device 100b from the recording unit 10 and the Target_Min from the setting unit 12, respectively. Then, the mapping unit 13 maps the bright light intensity sensor pixel value of the image input device 100b to Target_Mid and the dark light intensity sensor pixel value to Target_Min.

  With the above operation, the bright light intensity sensor pixel value and the dark light intensity sensor pixel value of the image input device 100b are calibrated to Target_Mid and Target_Min, respectively. Even if the calibration target is the image input device 100c, the same calibration result is obtained.

  Next, effects obtained by calibration by the calibration device 60 will be described.

  According to the above configuration, the calibration device 60 performs image input based on the sensor saturation pixel value, the bright light intensity sensor pixel value, the dark light intensity sensor pixel value, and Target_Max and Target_Min related to the image input device 100a. Target_Mid is calculated as the pixel value for the bright light intensity sensor pixel value of the device 100a.

  That is, by determining the image input device 100a that is the reference image input device, Target_Mid is calculated based on the sensor saturation pixel value, the bright light intensity sensor pixel value, the dark light intensity sensor pixel value, and Target_Max and Target_Min. Therefore, Target_Mid is acquired regardless of the image input device 100b.

  Then, the mapping unit 13 assigns Target_Mid and Target_Min to the bright light intensity sensor pixel value and the dark light intensity sensor pixel value, respectively, as the pixel values after calibration of the image input device 100b.

  Therefore, it is not necessary to consider the characteristics of the image input device 100b when calibrating the image input device 100b. Similarly, it is not necessary to consider the characteristics of the image input device 100c when calibrating the image input device 100c. That is, there is no need to individually set / change parameters for each image input device to be calibrated.

  As described above, since the calibration of the image input device 100b or the image input device 100c is performed based on the representatively selected image input device 100a, the calibration device 60 greatly reduces the time and labor required for calibration. However, variations in pixel values output from a plurality of image sensors can be unified. It goes without saying that the image input apparatus 100a itself can also be a calibration target.

  Finally, each block of the calibration devices 50 and 60, in particular, the calculation unit 11 and the mapping unit 13 may be configured by hardware logic, or may be realized by software using a CPU as follows.

  That is, the calibration devices 50 and 60 include a CPU (central processing unit) that executes instructions of a control program for realizing each function, a ROM (read only memory) that stores the program, and a RAM (random access) that expands the program. memory), a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is a record in which the program code (execution format program, intermediate code program, source program) of the control program of the calibration devices 50 and 60, which is software that implements the functions described above, is recorded so as to be readable by a computer. This can also be achieved by supplying the medium to the calibration devices 50 and 60 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the calibration devices 50 and 60 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  In addition, this invention is not limited to embodiment mentioned above, A various change is possible in the range shown to the claim. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  The present invention relates to a calibration device that unifies variations in pixel values output from a plurality of image sensors by calibrating pixel values output from an image sensor, and particularly includes such a calibration device. It is suitable for an image input device.

10 Recording unit 11 Calculation unit (calculation means)
12 Setting part 13 Mapping part (assignment means)
50, 60 Calibration device 100, 100a to 100c Image input device 101a to 101c Average output characteristic curve

Claims (10)

A calibration device that unifies pixel values output from a plurality of image sensors to the same pixel value for the same light intensity,
Among the plurality of image sensors, an image sensor to be calibrated is a target image sensor, a representatively selected image sensor is a reference image sensor, a light intensity under dark conditions is a first light intensity, and the first light. In the case where the two light intensities higher than the intensity are the second and third light intensities in order of increasing light intensity,
First allocation means for allocating a first allocated pixel value and a third allocated pixel value larger than the first allocated pixel value as pixel values for the first and third light intensities of the reference image sensor;
Based on the first to third reference pixel values, which are pixel values of the reference image sensor for the first to third light intensities, and the first and third assigned pixel values, the second of the reference image sensor. Calculating means for calculating a pseudo pixel value as a pixel value with respect to light intensity;
Second allocating means for allocating the first allocated pixel value and the pseudo-controlled pixel value to the first and second light intensities as pixel values after calibration of the target image sensor, respectively. A calibration device characterized by the above. A calibration device that unifies pixel values output from a plurality of image sensors to the same pixel value for the same light intensity,
Among the plurality of image sensors, an image sensor to be calibrated is a target image sensor, a representatively selected image sensor is a reference image sensor, a light intensity under dark conditions is a first light intensity, and the first light. In the case where the two light intensities higher than the intensity are the second and third light intensities in order of increasing light intensity,
First to third reference pixel values that are pixel values of the reference image sensor for the first to third light intensities, and first and first pixel values of the target image sensor for the first and second light intensities. Calculating means for calculating a pseudo pixel value as a pixel value of the target image sensor for the third light intensity based on two target pixel values;
As a pixel value after calibration of the target image sensor, a first assigned pixel value and a third assigned pixel value larger than the first assigned pixel value are set for the first and third light intensities. And a allocating means for allocating the first target pixel value to the first allocated pixel value and the pseudo control pixel value to the third allocated pixel value, respectively. The calibration apparatus according to claim 1, wherein the calculation unit calculates the pseudo pixel value according to an expression (A).
y = {x2 * (Z1-y1) + z2 * (y1-x1)} / (z1-x1) (A)
However,
y: pseudo pixel value of the reference image sensor x1: first reference pixel value of the reference image sensor y1: second reference pixel value of the reference image sensor z1: third reference pixel value of the reference image sensor x2: first of the reference image sensor Assigned pixel value z2: The third assigned pixel value of the reference image sensor. The calibration apparatus according to claim 2, wherein the calculation unit calculates the pseudo pixel value according to an expression (B).
z = {x2 * (y1-Z1) + y2 * (z1-x1)} / (y1-x1) (B)
However,
z: pseudo control pixel value of target image sensor x1: first reference pixel value of reference image sensor y1: second reference pixel value of reference image sensor z1: third reference pixel value of reference image sensor x2: first of target image sensor Target pixel value y2: The second target pixel value of the target image sensor.   In the average output characteristic curve showing the relationship between the intensity of light incident on the image sensor and the pixel value output from the image sensor with respect to the intensity of the light, the pixel value correlates with the change in light intensity. The third light intensity is the saturated light intensity of the reference image sensor when the light intensity that is the boundary between the light intensity region that changes with the light intensity region and the light intensity region that is higher than the light intensity region is defined as the saturated light intensity. The calibration apparatus according to claim 1, wherein the calibration apparatus is characterized in that:   6. The calibration apparatus according to claim 1, wherein each of the first and third assigned pixel values is arbitrarily determined. A calibration method for unifying pixel values output from a plurality of image sensors to the same pixel value for the same light intensity,
Among the plurality of image sensors, an image sensor to be calibrated is a target image sensor, a representatively selected image sensor is a reference image sensor, a light intensity under dark conditions is a first light intensity, and the first light. In the case where the two light intensities higher than the intensity are the second and third light intensities in order of increasing light intensity,
A first assignment step of assigning a first assigned pixel value and a third assigned pixel value larger than the first assigned pixel value as pixel values for the first and third light intensities of the reference image sensor;
Based on the first to third reference pixel values, which are pixel values of the reference image sensor for the first to third light intensities, and the first and third assigned pixel values, the second of the reference image sensor. A calculation step of calculating an artificial pixel value as a pixel value with respect to light intensity;
A second assignment step of assigning the first assigned pixel value and the pseudo-predicted pixel value to the first and second light intensities as pixel values after calibration of the target image sensor, respectively. A calibration method characterized by A calibration program that unifies pixel values output from a plurality of image sensors to the same pixel value for the same light intensity,
Among the plurality of image sensors, an image sensor to be calibrated is a target image sensor, a representatively selected image sensor is a reference image sensor, a light intensity under dark conditions is a first light intensity, and the first light. In the case where the two light intensities higher than the intensity are the second and third light intensities in order of increasing light intensity,
A first assignment step of assigning a first assigned pixel value and a third assigned pixel value larger than the first assigned pixel value as pixel values for the first and third light intensities of the reference image sensor;
Based on the first to third reference pixel values, which are pixel values of the reference image sensor for the first to third light intensities, and the first and third assigned pixel values, the second of the reference image sensor. A calculation step of calculating an artificial pixel value as a pixel value with respect to light intensity;
A second assignment step of assigning the first assigned pixel value and the pseudo-pixel value to the first and second light intensities as pixel values after calibration of the target image sensor, to a computer; Calibration program to be executed.   A computer-readable recording medium on which the calibration program according to claim 8 is recorded.   An image input device comprising the calibration device according to claim 1.

Images