JP5827868B2 - Electronic endoscope and fixed pattern noise removal method - Google Patents

Description

  The present invention relates to an electronic endoscope and a fixed pattern noise removal method, and more particularly to an electronic endoscope including a CMOS type image sensor and a fixed pattern noise removal method.

  In the medical field, an electronic insertion device is used in which an elongated insertion portion is inserted into a subject's body, and the subject's body is directly imaged and diagnosed by a small imaging device mounted near the distal end of the insertion portion. Endoscopy systems are widespread. A minute imaging device mounted in such an electronic endoscope system is equipped with a so-called image sensor, which irradiates the body of the subject with illumination light and receives the light reflected in the body of the subject. This is photoelectrically converted to accumulate signal charges for each pixel. Then, an image signal corresponding to the signal charge is output from the image sensor to obtain an observation image.

  By the way, as an image sensor mounted on a general imaging device, a CMOS image sensor (hereinafter referred to as a CMOS sensor) and a CCD image sensor are known. The CMOS sensor can be driven at a low voltage, and can easily meet the demands for a large number of pixels and high-speed reading. In addition, a CMOS process can be used in the manufacturing process, and peripheral circuits such as a drive circuit and a processing circuit can be mixedly mounted in the same chip, which is advantageous for downsizing.

  Here, as a configuration of a general CMOS sensor, for example, as illustrated in FIG. 14, pixels 900 including photoelectric conversion elements are arranged in a matrix, and each pixel 900 is formed by a vertical scanning circuit 902 and a horizontal scanning circuit 904. Are sequentially selected, and an output signal from each pixel 900 is amplified by an amplifier 906, and then converted into a digital signal by an A / D converter (ADC) 908 and output.

  However, in such a configuration, since gain adjustment is performed on the output signal at the final stage, it is necessary to drive the amplifier 906 at a relatively high frequency, and random noise having a magnitude corresponding to the frequency is added. The image becomes large. In particular, when the drive frequency of an analog circuit such as an amplifier is increased due to the recent increase in the number of pixels and the increase in the frame rate, more random noise is added.

  On the other hand, as shown in FIG. 15, a CMOS sensor in which a column amplifier 910 is mounted in parallel for each column is known. According to this method, the drive frequency of each column amplifier 910 can be lowered to a frequency corresponding to the reciprocal of the number of columns. As a result, random noise can be reduced even with a large number of pixels and a high frame rate. In addition, although not shown in the drawings, it is also known that an A / D converter (column ADC) is mounted in parallel for each column in order to realize high-speed reading simultaneously with reduction of random noise.

  By the way, in the conventional CMOS sensor provided with the column amplifier and the column ADC, the random noise can be reduced. However, the fixed pattern noise is generated in the image generated by the imaging signal output from the CMOS sensor due to the performance variation of the column amplifier and the column ADC. There is a problem that (vertical streak noise) is added.

  With respect to such a problem, in Patent Document 1, by applying a test voltage signal to a column amplifier (column amplifier) in a CMOS sensor and detecting the output voltage, the characteristic variation of the column amplifier can be reduced. There is disclosed a technique of detecting and obtaining correction data based on the characteristic variation, and removing fixed pattern noise caused by the characteristic variation from the sensor output during normal photographing using the correction data.

  Further, Patent Document 2 includes an error value storage memory that stores an error value for each column (low temperature error value) detected at a low temperature and an error value for each column (high temperature error value) detected at a high temperature. There is a technique for correcting an error value by interpolating two error values in accordance with the temperature of the CMOS sensor and removing fixed pattern noise by subtracting the corrected error value from an imaging signal output from the CMOS sensor. It is disclosed.

JP 2008-306565 A JP 2006-135423 A

  However, although the technique disclosed in Patent Document 1 can detect the variation in the characteristics of the column amplifier without being affected by random noise or errors caused by the imaging optical system, the variation in the temperature characteristics of the column amplifier. It is difficult to reliably remove fixed pattern noise that varies depending on

  On the other hand, according to the technique disclosed in Patent Document 2, processing for removing fixed pattern noise using low temperature and high temperature error values stored in the error value storage memory according to the temperature of the CMOS sensor. Therefore, fixed pattern noise can be surely removed without being affected by variations in the temperature characteristics of the column amplifier. However, Patent Document 2 discloses a device for applying to an electronic endoscope system. Nothing has been considered.

  Here, when the technique disclosed in Patent Document 2 is to be applied to an electronic endoscope system, a correction circuit and a correction memory for removing fixed pattern noise are usually not electronic endoscopes but electronic endoscopes. Generally, it is mounted on an external control device (processor device) configured separately from the endoscope. However, since a plurality of electronic endoscopes are exchanged for use in one processor device, a special correction circuit and a correction memory are required for each electronic endoscope, which increases the size and cost of the processor device. It becomes a factor inviting. As a result, there is a problem that the system becomes less versatile.

  The present invention has been made in view of such circumstances, and a special correction circuit or correction memory is not provided in an external control device (processor device) connected to an electronic endoscope having a CMOS type image sensor. Another object of the present invention is to provide an electronic endoscope and a fixed pattern noise removing method capable of reliably removing fixed pattern noise (longitudinal streak noise) without depending on temperature.

In order to achieve the above object, an electronic endoscope according to the present invention is provided at the distal end of an insertion portion to be inserted into a subject, and is equipped with a CMOS type image sensor that images the inside of the subject. In the endoscope, the image sensor is provided in parallel with a plurality of pixel units arranged in a row direction and a column direction orthogonal to the row direction, and for each pixel column arranged in the column direction, and arranged in the pixel column A plurality of column amplifiers for amplifying the output signals output from the pixel units , an A / D converter for converting the signals amplified by the plurality of column amplifiers into digital signals, and a tip of the insertion unit. a temperature sensor for detecting a temperature, a correction data corresponding to the temperature detected by said temperature sensor, to supply the correction data for correction data and gain errors for offset error of the plurality of column amplifiers A correction data supplying unit, on the basis of the correction data supplied from the correction data supplying unit, e Bei a correction circuit, the removal of fixed pattern noise contained in the signal output from the A / D converter, the The correction circuit is a circuit for removing fixed pattern noise caused by an offset error and a gain error as individual differences of the plurality of column amplifiers, and the signal output from the A / D converter is used for the gain error. The correction data is multiplied, the offset error correction data is subtracted from the multiplied signal, and the image sensor outputs a signal from which the fixed pattern noise has been removed by the correction circuit as an imaging signal .

According to the present invention, a temperature sensor is provided in a CMOS type image sensor, and output signal correction processing is performed in a self-contained manner in accordance with the temperature detected by the temperature sensor. As a result, it is possible to reliably remove the fixed pattern noise caused by the individual difference between the column amplifiers provided for each pixel column without being affected by the environmental temperature. Even when a plurality of electronic endoscopes are exchanged for one processor device, it is not necessary to add a special correction circuit or a correction memory to the processor device. Even if they are mixed, they can be used, and a highly versatile system can be constructed.

The plurality of pixel units, the plurality of column amplifiers, the A / D converter, the temperature sensor, the correction data supply unit, and the correction circuit are preferably provided in the same semiconductor chip.

  The correction data supply unit includes a plurality of correction data for each predetermined temperature, and selects correction data corresponding to a temperature closest to the temperature detected by the temperature sensor from the plurality of correction data. It is preferable to supply.

  The correction data supply unit includes reference correction data corresponding to a predetermined reference temperature, and data obtained by multiplying the reference correction data by a correction coefficient determined according to a temperature detected by the temperature sensor. It is preferable to supply the correction data. In that case, it is more preferable that the correction data supply unit includes a correction coefficient table indicating a relationship between the temperature and the correction coefficient, and determines the correction coefficient based on the correction coefficient table.

  Moreover, it is preferable that the temperature sensor is configured using a photoelectric conversion element disposed in an invalid pixel portion that does not contribute to imaging in the subject among the plurality of pixel portions.

  Further, the temperature sensor may use an average of output signals output from the plurality of invalid pixel portions as temperature information.

In order to achieve the above object, a fixed pattern noise removing method according to the present invention is provided at the distal end of an insertion portion of an electronic endoscope to be inserted into a subject, and is a CMOS type that images the inside of the subject. A fixed pattern noise removing method for removing fixed pattern noise from an output signal of the image sensor, wherein the image sensor includes a plurality of pixel units arranged in a row direction and a column direction orthogonal thereto, and in the column direction. A plurality of column amplifiers that are provided in parallel for each of the arranged pixel columns and amplify output signals output from the pixel units arranged in the pixel columns, respectively, and signals amplified by the plurality of column amplifiers are digital signals An A / D converter for converting to a temperature sensor, a temperature sensor for detecting the temperature of the tip of the insertion portion, and correction data corresponding to the temperature detected by the temperature sensor, A correction data supply unit for supplying correction data for offset error and gain error for a plurality of column amplifiers, and output from the A / D converter based on the correction data supplied from the correction data supply unit e Bei a correction circuit, the removal of fixed pattern noise contained in the signal, and detecting a temperature of the tip of the insertion portion by the image sensor and the temperature sensor provided integrally with said image The correction circuit, which is one of the peripheral circuits provided integrally with the sensor , is included in the output signal output from the A / D converter based on correction data corresponding to the temperature detected by the temperature sensor. seen including a correction step, the removal of fixed pattern noise, the correction step, offset as individual differences of the plurality of column amplifiers A step of removing fixed pattern noise caused by an error and a gain error, wherein the signal output from the A / D converter is multiplied by the correction data for the gain error, and the signal for the offset error is multiplied from the multiplied signal The image sensor outputs a signal from which the fixed pattern noise has been removed in the correction step as an imaging signal.

The correction data supply unit preferably selects and supplies correction data corresponding to a temperature closest to the temperature detected by the temperature sensor from among a plurality of correction data provided for each predetermined temperature.

Further, the correction data supplying unit, a data obtained by multiplying the correction coefficient determined in accordance with the reference correction data temperature detected by the temperature sensor with respect to corresponding to a predetermined reference temperature and the corrected data It is preferable to supply . At this time, it is more preferable that the correction data supply unit determines the correction coefficient based on a correction coefficient table indicating a relationship between the temperature and the correction coefficient.

According to the present invention, a temperature sensor is provided in a CMOS type image sensor, and output signal correction processing is performed in a self-contained manner in accordance with the temperature detected by the temperature sensor. As a result, it is possible to reliably remove the fixed pattern noise caused by the individual difference between the column amplifiers provided for each pixel column without being affected by the environmental temperature. Even when a plurality of electronic endoscopes are exchanged for one processor device, it is not necessary to add a special correction circuit or a correction memory to the processor device. Even if they are mixed, they can be used, and a highly versatile system can be constructed.

Overall configuration diagram showing the schematic configuration of the electronic endoscope system Configuration diagram showing configuration example of CMOS sensor Block diagram showing the electrical configuration of the electronic endoscope system A graph showing an example of an image signal containing fixed pattern noise The graph which expanded and showed a part of FIG. Schematic diagram showing an example configuration of a semiconductor chip Graph showing an example of the image signal before correction Graph showing correction data for offset error Graph showing the image signal after correction The block diagram which showed the structural example of the CMOS sensor which concerns on 2nd Embodiment The figure which showed an example of the correction coefficient table The block diagram which showed the structural example of the CMOS sensor which concerns on 3rd Embodiment The block diagram which showed the structural example of the CMOS sensor which concerns on 4th Embodiment Configuration diagram showing a configuration example of a general CMOS sensor Configuration diagram showing a configuration example of a CMOS sensor equipped with a column amplifier

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(First embodiment)
As shown in FIG. 1, the electronic endoscope system 11 includes an electronic endoscope 12, a processor device 13, a light source device 14, and the like. The electronic endoscope 12 includes an insertion unit 16, a hand operation unit 17, a universal cord 18, and the like.

  Since the insertion portion 16 is inserted into the body of the subject, the insertion portion 16 is provided so as to be freely curved according to the shape of the body of the subject. The distal end portion 16a of the insertion portion 16 has a built-in imaging device equipped with a CMOS sensor. Furthermore, on the end surface of the tip portion 16a, an illumination window that irradiates illumination light forward, an observation window that guides light from the body of the subject to the CMOS sensor, a forceps outlet from which various treatment tools are exposed, washing water, An air / water supply nozzle or the like through which air or the like is injected is provided.

  Further, a bending portion 19 is provided behind the tip portion 16a. The bending portion 19 is formed by connecting a plurality of bending pieces, and is connected to an angle knob 21 provided in the hand operation portion 17 by a wire inserted through the insertion portion 16. Therefore, the bending portion 19 is freely bent in a desired direction in the up, down, left, and right directions when the wire inserted into the insertion portion 16 is pushed and pulled by the turning operation of the angle knob 21 provided in the hand operation portion 17. .

  The hand operation unit 17 is provided with various operation buttons such as a forceps port 22 and an air / water supply button 23 in addition to the angle knob 21 for operating the bending portion 19 as described above. A treatment instrument having an injection needle, a high-frequency knife or the like provided at the tip of the wire is inserted into the forceps port 22. Further, the air / water supply button 23 controls the air and water supplied from an air / water supply device (not shown).

  The universal cord 18 electrically connects the electronic endoscope 12 to the processor device 13 and optically connects the electronic endoscope 12 to the light source device 14 via a connector 24 provided at a base end portion thereof. To do.

  The processor device 13 is connected to the electronic endoscope 12, the light source device 14, the monitor 26, and the like, and comprehensively controls the operation of the electronic endoscope system 11. Further, the light source device 14 irradiates illumination light toward the observation site through the light guide (see FIG. 3) inserted through the universal cord 18 and the insertion portion 16.

  As described above, the electronic endoscope system 11 uses the CMOS sensor 31 as an image sensor. As shown in FIG. 2, the CMOS sensor 31 includes an imaging region 32, a vertical scanning circuit 33, a column amplifier 34, a column selection transistor 36, an A / D converter (ADC) 37, a horizontal scanning circuit 38, and a fixed pattern noise removing circuit. 80, a temperature sensor 82, and the like. Each of these functional units is provided in the same semiconductor chip (semiconductor substrate).

  In the imaging region 32, pixels 41 are arranged in a matrix, and an image of an observation site is formed by an imaging optical system (not shown). The pixel 41 includes a photodiode D1, an amplification transistor M1, a pixel selection transistor M2, a reset transistor M3, and the like. The photodiode D1 generates and accumulates signal charges corresponding to the amount of incident light through photoelectric conversion. The signal charge accumulated in the photodiode D1 is amplified as an imaging signal by the amplifying transistor M1, and is output to the outside of the pixel 41 at a predetermined timing by the pixel selecting transistor M2. Further, the signal charge accumulated in the photodiode D1 is discarded by the reset transistor M3 at a predetermined timing.

  In the imaging region 32, a row selection line L1 and a row reset line L2 are routed from the vertical scanning circuit 33 in the horizontal direction (X direction), and column signal lines are routed from the column amplifier 34 in the vertical direction (Y direction). L3 is wired.

  The row selection line L1 is connected to the gate of the pixel selection transistor M2, and the row reset line L2 is connected to the gate of the reset transistor M3. The column signal line L3 is connected to the source of the pixel selection transistor M2, and is connected to the column selection transistor 36 of the corresponding column via the column amplifier 34.

  The vertical scanning circuit 33 generates a vertical scanning signal based on the timing signal input from the timing generator (TG) 42, selects the row selection line L1 row by row, and outputs the imaging signal to the column signal line L3. The row of the pixel 41 to be changed (hereinafter referred to as a horizontal line) is changed. The vertical scanning circuit 33 selects the horizontal row reset line L2 one row at a time, and changes the horizontal line for discarding signal charges. Further, when the selected row of the row selection line L1 and the row reset line L2 reaches the horizontal line positioned at the end of the imaging region 32, the vertical scanning circuit 33 selects the top horizontal line again and performs the above-described operation. repeat.

  The column amplifier 34 is provided for each column (column signal line L3), and amplifies the imaging signal of the pixel 41 connected to the row selection line L1 selected by the vertical scanning circuit 33. The amplification factor of the imaging signal by the column amplifier 34 is adjusted by inputting a gain adjustment signal to the column amplifier 34.

  The horizontal scanning circuit 38 generates a horizontal scanning signal based on the timing signal input from the TG 42, and performs on / off control of the column selection transistor 36.

  The column selection transistor 36 is provided between the output bus line 43 connected to the A / D converter 37 and the column amplifier 34, and transfers an imaging signal to the output bus line 43 in accordance with a horizontal scanning signal. Select a pixel. The A / D converter 37 converts the imaging signal sequentially transferred from the column amplifier 34 to the output bus line 43 from an analog signal to a digital signal and outputs the converted signal.

  As shown in FIG. 3, the electronic endoscope 12 includes the above-described CMOS sensor 31, the objective lens 51, the CPU 52, the timing generator (TG) 42, the light guide 53, and the like.

  The CMOS sensor 31 is provided at the distal end portion 16a of the insertion portion 16 as described above, and the objective lens 51 is disposed in front of the CMOS sensor 31. An observation window 54 is provided on the end face of the distal end portion 16 a of the insertion portion 16, and light from the subject incident from the observation window 54 is imaged on the imaging region 32 of the CMOS sensor 31 by the objective lens 51. The CMOS sensor 31 photoelectrically converts the subject image formed in the imaging region 32 for each pixel 41, accumulates signal charges corresponding to the exposure amount in each pixel 41, and outputs them as an imaging signal at a predetermined timing. The driving timing of the CMOS sensor 31 is controlled by a timing signal input from the TG 42.

  The CPU 52 communicates with the CPU 61 of the processor device 13 and comprehensively controls each unit of the electronic endoscope 12. For example, the CPU 52 controls the operation of the CMOS sensor 31 by generating a predetermined timing signal in the TG 42 based on a signal from the CPU 61 of the processor device 13. Further, the CPU 52 inputs a gain adjustment signal to the column amplifier 34, thereby adjusting the amplification factor from the CMOS sensor 31 and outputting an imaging signal.

  The light guide 53 is inserted into the universal cord 18 and the insertion portion 16, one end is connected to the illumination window 56 provided on the end surface of the distal end portion 16 a of the insertion portion 16, and the other end is connected to the light source device 14. Has been. The illumination light from the light source device 14 is irradiated from the illumination window 56 to the observation site through the light guide 53.

  As shown in FIG. 3, the processor device 13 includes a CPU 61, a DSP 62, a D / A conversion circuit 63, a RAM 64, and the like.

  The CPU 61 controls the operation of each part of the processor device 13 and communicates with the CPU 52 of the electronic endoscope 12 and the CPU 71 of the light source device 14 to control the operation of the electronic endoscope 12 and the light source device 14. The electronic endoscope system 11 is comprehensively controlled.

  The DSP 62 performs various signal processing such as color interpolation, white balance adjustment, and gamma correction on the imaging signal output from the CMOS sensor 31. The image pickup signal output from the CMOS sensor 31 is held in the RAM 64 as image data when various image processes are performed by the DSP 62. The DSP 62 reads necessary image data from the RAM 64, converts it into a video signal such as an NTSC signal, and D / A converts it by the D / A conversion circuit 63, thereby displaying the image data on the monitor 26.

  As shown in FIG. 3, the light source device 14 includes a CPU 71, a light source 72, a wavelength selection filter 73, and the like. The CPU 71 communicates with the CPU 61 of the processor device 13 and controls the light source device 14 by driving the light source 72 and the wavelength selection filter 73 in accordance with the drive timing of the CMOS sensor 31 and the like.

  The light source 72 is a high-intensity light source that emits high-intensity light over a wide wavelength band, such as a xenon lamp or a halogen lamp, and the CPU 71 controls switching on and off. Further, the light emitted from the light source 72 is efficiently guided to the light guide 53 by the condenser lens 74.

  The wavelength selection filter 73 is a filter that restricts the light emitted from the light source 72 to light in a specific wavelength band, and is inserted or retracted between the light source 72 and the condenser lens 74 based on an instruction from the CPU 71. Is done. As a result, the light source device 14 can freely switch the wavelength band of the illumination light as necessary, such as settings and doctor operations. For example, the light source device 14 can output not only white visible light but also light other than visible light such as infrared light and special light in which the ratio of RGB components of visible light is adjusted. It has become.

  In the electronic endoscope system 11 configured as described above, the CMOS sensor 31 is provided with a column amplifier 34 for each column (see FIG. 2), so that each pixel 41 is uniformly distributed due to individual differences of the column amplifiers 34. Even if the exposure is performed, the output varies from column to column, which causes a fixed pattern noise (vertical streak noise) to be added to the imaging signal output from the CMOS sensor 31.

  FIG. 4 is a graph showing an example of data for one line in the horizontal direction in an imaging signal to which fixed pattern noise is added. FIG. 5 is an enlarged graph showing a part of FIG. 4 and 5, the horizontal axis represents the pixel number in the horizontal direction, and the vertical axis represents the output value (pixel value) of the pixel. Also shown are data when the ambient temperature around the CMOS sensor 31 is changed to 24 ° C., 36 ° C., 45 ° C., and 57 ° C., respectively. Here, in order to remove random noise, the average value for 10 lines is taken as the output value of each pixel.

  As shown in FIGS. 4 and 5, the actual output value of each pixel is larger than the ideal output value (64 in this example) of each pixel when there is no individual difference between the column amplifiers 34. It ’s small. Such variation in the output value of each pixel occurs due to performance variation of the column amplifier 34 provided for each column. Further, due to variations in temperature characteristics of the column amplifiers 34, the output values of the pixels vary for each temperature. In the examples shown in FIGS. 4 and 5, the variation in the output value of each pixel increases as the temperature decreases, but the present invention is not limited to this example. For these reasons, it is effective to correct according to the environmental temperature around the CMOS sensor 31 in order to surely remove the fixed pattern noise.

  Therefore, in the present embodiment, as shown in FIG. 2, a fixed pattern noise removing circuit 80 and a temperature sensor 82 are provided in the same semiconductor chip as the CMOS sensor 31, and an imaging signal output from the CMOS sensor 31 according to the environmental temperature. The correction processing for removing the fixed pattern noise included in the image is performed in a self-contained manner. Note that there are an offset error and a gain error as individual differences between the column amplifiers 34, but here, for convenience of explanation, a case where only the offset error is corrected will be described as an example. A mode for correcting the gain error will be described later.

  The fixed pattern noise removal circuit 80 is provided in the subsequent stage of the A / D converter 37 and includes a plurality of line memories 84A to 84C, a subtractor 86, a line memory switching unit 88, and the like. In the line memories 84A to 84C, correction data for offset errors corresponding to predetermined temperatures (for example, 30 ° C., 40 ° C., and 50 ° C.) are stored in advance. These correction data are supplied to the subtracter 86 according to the switching state of the line memory switching unit 88 described later. 2 illustrates the configuration including the three line memories 84A to 84C, the number of line memories is not particularly limited. However, if the number of line memories is large, correction data can be stored in the line memory for each fine temperature. On the other hand, the size of the CMOS sensor chip increases, so the number of line memories is as small as possible. Is preferred.

  As a method of creating correction data for offset error, first, an object having a light-shielded state or a constant light amount is photographed by an imaging device including the CMOS sensor 31 under an environment of a predetermined temperature condition (for example, 30 ° C.). At this time, when shooting an object other than a black subject, it is preferable to perform shading correction in order to suppress the influence of shading of the light source and the lens.

  Next, an average value for several to several tens of horizontal lines is obtained in the CMOS sensor 31 and used as line data for one line. Thus, by taking the average value for a plurality of lines, the influence of random noise can be reduced, and correction data with higher accuracy can be created. Subsequently, an overall average value of the line data is obtained. Then, correction data for offset error is obtained by subtracting the overall average value from the line data. The above process is repeated a plurality of times while changing the temperature conditions.

  The correction data for the offset error thus obtained is written into the line memories 84A to 84C corresponding to each temperature condition. These processes are performed when the CMOS sensor 31 is manufactured. However, the present invention is not limited to this, and may be performed periodically when the electronic endoscope 12 is used or stored.

  The line memory switching unit 88 includes a changeover switch provided between the line memories 84 </ b> A to 84 </ b> C and the subtracter 86, and subtracters are selected from the line memories 84 </ b> A to 84 </ b> C according to the temperature detected by the temperature sensor 82. A line memory connected to 86 is selected.

  The subtracter 86 outputs a difference between the output signal of the A / D converter 37 and the correction data supplied from the line memory selected by the line memory switching unit 88.

  The temperature sensor 82 is provided in the same semiconductor chip as the CMOS sensor 31 as described above, and detects the temperature inside the distal end portion 16a of the insertion portion 16 where the CMOS sensor 31 is disposed. The temperature detected by the temperature sensor 82 is notified to the line memory switching unit 88.

  The position at which the temperature sensor 82 is disposed is not particularly limited as long as it is provided in the same semiconductor chip as the CMOS sensor 31. However, in the present embodiment, a semiconductor as shown in FIG. It is configured by a dedicated temperature sensor diode mounted on the peripheral circuit region 106 in the chip 100. As another preferable mode, a mode in which the photodiode D1 of the invalid pixel (optical black) of the invalid pixel region 104 arranged in the periphery of the valid pixel region 102 in the imaging region 32 is used as a temperature sensor, or the invalid pixel region 104. There is a mode in which an average of the output values of invalid pixels for one line is used as temperature information. In any aspect, the temperature sensor 82 can be disposed in the same semiconductor chip as the CMOS sensor 31 and can be mounted on the distal end portion 16 a of the insertion portion 16.

  When the inside of the body cavity is observed with the electronic endoscope system 11 configured as described above, the electronic endoscope 12, the light source device 14, and the monitor 26 are turned on and the electronic endoscope 12 is inserted. The part 16 is inserted into the body cavity, and an image in the body cavity imaged by the CMOS sensor 31 is observed on the monitor 26 while illuminating the body cavity with the illumination light from the light source device 14.

  At this time, the imaging signal output from each pixel 41 of the CMOS sensor 31 is amplified by a column amplifier 34 provided for each column, and then converted from an analog signal to a digital signal by an A / D converter 37. Is output. The imaging signal output from the A / D converter 37 is input to the subtracter 86 of the fixed pattern noise removal circuit 80. FIG. 7 shows an example of the imaging signal input to the subtractor 86 of the fixed pattern noise removal circuit 80 at this time. In the imaging signal shown in FIG. 7, the portion that is originally flat in the horizontal direction is wavy due to the offset error of each column amplifier 34.

  On the other hand, when the detected temperature of the temperature sensor 82 (the internal temperature of the distal end portion 16 a of the insertion portion 16) is notified to the line memory switching unit 88, the line memory switching unit 88 causes the line memory to correspond to the detected temperature of the temperature sensor 82. A predetermined line memory is selected from 84A to 84C.

  Here, as a line memory switching method, when there is a line memory storing correction data corresponding to the temperature detected by the temperature sensor 82 in each of the line memories 84A to 84C, the line memory is selected. . For example, when correction data corresponding to 30 ° C., 40 ° C., and 50 ° C. is stored in the line memories 84A to 84C as described above, when the temperature detected by the temperature sensor 82 is 30 ° C., the connection destination of the subtracter 86 is As a result, the line memory 84A is selected.

  When there is no line memory storing correction data corresponding to the temperature detected by the temperature sensor 82, the line memory storing correction data corresponding to the closest temperature is selected. For example, in the case of the above example, when the temperature detected by the temperature sensor 82 is 36 ° C., the line memory 84B is selected as the connection destination of the subtractor 86.

  Note that the line memory switching method is not limited to the method described above, and is stored in each of the line memories 84A to 84C when the temperature detected by the temperature sensor 82 does not match the corresponding temperature of the correction data prepared in advance. The correction data for each temperature may be supplied to the subtracter 86 as post-interpolation correction data obtained by interpolation processing. For example, in the case of 35 ° C. in the above-described example, an intermediate value between 30 ° C. correction data and 40 ° C. correction data is used as the correction data.

  When the line memory is switched by the line memory switching unit 88 in this way, correction data corresponding to the temperature detected by the temperature sensor 82 is supplied from the line memory to the subtracter 86. An example of the correction data supplied to the subtracter 86 at this time is shown in FIG. The correction data shown in FIG. 8 indicates the magnitude of the offset error when the 0 position in the vertical axis direction is used as a reference.

  The subtracter 86 outputs a corrected image signal obtained by subtracting correction data from the image signal output from the A / D converter 37. An example of the imaging signal output from the subtracter 86 at this time is shown in FIG. The image pickup signal shown in FIG. 9 is obtained by subtracting the correction data shown in FIG. 8 from the image pickup signal shown in FIG. 7, and the fixed pattern noise caused by the offset error of each column amplifier 34 is removed. .

  In this way, the corrected imaging signal from which the fixed pattern noise is removed by the fixed pattern noise removing circuit 80 is output from the CMOS sensor 31 and transmitted to the processor device 13 via the universal code 18.

  The processor device 13 receives the imaging signal output from the CMOS sensor 31 by the DSP 62, converts it into image data by the DSP 62, and displays an image on the monitor 26.

  According to the electronic endoscope 12 of the present embodiment, the CMOS sensor 31 is provided with the fixed pattern noise removing circuit 80 and the temperature sensor 82 in the same semiconductor chip, and the imaging signal corresponding to the detected temperature of the temperature sensor 82 is provided. The correction process is performed in a self-contained manner. As a result, it is possible to reliably remove the fixed pattern noise caused by the individual difference of each column amplifier 34 without being affected by the environmental temperature. Further, even when a plurality of electronic endoscopes 12 are exchanged for one processor device 13, it is not necessary to add new correction functions and correction data to the processor device 13, and new and old processors The devices 13 can be used together, and a highly versatile system can be constructed.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. Hereinafter, description of parts common to the first embodiment will be omitted, and description will be made focusing on characteristic parts of the present embodiment.

  The second embodiment is different from the first embodiment in that the number of line memories provided in the fixed pattern noise removing circuit 80 is not plural but one.

  FIG. 10 is a schematic configuration diagram illustrating a configuration example of the CMOS sensor 31 according to the second embodiment. In FIG. 10, components that are the same as or similar to those in FIG.

  As shown in FIG. 10, the fixed pattern noise removing circuit 80 includes a line memory 84, a subtracter 86, a coefficient determining unit 89, a multiplier 90, and the like. In the line memory 84, reference correction data corresponding to a predetermined reference temperature (for example, 30 ° C.) is stored in advance. The reference correction data stored in the line memory 84 is supplied to the multiplier 90.

  The coefficient determination unit 89 includes a correction coefficient table indicating the relationship between the temperature and the correction coefficient, determines a correction coefficient from the temperature detected by the temperature sensor 82 based on the correction coefficient table, and uses the determined correction coefficient as a multiplier 90. To supply.

  Here, an example of the correction coefficient table provided in the coefficient determination unit 89 is shown in FIG. In the correction coefficient table shown in FIG. 11, the reference temperature is 30 ° C., and the correction coefficient at that time is 1.000. The correction coefficient gradually decreases as the temperature becomes higher than 30 ° C., and the correction coefficient gradually increases as the temperature becomes lower than 30 ° C. Such a correction coefficient for each temperature is obtained in advance.

  For convenience of explanation, the temperature is set every 5 ° C., but the interval is not particularly limited, and the interval may be smaller or larger than 5 ° C. If there is no temperature in the correction coefficient table that matches the temperature detected by the temperature sensor 82, the correction coefficient corresponding to the closest temperature is used, or the correction coefficient corresponding to the temperature detected by the temperature sensor 82. May be calculated by interpolation processing.

  The multiplier 90 outputs correction data obtained by multiplying the reference correction data supplied from the line memory 84 by the correction coefficient determined by the coefficient determination unit 89. The correction data output from the multiplier 90 is supplied to the subtracter 86.

  The subtractor 86 outputs a corrected image signal obtained by subtracting the correction data given from the multiplier 90 from the image signal output from the A / D converter 37.

  As described above, according to the second embodiment, the fixed pattern noise removing circuit 80 is provided with one line memory 84, and the reference correction data on the line memory 84 is determined according to the temperature detected by the temperature sensor 82. A fixed pattern noise removal process is performed based on correction data obtained by multiplying the correction coefficient. In particular, the present embodiment is suitable when the offset error of each column amplifier 34 changes linearly according to temperature, and the number of line memories can be reduced compared to the first embodiment, and the circuit configuration is simplified. Can be. This makes it possible to reduce the diameter of the distal end portion of the insertion portion 16 of the electronic endoscope 12.

(Third embodiment)
Next, a third embodiment of the present invention will be described. Hereinafter, description of parts common to the first or second embodiment will be omitted, and description will be made focusing on characteristic parts of the present embodiment.

  In the first and second embodiments, this is a mode suitable when the main generation factor of fixed pattern noise is an offset error of each column amplifier 34, whereas in the third embodiment, the generation factor is This is a mode suitable for a gain error of each column amplifier 34.

  FIG. 12 is a schematic configuration diagram illustrating a configuration example of the CMOS sensor 31 according to the third embodiment. In FIG. 12, components that are the same as or similar to those in FIG.

  As shown in FIG. 12, the fixed pattern noise removal circuit 80 includes a gain error correction data supply unit 92, a multiplier 94, and the like.

  When the gain error correction data supply unit 92 acquires the temperature detected by the temperature sensor 82, the gain error correction data supply unit 92 supplies gain multiplier correction data corresponding to the detected temperature to the multiplier 94. The gain error correction data supply unit 92 may be configured from a plurality of line memories 84A to 84C, a line memory switching unit 88, etc. as in the first embodiment (see FIG. 2). As in the second embodiment, the line memory 84, the coefficient determination unit 89, the multiplier 90, and the like may be used (see FIG. 10). As the correction data for gain error correction, data obtained by dividing the amplifier gain of each column amplifier 34 by the average gain value (that is, normalized) may be used.

  The multiplier 94 outputs a corrected imaging signal obtained by multiplying the imaging signal output from the A / D converter 37 by the correction data for offset error supplied from the correction data supply unit 92.

  As described above, according to the third embodiment, it is possible to reliably remove the fixed pattern noise caused by the gain error of each column amplifier 34 without being affected by the environmental temperature.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described. Hereinafter, description of parts common to the first to third embodiments will be omitted, and description will be made focusing on characteristic parts of the present embodiment.

  The fourth embodiment is a mode suitable when the fixed pattern noise is caused by both the offset error and the gain error of each column amplifier 34.

  FIG. 13 is a schematic configuration diagram illustrating a configuration example of the CMOS sensor 31 according to the fourth embodiment. In FIG. 13, the same reference numerals are given to components common or similar to those in FIG. 2.

  As shown in FIG. 13, the fixed pattern noise removing circuit 80 includes an offset error correction data supply unit 96, a gain error correction data supply unit 92, a subtractor 86, a multiplier 94, and the like. In the example shown in FIG. 13, the multiplier 94 and the subtracter 86 are sequentially arranged at the subsequent stage of the A / D converter 37, but they may be arranged in the reverse order.

  The offset error correction data supply unit 96 has the same configuration as that of the first or second embodiment, similar to the gain error correction data supply unit 92 described above.

  According to the fourth embodiment, it is possible to reliably remove the fixed pattern noise caused by the offset error and gain error of each column amplifier 34 without being affected by the environmental temperature.

  In each of the above-described embodiments, the configuration in which the column amplifier 34 is provided as the processing circuit provided for each column in the CMOS sensor 31 is shown. However, the present invention is not limited thereto, and an A / D converter or other processing is provided for each column. The present invention can be suitably applied to a configuration provided with a circuit.

  As described above, the electronic endoscope and the fixed pattern noise removing method of the present invention have been described in detail. However, the present invention is not limited to the above examples, and various improvements and modifications can be made without departing from the gist of the present invention. Of course, you may also do.

  DESCRIPTION OF SYMBOLS 11 ... Electronic endoscope system, 12 ... Electronic endoscope, 13 ... Processor apparatus, 14 ... Light source device, 16 ... Insertion part, 16a ... Tip part, 17 ... Hand operation part, 26 ... Monitor, 31 ... CMOS sensor, 32 ... Imaging area, 33 ... Vertical scanning circuit, 34 ... Column amplifier, 37 ... A / D converter, 38 ... Horizontal scanning circuit, 41 ... Pixel, 42 ... TG, 52 ... CPU, 80 ... Fixed pattern noise removal circuit, 82 ... Temperature sensors 84, 84A, 84B, 84C ... Line memory, 86 ... Subtractor, 88 ... Line memory switching unit, 89 ... Coefficient determination unit, 92 ... Gain error correction data supply unit, 94 ... Offset error correction Data supply unit, 100 ... semiconductor chip, 102 ... effective pixel area, 104 ... invalid pixel area, 106 ... peripheral circuit area

Claims (11)

An electronic endoscope provided at a distal end of an insertion portion to be inserted into a subject and provided with a CMOS type image sensor for imaging the inside of the subject,
The image sensor is
A plurality of pixel portions arranged in a row direction and a column direction perpendicular thereto,
A plurality of column amplifiers provided in parallel for each of the pixel columns arranged in the column direction, each amplifying an output signal output from the pixel section arranged in the pixel column;
An A / D converter for converting signals amplified by the plurality of column amplifiers into digital signals;
A temperature sensor for detecting the temperature of the tip of the insertion portion;
Correction data corresponding to the temperature detected by the temperature sensor, a correction data supply unit for supplying correction data for offset error and correction data for gain error of the plurality of column amplifiers ;
The correction data based on the correction data supplied from the supply unit, e Bei and a correction circuit for removing fixed pattern noise contained in the signal output from the A / D converter,
The correction circuit is a circuit for removing fixed pattern noise caused by an offset error and a gain error as individual differences of the plurality of column amplifiers, wherein the gain error signal is added to a signal output from the A / D converter. The correction data for the offset error is subtracted from the multiplied signal,
The image sensor is an electronic endoscope that outputs a signal from which the fixed pattern noise has been removed by the correction circuit as an imaging signal . The plurality of pixel units, the plurality of column amplifiers, the A / D converter, the temperature sensor, the correction data supply unit, and the correction circuit are provided in the same semiconductor chip. Electronic endoscope.   The correction data supply unit includes a plurality of correction data for each predetermined temperature, and selects and supplies correction data corresponding to a temperature closest to the temperature detected by the temperature sensor among the plurality of correction data. The electronic endoscope according to claim 1 or 2.   The correction data supply unit includes reference correction data corresponding to a predetermined reference temperature, and corrects data obtained by multiplying the reference correction data by a correction coefficient determined according to a temperature detected by the temperature sensor. The electronic endoscope according to claim 1 or 2, which is supplied as data.   The electronic endoscope according to claim 4, wherein the correction data supply unit includes a correction coefficient table indicating a relationship between the temperature and the correction coefficient, and determines the correction coefficient based on the correction coefficient table.   The said temperature sensor is comprised using the photoelectric conversion element arrange | positioned in the invalid pixel part which does not contribute to the imaging in the said test object among these pixel parts. Electronic endoscope.   The electronic endoscope according to claim 6, wherein the temperature sensor uses, as temperature information, an average of output signals output from the plurality of invalid pixel units. A fixed pattern noise removal method for removing fixed pattern noise from an output signal of a CMOS type image sensor that is provided at a distal end of an insertion portion of an electronic endoscope to be inserted into a subject and images the inside of the subject. ,
The image sensor includes a plurality of pixel units arranged in a row direction and a column direction orthogonal to the row direction, and a plurality of pixel units arranged in parallel in the column direction. The image sensor includes the pixel units arranged in the pixel column. A plurality of column amplifiers that respectively amplify the output signals to be output, an A / D converter that converts the signals amplified by the plurality of column amplifiers into digital signals, and a temperature sensor that detects the temperature at the tip of the insertion portion A correction data supply unit that supplies correction data for offset errors and correction data for gain errors of the plurality of column amplifiers according to the temperature detected by the temperature sensor; and the correction data based on the correction data supplied from the supply unit, Bei example and a correction circuit for removing fixed pattern noise contained in the signal output from the a / D converter,
A detection step of detecting the temperature of the tip of the insertion portion by the image sensor and the temperature sensor provided integrally,
An output signal output from the A / D converter based on correction data corresponding to the temperature detected by the temperature sensor , the correction circuit being one of the peripheral circuits provided integrally with the image sensor. seen including a correction step, the removing fixed pattern noise contained in,
The correction step is a step of removing fixed pattern noise resulting from an offset error and a gain error as individual differences of the plurality of column amplifiers, wherein the gain error signal is added to the signal output from the A / D converter. The correction data for the offset error is subtracted from the multiplied signal,
The image sensor is a fixed pattern noise removal method for outputting a signal from which the fixed pattern noise has been removed in the correction step as an imaging signal . The correction data supply unit according to claim 8 to supply by selecting the correction data corresponding to the closest temperature to the temperature detected by the temperature sensor from a plurality of correction data provided for each predetermined temperature Fixed pattern noise removal method. The correction data supplying unit, supplying data obtained by multiplying the correction coefficient determined in accordance with the temperature sensor by the temperature detected for the reference correction data corresponding to a predetermined reference temperature and the corrected data The fixed pattern noise removing method according to claim 8 , wherein the fixed pattern noise is eliminated. The fixed pattern noise removal method according to claim 10 , wherein the correction data supply unit determines the correction coefficient based on a correction coefficient table indicating a relationship between the temperature and the correction coefficient.

Images