JP3702222B2 - Imaging apparatus and video signal processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging apparatus having an imaging element that images a subject and outputs a video signal, and a video signal processing method applied to the apparatus.
[0002]
[Prior art]
An imaging apparatus used as a video camera or the like uses an imaging device such as a CCD (charge coupled device) and performs predetermined signal processing (KNEE, gamma, detail, white balance, etc.) on a video signal obtained from the imaging device. The final video signal is obtained by applying.
[0003]
However, an imaging element such as a CCD generally has a narrow dynamic range, and it is difficult to faithfully image a subject with a large luminance difference between a dark part and a bright part. As a technique to alleviate such problems, there is known a method for expanding the dynamic range by adding (synthesizing) two or more (different brightness) video signals obtained by imaging a subject under different imaging conditions at an appropriate ratio. ing.
[0004]
In addition to the above, various methods as described below are known as methods for expanding the dynamic range.
[0005]
For example, in Japanese Patent Laid-Open No. 11-313247, when a high-luminance image signal obtained by adding together image unit pixels and adjacent surrounding pixels and an original image signal that has passed through a delay circuit are synthesized, By increasing the proportion of the original image when the luminance level of the image is high, and increasing the proportion of the high luminance image when the luminance level of the original image is low, it is possible to brighten dark portions from one image, and A method for expanding the dynamic range without reducing the resolution in a bright part is disclosed.
[0006]
Japanese Patent Laid-Open No. 2000-228747 discloses that each image in an image group captured under different exposure conditions is divided into an appropriate exposure region and an inappropriate exposure region based on a predetermined image signal level, and divided appropriate exposure. A method is disclosed in which tone correction of a region is performed for each image, and an appropriate exposure region for each tone-corrected image is synthesized to generate one wide dynamic range image.
[0007]
Japanese Patent Laid-Open No. 2000-307921 discloses that a signal of an optical image picked up by a CCD is compressed and temporarily stored before being processed by an image processing unit that performs digital processing. In a configuration in which it is read out, decompressed / restored, digitally processed by the image processing unit and stored in a memory, a multi-gradation image is obtained by imaging with a greater number of gradations than can be stored in the memory. The image signal with different tones can be reproduced from the stored data by compressing and storing the signal, and the photographer is appropriate from a plurality of images with different exposure conditions, whether immediately after shooting or after shooting. A method for easily selecting an image in an intended exposure state and reproducing an image having an appropriate dynamic range is disclosed.
[0008]
[Problems to be solved by the invention]
Of the various methods described above, the method that adds (synthesizes) two or more types of video signals obtained by imaging the subject under different imaging conditions is to narrow down the aperture for high-brightness video signals that would saturate with a normal aperture. The video signal that is not saturated is added (synthesized), and there is an advantage that an image can be taken without saturating a dark part to a bright part of the subject.
[0009]
However, with this method, the level at which the video signal corresponding to the low-speed shutter is compressed is the same on the entire screen. Therefore, when there is a particularly bright part (high luminance part) in the screen, or a bright part and a dark part in a backlight state When there is a portion corresponding to a subject having a very large luminance difference, there is a problem that sufficient contrast cannot be obtained for that portion.
[0010]
The present invention has been made in view of the above circumstances, and is applied to an imaging apparatus and apparatus capable of obtaining sufficient contrast even when a high-luminance portion or a portion corresponding to a subject having a very large luminance difference is present on the screen. An object is to provide an imaging method.
[0011]
[Means for Solving the Problems]
An image pickup apparatus according to the present invention increases an exposure amount by an image pickup device that picks up a subject and outputs a video signal, an exposure amount control unit that controls an exposure amount to the image pickup device, and the exposure amount control unit. The first video signal output from the imaging device when the image is taken is input, and the level of the first reference value or higher among the signals of the individual image areas included in the first video signal A compression unit that outputs a signal after level-compressing a signal of the image region for each image region, and a second video signal output from the image sensor when the exposure amount control unit reduces the exposure amount and captures an image. Input, compressing the level of an image area signal having a level lower than the second reference value among the signals of the individual image areas included in the second video signal, and not less than the second reference value Image area with level Compression / expansion means for outputting a signal after level-extending the signal for each image area, and addition for outputting the signal outputted from the compression means and the signal outputted from the compression / expansion means for each image area. And a change in level of a signal of an image area having a level equal to or higher than the first reference value included in the first video signal output from the imaging device, for each image area, By changing the level compression rate in the compression means, Control means for changing the ratio of the two signals before being added by the adding means; Then, the control means increases the signal level of the image area having a level equal to or higher than the first reference value included in the first video signal output from the image sensor from the compression means. Means for controlling so that the level of the output signal is further lowered It is characterized by that.
[0013]
The image pickup apparatus according to the present invention increases an exposure amount by an image pickup device that picks up an image of a subject and outputs a video signal, an exposure amount control unit that controls an exposure amount to the image pickup device, and the exposure amount control unit. The first video signal output from the image sensor when the image is picked up is input, and the level of the first reference value or higher among the signals of the individual image areas included in the first video signal. Compression means for level-compressing a signal of an image area for each image area and outputting the signal, and a second video output from the image sensor when the exposure amount control means reduces the exposure amount and images the image area A signal is input, and among the signals of the individual image regions included in the second video signal, the signal of the image region having a level less than the second reference value is level-compressed and the second reference value A picture with the above level A compression / expansion unit that outputs a signal after level-extending the signal of each region for each image region, and a signal output from the compression unit and a signal output from the compression / expansion unit are added for each image region and output. Adding means, a first multiplication means for multiplying the signal output from the compression means by a signal corresponding to the first coefficient, and a signal output from the compression / expansion means for a signal corresponding to the second coefficient. In accordance with a level change of a signal of an image area having a level equal to or higher than the first reference value included in the first video signal output from the image sensor. And control means for changing the ratio of the two signals before being added by the adding means by changing the ratio between the first coefficient and the second coefficient for each time. The control means, the higher the level of the signal of the image area having the level equal to or higher than the first reference value included in the first video signal output from the image sensor, the higher the first Means for controlling the ratio of the second coefficient to the coefficient to be increased It is characterized by that.
[0015]
The video signal processing method according to the present invention is a video signal processing method applied to an imaging apparatus having an imaging device for imaging a subject and outputting a video signal. Of the individual image area signals included in the first video signal output from the image sensor, the image area signal having a level equal to or higher than the first reference value is level-compressed for each image area, and exposure is performed. A signal of an image area having a level less than a second reference value among signals of individual image areas included in the second video signal output from the image sensor when imaging is performed with a reduced amount. Level-compressing and level-extending the signal of the image area having a level equal to or higher than the second reference value for each image area, and the level-compressed signal and the level-compressed and level-extended signal for each image area Addition The level compression is performed for each image area in accordance with a level change of a signal in the image area having a level equal to or higher than the first reference value included in the first video signal output from the image sensor. By changing the level compression ratio, the ratio of the two signals before the addition processing is changed. The level of the signal output from the compression means is higher as the level of the signal in the image area having the level equal to or higher than the first reference value included in the first video signal output from the image sensor is higher. Is controlled to be lower It is characterized by that.
[0016]
The video signal processing method according to the present invention is a video signal processing method applied to an imaging apparatus having an imaging device for imaging a subject and outputting a video signal. Of the individual image area signals included in the first video signal output from the image sensor, the image area signal having a level equal to or higher than the first reference value is level-compressed for each image area, and exposure is performed. A signal of an image area having a level less than a second reference value among signals of individual image areas included in the second video signal output from the image sensor when imaging is performed with a reduced amount. Level-compressing and level-extending the signal of the image area having a level equal to or higher than the second reference value for each image area, and the level-compressed signal and the level-compressed and level-extended signal for each image area Addition And multiplying the level-compressed signal by the signal corresponding to the first coefficient, and multiplying the compressed / decompressed signal by the signal corresponding to the second coefficient, and outputting the signal from the image sensor. By changing the ratio between the first coefficient and the second coefficient in accordance with the level change of the signal of the image area having a level equal to or higher than the first reference value included in the first video signal, Change the ratio of the two signals before they are added The higher the level of the signal in the image area having the level equal to or higher than the first reference value included in the first video signal output from the image sensor, the higher the second coefficient for the first coefficient. Control the coefficient ratio to be large It is characterized by that.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0018]
(First embodiment)
FIG. 1 is a block diagram showing a configuration of an imaging apparatus according to the first embodiment of the present invention.
[0019]
In FIG. 1, reference numeral 1 denotes a lens diaphragm for adjusting the exposure amount of imaging input to the imaging apparatus. The lens diaphragm 1 sets the exposure amount for imaging in multiple stages. The imaging whose exposure amount is adjusted by the lens diaphragm (exposure amount adjusting means) 1 is guided to the imaging device (CCD) 2 and is photoelectrically converted by this imaging device 2 into an analog video signal.
[0020]
The image sensor 2 has an electronic shutter function, and multi-stage shutter speeds can be obtained. Either the aperture value of the lens aperture 1 or the shutter speed of the image sensor 2 can be used for selecting the exposure condition. Here, it is assumed that the exposure amount adjustment of the image set with the high-speed shutter is performed by the shutter speed control, and the exposure amount adjustment of the image set with the low-speed shutter is performed by the aperture value control of the lens aperture 1.
[0021]
Analog video signals captured under different exposure conditions in the image sensor 2 are converted into digital video signals by a sample / hold circuit (S / H) 3 and an analog / digital converter (A / D) 4, and a signal processing circuit 5 and predetermined video signal processing (KNEE, gamma, detail, white balance, etc.) is performed.
[0022]
The signal processing circuit 5 extracts brightness data of a video signal captured with a small exposure amount and a large exposure amount from the video signal output from the A / D 4 and outputs the brightness data to the brightness detection circuit 14. To do. The brightness detection circuit 14 detects the integrated value or peak value of the brightness of the entire screen or a predetermined area from the input brightness data, and outputs it to the control circuit 13.
[0023]
The control circuit 13 is constituted by a CPU or the like, and based on the brightness detection data from the brightness detection circuit 14, controls the aperture value of the lens aperture 1, controls the shutter speed of the image sensor 2, and a selector switch 7 described later. Is switched. In this case, the aperture value of the lens aperture 1 is controlled based on the brightness data of the image signal captured with a large exposure amount, and the shutter speed of the image sensor 2 is captured with a small exposure amount. It is desirable to control based on the brightness data of the video signal.
[0024]
Further, the control circuit 13 determines a reference level REF1 for controlling the compression circuit 8 to be described later and a reference level REF2 for controlling the compression / decompression circuit 9, and controls the compression rate and the expansion rate. Hereinafter, these controls are referred to as REF1 control and REF2 control, respectively.
[0025]
The reference level REF1 determined by the control circuit 13 is not sent directly to the compression circuit 8, but is sent after a predetermined process is performed by the control circuit 15 described later.
[0026]
The video signal S1 output from the signal processing circuit 5 is input to a one-screen memory 6 that stores one screen of an image. The one-screen memory 6 delays the input video signal S1 by one screen, and the delayed video signal S2 is supplied to the fixed terminals a1 and b2 of the changeover switch 7. The video signal S1 output from the signal processing circuit 5 is also supplied directly to the fixed terminals b1 and a2 of the changeover switch 7 without being delayed.
[0027]
The one-screen memory 6 includes a first memory block and a second memory block each having a storage capacity equivalent to a video signal for one screen, and two switches for switching input / output of these two memories. Reading / writing of the first memory block and the second memory block and switching of the two switches are performed based on a memory control signal from the control circuit 13.
[0028]
Here, a video signal with a large exposure amount captured with the shutter speed set to a low speed is defined as VS, and a video signal with a small exposure amount captured with the shutter speed set at a high speed is defined as VF.
[0029]
The changeover switch 7 is controlled by the control circuit 13 so that the movable terminals c1 and c2 are alternately connected to the fixed terminals a1 and b1 and a2 and b2 at an appropriate timing. As a result, the video signal VS with a large exposure amount is output from the terminal c1, and the video signal VF with a small exposure amount is output from the terminal c2.
[0030]
The video signal VS output from the terminal c1 of the changeover switch 7 is input to the compression circuit 8, and the compression circuit 8 performs compression processing in units of pixels based on the control signal from the control circuit 15. This compressed video signal is designated as VS ′. The video signal VF output from the terminal c2 is input to the compression / expansion circuit 9, and the compression / expansion circuit 9 performs compression / expansion processing in units of pixels based on the control signal from the control circuit 13. . This compressed / decompressed video signal is designated as VF ′.
[0031]
The compression rate in the compression circuit 8 is controlled by the control circuit 15 that receives the REF1 control signal. Hereinafter, this control is referred to as REF1 ′ control. The compression / decompression rate in the compression / decompression circuit 9 is REF2 controlled by the control circuit 13 as described above. Details of the REF1 control, the REF1 ′ control, and the REF2 control will be described later.
[0032]
The video signal VS ′ output from the compression circuit 8 and the video signal VF ′ output from the compression / decompression circuit 9 are added by the adder 10 and then input to the NTSC encoder 11 and encoded into the NTSC signal. It is converted into an analog video signal by a digital / analog converter (D / A) 12 and output as a final video signal.
[0033]
In the control circuit 13, the reference levels REF1 and REF2 can be varied in order to control the ratio of combining the video signals VS ′ and VF ′ in accordance with the brightness detection data. That is, when the brightness detection circuit 14 detects from the input brightness data that the image has a small exposure amount, the control circuit 13 increases the reference levels REF1 and REF2, and is obtained from the adder 10. Control is performed so that the video signal includes many video signals VS ′ obtained by the low-speed shutter. With this configuration, an optimum contrast can be obtained from a dark subject with a small dynamic range.
[0034]
When the brightness detection circuit 14 detects from the input brightness data that the image has a large amount of exposure, the control circuit 13 decreases the reference levels REF 1 and REF 2 and obtains from the adder 10. Control is performed so that a large number of video signals VF ′ obtained by a high-speed shutter are included in the obtained video signals. With this configuration, an optimum contrast can be obtained even from a bright subject having a large dynamic range.
[0035]
Incidentally, the video signal VS output from the terminal c1 of the changeover switch 7 is also input to the control circuit 15. The control circuit 15 determines whether the level of the video signal VS is higher than or equal to the reference level REF1 in real time for each pixel. If there is a level higher than REF1, the higher the level of the video signal VS, the higher the compression. REF1 'having a lower level than REF1 received from the control circuit 13 is supplied so that the compression rate in the circuit 8 is increased (so that the level of the video signal VS' is further lowered).
[0036]
With this configuration, when there is a particularly bright part (high brightness part) on the screen, or when there is a part corresponding to a subject in which the brightness difference between the bright part and the dark part is extremely large in a backlight state, Since the ratio of the video signal VF ′ to the signal VS ′ is further increased, a sufficient contrast can be obtained for that portion.
[0037]
FIG. 2 is a block diagram illustrating an example of an internal configuration of the control circuit 15 in the imaging apparatus of FIG.
[0038]
The control circuit 15 shown in FIG. 2A includes an adder 16, an adder 17, a multiplier 18, a switch 19, a limiter circuit 20, a filter circuit 21, and a comparator 22.
[0039]
The adder 16, the adder 17, and the multiplier 18 are portions that perform arithmetic processing corresponding to the case where the level of the video signal VS is equal to or higher than the reference level REF1. This arithmetic processing has a characteristic that when the level of the video signal VS is equal to or higher than the reference level REF1, the higher the level of the video signal VS, the lower the signal level at the subsequent stage of the switch 19.
[0040]
The comparator 22 compares the image signal VS picked up with an increased exposure amount with a reference level REF1 for controlling the compression amount, and switches and controls the switch 19 according to the comparison result. The switch 19 switches the signal input destination according to the output signal of the comparator 22.
[0041]
The limiter circuit 20 limits the level of an input signal so as not to be lower than a predetermined value, and prevents the image from becoming unnatural due to REF1 ′ being lowered more than necessary. The limiter circuit 20 is driven only when the contact point a of the switch 19 and the setting c are in a connected state.
[0042]
The filter circuit 21 is realized, for example, as a high-frequency cut filter for signals, and adjusts so that the signal level difference between adjacent pixels does not exceed a predetermined value. The filter circuit 21 prevents an image from becoming unnatural due to a sudden change in brightness.
[0043]
When the signals VS and REF 1 are input to the control circuit 15, the comparison is performed by the comparator 22 for each pixel in the control circuit 15. In accordance with the comparison result, the following arithmetic processing is performed, and the value of the reference level REF1 ′ to be sent to the compression circuit 8 is calculated.
[0044]
When VS ≧ REF1
REF1 '= (REF1- (VS-REF1)) * K
When VS <REF1
REF1 '= REF1
Here, K is a coefficient corresponding to the slope of the attenuation portion of REF1 ′, and is set to an optimum value according to the imaging condition (use) of the imaging apparatus (in the experiment, the range is 0.5 to 1). Confirmed to be good).
[0045]
That is, when the level of VS is REF1 or more, the contact a and the contact c of the switch 19 are connected by the output of the comparator 22. At this time, arithmetic processing is performed by the adder 16, adder 17, and multiplier 18 based on the input VS, REF1, and coefficient K, and a signal corresponding to the arithmetic processing result is output from the contact c of the switch 19. The The signal output from the switch 19 is subjected to limit processing by the limiter circuit 20, filtered by the filter circuit 21, and then output as REF1 '. The outputted REF 1 ′ is sent to the compression circuit 8.
[0046]
On the other hand, when the level of VS is less than REF1, the contact b and the contact c of the switch 19 are connected by the output of the comparator 22. At this time, the inputted REF1 is outputted from the contact c of the switch 19 as it is. The signal output from the switch 19 passes through without being processed by the limiter circuit 20, is filtered by the filter circuit 21, and then is output as REF1 '. The outputted REF 1 ′ is sent to the compression circuit 8.
[0047]
In the configuration example of FIG. 2A, the case where the limiter circuit 20 is provided after the switch 19 has been described. Instead, the limiter circuit 20 is switched to the switch 19 as in the configuration example of FIG. You may make it provide in the front | former stage of the (contact point a side). In this case, control for driving / stopping the limiter circuit 20 according to the switching state of the switch 19 is not required.
[0048]
FIG. 3 is a diagram illustrating signal waveforms of respective units in the imaging apparatus of FIG. The horizontal axis corresponds to the amount of light incident on the image sensor 2, and the vertical axis corresponds to the output level of the electrical signal.
[0049]
FIG. 3A shows the waveforms of the signals VS and VF output from the changeover switch 7. Note that the white clip value corresponding to the maximum value of the dynamic range is WC, and the signals VS and VF never exceed WC.
[0050]
FIG. 3B shows the waveform of the signal VS ′ output from the compression circuit 8.
[0051]
When the level of the signal VS is less than REF1, the reference level REF1 ′ having the same level as REF1 is applied. In this case, a signal VS ′ having the same level as the signal VS is generated.
[0052]
On the other hand, when the level of the signal VS is equal to or higher than REF1, the reference level REF1 ′ having a level lower than REF1 is applied. In this case, the higher the level of the signal VS, the lower the reference level REF1 ′, and the signal VS ′ having the same level as the REF1 ′ is generated. The slope of the waveform at this time is determined by the coefficient K. Further, the reference level REF1 ′ that decreases due to this inclination is limited by the limiter circuit 20 so as not to be less than a predetermined value (the reference level REF1 ′ is kept at a constant level no matter how high the level of the signal VS). A signal VS ′ having the same level as REF1 ′ is generated.
[0053]
FIG. 3C shows how the signal VF input to the compression / decompression circuit 9 is processed. That is, in the compression / decompression circuit 9, a portion (shaded portion in the figure) obtained by subtracting REF2 from the signal VF is extracted, and the extracted portion is amplified (gain correction). The amplified signal is output from the compression / decompression circuit 9 as a signal VF ′.
In other words, the compression / expansion circuit 9 performs level compression on the portion below REF2 in the signal VF, and outputs the signal after level expansion on the portion above REF2.
[0054]
FIG. 3D shows the output signal VS ′ of the compression circuit 8 (the signal obtained in FIG. 3B) and the output signal VF ′ of the compression / decompression circuit 9 (the signal obtained in FIG. 3C). (Amplified (Gain Correction)) is added.
[0055]
That is, the signal in the shaded area obtained in FIG. 3C is amplified (expanded) in the compression / decompression circuit 9 to be a signal corresponding to the shaded area in FIG. Will be added.
[0056]
In the compression / decompression circuit 9, when the signal in the hatched portion obtained in FIG. 3C is amplified, the level of the signal after addition is changed by changing the amplification factor according to the level of VF. It is desirable to adjust appropriately. In the example of FIG. 3 (d), the first amplification factor is applied in the range (in the horizontal axis direction) from the point where VS ′ starts to fall together with REF1 ′ to the point where the lowering is stopped by the limiter 20. By applying a second gain smaller than the first gain in the range after the point where the descent is stopped by (in the horizontal axis direction), adjustment is made so that the signal after addition does not exceed WC as much as possible. ing. The range in which the first amplification factor is applied and the range in which the second amplification factor is applied can be known from the control circuit 15.
[0057]
FIG. 4 is a diagram for explaining the filtering process of the filter 21 in the control circuit 15 of FIG.
[0058]
FIG. 4A shows a state in which a subject is imaged from the room with the camera facing the window. There is a person standing in the room, and a strong light from outside is shining on the part with the face. The outside is brighter than the room, and there are trees outside.
[0059]
FIG. 4B shows the change in gradation when the image is obtained using the filter 21 for the line portion shown in FIG. 4A, and the level when the image is obtained without using the filter 21. The tone change is shown in FIG.
[0060]
When the filter 21 is used, REF1 ′ (output of the filter 21) does not change in a small area such as a bright part of the face, and brightness (luminance) gradation changes in a part of the face, which is unnatural. It will never be. In addition, even in a portion where the luminance difference is large, such as a window border or a tree outline, the brightness gradation does not change suddenly and does not become unnatural. When the gradation changes suddenly in a specific part, the human eye is emphasized and seen. In the present embodiment, the use of the filter 21 can remove gradation change in a portion with a small area, and can smoothly change gradation in a portion with a large area, so that an image that looks more natural can be obtained. it can.
[0061]
Next, the operation of the control circuit 15 according to the present embodiment will be described with reference to FIG.
[0062]
When the signals VS and REF1 are input to the control circuit 15, they are compared for each pixel (step A1).
[0063]
In step A2, when the level of the signal VS is equal to or higher than REF1 (Yes in step A2), the higher the level of the signal VS, the lower the reference level REF1 ′ is generated (step A3). The reference level REF1 ′ thus generated is subjected to a limiter process so as not to be less than a predetermined value (step A4).
[0064]
On the other hand, if the level of the signal VS does not reach REF1 in Step A2 (No in Step A2), a reference level REF1 ′ having the same level as REF1 is generated (Step A5).
[0065]
The reference level REF1 ′ processed / generated in step A4 or step A5 is filtered for each pixel so that the brightness does not change suddenly and the image is not unnatural (step A6), and is compressed by the control circuit 15. It is sent to the circuit 8 (step A7).
[0066]
As a result, the compression circuit 8 outputs the signal VS ′ based on the reference level REF1 ′ received from the control circuit 15.
[0067]
As described above, according to the first embodiment, the control circuit 15 determines whether the level of the video signal VS is equal to or higher than the reference level REF1 for each pixel. The higher the level of VS, the higher the compression rate in the compression circuit 8 (so that the level of the video signal VS ′ is further lowered), so that the level of REF1 is lower than that of REF1 received from the control circuit 13. ′ Is supplied. With this configuration, when there is a particularly bright part (high brightness part) on the screen, or when there is a part corresponding to a subject in which the brightness difference between the bright part and the dark part is extremely large in a backlight state, Since the ratio of the video signal VF ′ to the signal VS ′ is further increased, a sufficient contrast can be obtained for that portion. Furthermore, by providing the limiter circuit 20 and the filter circuit 21, it becomes possible to obtain an image with further improved quality.
[0068]
(Second Embodiment)
By the way, when the control circuit 13 controls REF1 and REF2, if the relationship between the two cannot be maintained as ideal, the gradation continuity of the signals VS and VF before being added (synthesized) by the adder 10 is increased. There is a problem that the quality of the signal after the addition is deteriorated. In particular, since the signals VS and VF are processed by a non-linear circuit (a circuit that performs gamma processing or the like) included in the signal processing circuit 5, the brightness detection result also fluctuates. It becomes difficult to control REF2 as ideal.
[0069]
Here, as shown in FIG. 6A, consider a case where an error occurs in REF2 among REF1 for controlling the signal VS and REF2 for controlling the signal VF.
[0070]
FIG. 6B shows a case where an error of “−Δ” occurs in REF2, and FIG. 6C shows a case where an error of “+ Δ” occurs in REF2. In any case, due to the error of REF2, an error also occurs in the signal VF ′ to be added. Even when the error of REF2 is eliminated and the waveform as shown in FIG. 6D is realized, it is difficult to achieve complete gradation continuity.
[0071]
In an actual product, REF1 and REF2 control data for brightness and the like detected based on actually measured data is held in a ROM, and REF1 and REF2 are determined by reading the data. However, if the resolution is not sufficient due to the ROM capacity limitation, the error of REF1 and REF2 increases. In addition, errors due to characteristic variations in the CCD and various circuits can be considered. In actual products, control is performed within an error range that does not cause any practical problems, but the quality is not sufficient.
[0072]
Therefore, in the second embodiment, a method of maintaining the continuity of the gradations of the signals VS and VF before being added (synthesized) by the adder 10 and improving the quality of the signal after the addition will be described. .
[0073]
FIG. 7 is a block diagram showing a configuration of an imaging apparatus according to the second embodiment of the present invention. Elements common to those in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted. Hereinafter, a description will be given focusing on parts different from FIG.
[0074]
In the first embodiment described above, the case where the control circuit 15 is provided has been described. However, in the second embodiment, the control circuit 31 is provided instead, and the first multiplier 29 and the second multiplier 30 are provided.
[0075]
The first multiplier 29 multiplies the signal output from the compression circuit 8 by a signal corresponding to the first coefficient (G1 control signal), and outputs a signal VS1. The second multiplier 30 multiplies the signal output from the compression / expansion circuit by a signal corresponding to the second coefficient (G2 control signal), and outputs a signal VF1. When the adder 10 adds the signal VS1 and the signal VF1, the addition result is output to the encoder 11 as the signal VMIX.
[0076]
The video signal VS output from the terminal c 1 of the changeover switch 7 is also input to the control circuit 31. The control circuit 31 determines whether the level of the video signal VS is higher than or equal to the reference level REF1 in real time for each pixel. If there is a level higher than REF1, the higher the level of the video signal VS is, the higher the level is. Based on a predetermined calculation formula so that the ratio of the second coefficient G2 to the first coefficient G1 is increased, the G1 control signal and the G2 control signal respectively corresponding to the set G1 and G2 are set to the first multiplier. 29, supplied to the second multiplier 30.
[0077]
With this configuration, when there is a particularly bright part (high brightness part) on the screen, or when there is a part corresponding to a subject in which the brightness difference between the bright part and the dark part is extremely large in a backlight state, Since the ratio of the video signal VF1 to the signal VS1 is further increased, not only can a sufficient contrast be obtained with respect to that portion, but also smooth addition processing between signals can be realized without impairing the continuity of gradation. The quality of the image can be further improved.
[0078]
FIG. 8 is a block diagram showing an example of the internal configuration of the control circuit 31 in the imaging apparatus of FIG.
[0079]
The control circuit 31 shown in FIG. 8 includes an adder 32, a multiplier 33, an adder 34, a switch 35, and a comparator 36.
[0080]
The adder 32, the multiplier 33, and the adder 34 are portions that perform arithmetic processing corresponding to the case where the level of the video signal VS is equal to or higher than the reference level REF1. In this arithmetic processing, when the level of the video signal VS is equal to or higher than the reference level REF1, the higher the level of the video signal VS, the lower the level of the signal G1 output to the subsequent stage of the switch 35 and the level of the signal G2 (However, G1 + G2 = 1).
[0081]
The comparator 36 compares the image signal VS picked up with an increased exposure amount with a reference level REF1 for controlling the compression amount, and switches and controls the switch 35 according to the comparison result. The switch 35 switches the signal input destination according to the output signal of the comparator 36.
[0082]
When the signals VS and REF1 are input to the control circuit 31, a comparison is performed by the comparator 36 for each pixel in the control circuit 31. Depending on the comparison result, the following arithmetic processing is performed to calculate the first coefficient G1 and the second coefficient G2 to be sent to the first multiplier 29 and the first multiplier 30, respectively. .
[0083]
When VS ≧ REF1
G = B × (VS-REF1)
G1 = 1-G
G2 = G
When VS <REF1
G1 = 1
G2 = 0
Here, B is a correction coefficient for determining the curve of the signal VS1, and is set to be optimal by experiment or the like (in the experiment, it was confirmed that the range of 0.5 to 1.5 was favorable. ).
[0084]
That is, when the level of VS is REF1 or more, the contact a1 and the contact c1 of the switch 35 are connected by the output of the comparator 36, and the contact a2 and the contact c2 are connected. At this time, arithmetic processing by the adder 32, multiplier 33, and adder 34 is performed based on the input VS, REF1, coefficient B, and the like, and a signal corresponding to the arithmetic processing result is represented by the contact c1 of the switch 19 and c2 are output as G1 and G2, respectively. The output G1 and G2 are sent to the first multiplier 29 and the first multiplier 30, respectively. What is important here is that control is performed so that G1 + G2 = 1. Thus, as the level of the signal VS increases, the addition of the signal VF1 to the signal VS1 is performed smoothly (a little by little).
[0085]
On the other hand, when the level of VS is less than REF1, the contact b1 and the contact c1 of the switch 35 are connected by the output of the comparator 36, and the contact b2 and the contact c2 are connected. At this time, a signal having a value of 1 is output as G1, and a signal having a value of 0 is output as G2. The output G1 and G2 are sent to the first multiplier 29 and the first multiplier 30, respectively.
[0086]
FIG. 9 is a diagram illustrating signal waveforms of respective units in the imaging apparatus of FIG. The horizontal axis corresponds to the amount of light incident on the image sensor 2, and the vertical axis corresponds to the output level of the electrical signal.
[0087]
FIG. 9A shows the waveforms of the signals VS and VF output from the changeover switch 7. Note that the white clip value corresponding to the maximum value of the dynamic range is WC, and the signals VS and VF never exceed WC.
[0088]
FIG. 9B shows the waveform of the signal VS ′ output from the compression circuit 8 and the waveform of the VF ′ after gain correction output from the compression / expansion circuit 9 (shaded portion in the figure). In the figure, for the sake of simplicity, a case where the gain correction amount in the compression / expansion circuit 9 is 1 is shown.
[0089]
FIG. 9C shows the addition of the signal VMIX to the signal VS1 after the signal VS ′ is multiplied by G1 and the signal VF1 (the hatched portion in the figure) after the signal VF ′ is multiplied by G2. It shows how to get. As can be seen from the signal waveform in the figure, it is possible to eliminate the gray level discontinuity portion (see FIGS. 6B and 6C) that may occur in the past. It is also possible to control the level of the signal VS1 to be equal to or lower than REF1 by appropriately adjusting the value of the correction coefficient B shown in the above formula. That is, by controlling so that the addition ratio of VF1 to VS1 is increased for brighter portions, more signals that are not saturated with respect to the bright portions can be added, and the contrast of the bright portions of the screen can be improved.
[0090]
In addition, depending on the value of the correction coefficient B, the amount of attenuation of VS1 is too large with respect to REF1, and the originally bright part becomes very dark (a tone reversal occurs), and the whole image is felt unnatural. There is. In order to cope with this, as shown in FIG. 9D, it is desirable to limit the level of the signal VS1 by using a limiter so that it does not become less than a predetermined value.
[0091]
Next, the operation of the control circuit 31 of this embodiment will be described with reference to FIG.
[0092]
When the signals VS and REF1 are input to the control circuit 31, they are compared for each pixel (step B1).
[0093]
In step B2, if the level of the signal VS is equal to or higher than REF1 (Yes in step B2), the higher the level of the signal VS is, the higher the ratio of the signal VF1 to the signal VS1 is. , G1 + G2 = 1) is generated based on a predetermined calculation formula (step B3).
[0094]
On the other hand, if the level of the signal VS does not reach REF1 in step B2 (No in step B2), a coefficient G1 having a value of 1 and a coefficient G2 having a value of 0 are generated (step B4).
[0095]
The coefficients G1 and G2 generated in step B3 or step B4 are sent from the control circuit 31 to the multiplier 29 and the multiplier 30, respectively (step B5).
[0096]
As a result, the multiplier 29 multiplies the signal VS ′ by the coefficient G1 and outputs the signal VS1, while the multiplier 30 multiplies the signal VF ′ by the coefficient G2 and outputs the signal VF1. Addition processing of VS1 and signal VF1 is performed by the adder 10.
[0097]
As described above, according to the second embodiment, the control circuit 31 determines whether the level of the video signal VS is higher than the reference level REF1 in real time for each pixel. The higher the level of the video signal VS is, the higher the ratio of the second coefficient G2 to the first coefficient G1 is set based on a predetermined calculation formula, and G1 control corresponding to the set G1 and G2 respectively. The signal and the G2 control signal are supplied to the first multiplier 29 and the second multiplier 30. With this configuration, if there is a particularly bright part (high brightness part) on the screen, or if there is a part corresponding to a subject in which the brightness difference between the bright part and the dark part is extremely large in a backlight state, Since the ratio of the video signal VF1 to the signal VS1 is further increased, not only can a sufficient contrast be obtained with respect to that portion, but also a smooth addition process between signals can be realized without impairing the continuity of gradation. The quality of the image can be further improved.
[0098]
Note that the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention.
[0099]
【The invention's effect】
As described above in detail, according to the present invention, a sufficient contrast can be obtained even if a high-luminance portion or a portion corresponding to a subject having a very large luminance difference exists on the screen.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an imaging apparatus according to a first embodiment of the present invention.
2 is a block diagram showing an example of the internal configuration of a control circuit 15 in the imaging apparatus of FIG.
3 is a diagram showing signal waveforms at various parts in the imaging apparatus of FIG. 1. FIG.
4 is a diagram for explaining filtering processing of a filter in the control circuit 15 of FIG. 2;
FIG. 5 is a flowchart for explaining an operation by the control circuit 15 of the embodiment;
FIG. 6 is a diagram for explaining a problem of gradation discontinuity in addition processing.
FIG. 7 is a block diagram showing a configuration of an imaging apparatus according to a second embodiment of the present invention.
8 is a block diagram showing an example of the internal configuration of a control circuit 31 in the imaging apparatus of FIG.
9 is a diagram showing signal waveforms at various parts in the imaging apparatus shown in FIG. 7;
FIG. 10 is a flowchart for explaining an operation by the control circuit 31 of the embodiment;
[Explanation of symbols]
1 ... Lens aperture
2 ... Image sensor
3. Sample hold circuit
4. Analog / digital converter
5. Signal processing circuit
6. One screen memory
7 ... Changeover switch
8 ... Compression circuit
9: Compression / decompression circuit
10 ... Adder
11 ... NTSC encoder
12 ... Digital / analog converter
13 ... Control circuit
14 ... Brightness detection circuit
15 ... Control circuit
16 ... adder
17 ... Adder
18 ... Multiplier
19 ... Switch
20 ... Limiter circuit
21 ... Filter circuit
22 ... Comparator
29 ... 1st multiplier
30 ... second multiplier
31 ... Control circuit
32 ... Adder
33 ... Multiplier
34 ... Adder
35 ... Switch
36 ... Comparator

Claims (6)

An image sensor that images a subject and outputs a video signal;
Exposure amount control means for controlling the exposure amount to the image sensor;
A first video signal output from the image sensor when the exposure amount is controlled by the exposure amount control means is input, and signals of individual image areas included in the first video signal are input. Among them, compression means for outputting a signal after level-compressing a signal of an image area having a level equal to or higher than a first reference value for each image area;
A second video signal output from the imaging device is input when imaging is performed with a reduced exposure amount by the exposure amount control means, and signals of individual image regions included in the second video signal are input. Among them, the compression of level-compressing a signal in an image area having a level less than the second reference value and outputting the signal after expanding the level of the signal in the image area having a level higher than the second reference value for each image area. Stretching means;
Adding means for adding and outputting the signal output from the compression means and the signal output from the compression / expansion means for each image region;
The level compression rate in the compression means for each image area according to the level change of the signal of the image area having a level equal to or higher than the first reference value included in the first video signal output from the image sensor the by changing, and control means for varying the ratio of the two signals before being summed by the adding means,
The control unit outputs the signal from the compression unit as the level of the signal in the image region having a level equal to or higher than the first reference value included in the first video signal output from the imaging device is higher. An image pickup apparatus comprising means for controlling the signal level to be further reduced . An image sensor that images a subject and outputs a video signal;
Exposure amount control means for controlling the exposure amount to the image sensor;
A first video signal output from the image sensor when the exposure amount is controlled by the exposure amount control means is input, and signals of individual image areas included in the first video signal are input. Among them, compression means for outputting a signal after level-compressing a signal of an image area having a level equal to or higher than a first reference value for each image area;
A second video signal output from the imaging device is input when imaging is performed with a reduced exposure amount by the exposure amount control means, and signals of individual image regions included in the second video signal are input. Among them, the compression of level-compressing a signal in an image area having a level less than the second reference value and outputting the signal after expanding the level of the signal in the image area having a level higher than the second reference value for each image area. Stretching means;
Adding means for adding and outputting the signal output from the compression means and the signal output from the compression / expansion means for each image region;
First multiplying means for multiplying a signal output from the compression means by a signal corresponding to a first coefficient;
Second multiplication means for multiplying a signal output from the compression / expansion means by a signal corresponding to a second coefficient;
The first coefficient and the first coefficient for each image area according to a level change of a signal of an image area having a level equal to or higher than the first reference value included in the first video signal output from the image sensor. Control means for changing the ratio of the two signals before being added by the adding means by changing the ratio with the coefficient of 2 .
The control means increases the signal level of the image region having a level equal to or higher than the first reference value included in the first video signal output from the image sensor, with respect to the first coefficient. An image pickup apparatus comprising: means for controlling the ratio of the second coefficient to be increased . The control means includes means for controlling the added value of the first coefficient and the second coefficient to be constant when changing the ratio of the first coefficient and the second coefficient. The imaging apparatus according to claim 2 . The individual image areas, the imaging apparatus according to any one of claims 1 to 3, characterized in that correspond to individual pixels in the image pickup device. A video signal processing method applied to an imaging device having an imaging device that images a subject and outputs a video signal,
Signals of image areas having a level equal to or higher than the first reference value among the signals of the individual image areas included in the first video signal output from the image sensor when the exposure amount is increased. Level compression for each image area,
A signal of an image area having a level less than a second reference value among signals of individual image areas included in the second video signal output from the image pickup device when imaged with a reduced exposure amount And level-extending a signal of an image area having a level equal to or higher than the second reference value for each image area,
The level-compressed signal and the level-compressed and level-extended signal are added for each image area,
The level compression rate in the level compression for each image area according to the level change of the signal of the image area having a level equal to or higher than the first reference value included in the first video signal output from the image sensor. By changing the ratio of the two signals before the addition process ,
The higher the level of the signal in the image area having the level equal to or higher than the first reference value included in the first video signal output from the image sensor, the higher the level of the signal output from the compression means. A video signal processing method, characterized by performing control so as to be further reduced . A video signal processing method applied to an imaging device having an imaging device that images a subject and outputs a video signal,
Signals of image areas having a level equal to or higher than the first reference value among the signals of the individual image areas included in the first video signal output from the image sensor when the exposure amount is increased. Level compression for each image area,
A signal of an image area having a level less than a second reference value among signals of individual image areas included in the second video signal output from the image pickup device when imaged with a reduced exposure amount And level-extending a signal of an image area having a level equal to or higher than the second reference value for each image area,
The level-compressed signal and the level-compressed and level-extended signal are added for each image area,
Multiplying the level-compressed signal by a signal corresponding to a first coefficient, and multiplying the compressed / decompressed signal by a signal corresponding to a second coefficient;
The first coefficient and the second coefficient according to a level change of a signal in an image area having a level equal to or higher than the first reference value included in the first video signal output from the image sensor. By changing the ratio, the ratio of the two signals before the addition processing is changed ,
The higher the level of the signal in the image area having the level equal to or higher than the first reference value included in the first video signal output from the imaging device, the higher the second coefficient relative to the first coefficient. A video signal processing method characterized in that control is performed so that the ratio is increased .

Images