JP7439667B2 - Video signal processing device and video signal processing method - Google Patents

Description

The present invention relates to a video signal processing device and a video signal processing method that process video signals of three primary colors input from an image sensor.

Three-dimensional noise reduction (3DNR) is one of the signal processing methods for improving the quality of captured images. In three-dimensional noise reduction, noise is detected from frame differences between consecutive frame images, and the detected noise is removed.

In three-dimensional noise reduction, in order to suppress afterimages and color shifts, the difference values for each color (RGB) between the current frame image and the previous frame image are calculated, and based on the difference value of a certain color, the color is A method has been proposed in which the difference value of the relevant color is determined by interpolation even for pixels that do not have a color, and the cyclic coefficient is controlled based on the difference value of each color. Specifically, when the difference value of a certain color is large, the cyclic coefficient is decreased to reduce the noise removal value to suppress afterimages and color shift (for example, see Patent Document 1).

In the above method, when the difference value of a certain color becomes large, the difference value of other colors also becomes large due to interpolation, so the cyclic coefficient of any color becomes small, and the noise removal effect of any color is reduced.

Japanese Patent Application Publication No. 2010-147840

In three-dimensional noise reduction, if there is movement in an image, a large difference may occur in the amount of noise removed for each color. This difference in the amount of noise removed becomes a cause of hue shift and false color.

The present embodiment has been developed in view of these circumstances, and its purpose is to provide a technique for improving the quality of three-dimensional noise reduction.

In order to solve the above problems, a video signal processing device according to an aspect of the present embodiment is a video signal processing device that processes three primary color video signals input from an image sensor, and which processes a current frame image and a previous frame image. a frame difference signal generation unit that generates a frame difference signal; a noise extraction unit that extracts a noise component from the frame difference signal; and a level of a red component noise extraction signal and a blue component noise extracted by the noise extraction unit. a correction section for correcting a signal level; a green component noise extraction signal extracted by the noise extraction section; a red component noise extraction signal output from the correction section; and a blue component noise extraction signal output from the correction section. a coefficient multiplication unit that multiplies each of the signals by a cyclic coefficient; and a red component noise extraction signal and a green component noise extraction signal after multiplication by the cyclic coefficient from the red component signal, green component signal, and blue component signal of the current frame image. , and a noise removal unit that removes the blue component noise extraction signals, respectively. The correction unit brings the level of the red component noise extraction signal close to the level of the green component noise extraction signal when the absolute value of the difference between the level of the red component noise extraction signal and the green component noise extraction signal is greater than or equal to a first threshold value. Correction is performed to bring the level of the blue component noise extraction signal closer to the level of the green component noise extraction signal when the absolute value of the difference between the level of the blue component noise extraction signal and the green component noise extraction signal is greater than or equal to a second threshold value. I do.

Another aspect of this embodiment is a video signal processing method. This method is a video signal processing method that processes three primary color video signals input from an image sensor, generates a frame difference signal between a current frame image and a previous frame image, and extracts noise components from the frame difference signal. When the absolute value of the difference between the level of the red component noise extraction signal and the green component noise extraction signal is greater than or equal to a first threshold, correction is performed to bring the level of the red component noise extraction signal closer to the level of the green component noise extraction signal. , when the absolute value of the difference between the level of the blue component noise extraction signal and the green component noise extraction signal is greater than or equal to a second threshold, performing correction to bring the level of the blue component noise extraction signal closer to the level of the green component noise extraction signal; Each of the green component noise extraction signal, red component noise extraction signal, and blue component noise extraction signal is multiplied by a cyclic coefficient, and from the red component signal, green component signal, and blue component signal of the current frame image, after multiplication by the cyclic coefficient, The red component noise extraction signal, the green component noise extraction signal, and the blue component noise extraction signal are respectively removed.

Note that any combination of the above components and the expression of this embodiment converted between methods, devices, systems, recording media, computer programs, etc. are also effective as aspects of this embodiment.

According to this embodiment, the quality of three-dimensional noise reduction can be improved.

FIG. 2 is a diagram showing the configuration of an imaging system according to a comparative example. 1 is a diagram showing the configuration of an imaging system according to an example. FIG. 3 is a diagram showing a Bayer array. It is a figure showing an example of composition of a three-dimensional noise reduction part concerning an example. FIG. 7 is a diagram illustrating a specific image of amplitude range limiting processing performed by the first amplitude converter. FIG. 3 is a diagram showing an RGB-IR type color filter Cf.

FIG. 1 is a diagram showing the configuration of an imaging system 1 according to a comparative example. The imaging system 1 can be used, for example, as a drive recorder, a driver monitoring system, or the like. The imaging system 1 includes an imaging device 2 and a video signal processing device 3.

The image sensor 2 converts incident light into an electrical signal. As the image sensor 2, a solid-state image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Devices) image sensor is used. The image sensor 2 outputs a video signal defined by RAW data of three primary colors (RGB) to the video signal processing device 3.

The video signal processing device 3 includes an HDR (High Dynamic Range) synthesis section 10, a white balance adjustment section 20, a gamma correction section 30, a luminance/color difference conversion section 40, a three-dimensional noise reduction section 50, and a video output section 60. These components can be realized by cooperation of hardware and software resources, or only by hardware resources. Hardware resources include CPU, ROM, RAM, GPU (Graphics Processing Unit), DSP (Digital Signal Processor), ISP (Image Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and others. LSI can be used. Programs such as firmware can be used as software resources.

The HDR synthesis unit 10 generates a frame image with a wide dynamic range by synthesizing a plurality of frame images captured at different exposures. For example, the HDR synthesis unit 10 uses a frame image captured with high exposure for dark areas, uses a frame image captured with standard exposure for medium brightness areas, and uses a frame image captured with low exposure for bright areas. Use a frame image. This makes it possible to suppress overexposure and underexposure.

The white balance adjustment section 20 adjusts white balance according to the color temperature of the light source. For example, in the case of a cool color light source with a high color temperature (for example, outdoors on a cloudy day), the white balance is adjusted to increase the signal ratio of the blue component. On the other hand, in the case of a warm light source with a low color temperature (for example, an incandescent light bulb), the white balance is adjusted to increase the signal ratio of the red component. In the case of automatic setting, the white balance adjustment unit 20 analyzes the captured image and adjusts the white balance to the optimum white balance. In the case of manual setting, the white balance adjustment section 20 adjusts the white balance to the one set by the user.

The gamma correction section 30 corrects the gradation of the video signal. Generally, the input/output characteristics of a CMOS image sensor have a relationship of D=E ^γ (E is the amount of incident light and D is the amount of output signal). The gamma correction unit 30 corrects the video signal as shown in (Equation 1) below.
D'=D ^(1/γ) ...(Formula 1)

For example, 2.2 is used for γ. As a result, signals in areas where the signal level is low (dark) are boosted, and visibility is improved.

Note that the order of the white balance adjustment by the white balance adjustment section 20 and the gamma correction by the gamma correction section 30 may be reversed.

The luminance/color difference conversion unit 40 converts the video signal in the three primary color format into the video signal in the brightness/color difference format. For example, an RGB format video signal is converted to a YUV (YCbCr) format video signal.

The three-dimensional noise reduction unit 50 performs three-dimensional noise reduction on the video signal converted into the luminance/color difference format to remove noise.

The video output unit 60 converts the video signal into a signal compliant with the output format and outputs the signal. For example, when outputting using HDMI (registered trademark), the video output unit 60 converts the video signal into a video signal in the HDMI output format.

In this manner, in the comparative example, three-dimensional noise reduction is performed after HDR synthesis, white balance adjustment, gamma correction, and conversion to a luminance/color difference format. As described above, when the signal in the dark area is amplified by gamma correction, the noise in the dark area is also amplified at the same time. In this way, even if three-dimensional noise reduction is performed after the noise in the dark region is amplified by gamma correction, the effect of reducing the noise in the dark region will be limited.

FIG. 2 is a diagram showing the configuration of the imaging system 1 according to the embodiment. In the imaging system 1 according to the embodiment, the three-dimensional noise reduction section 50 is arranged before the gamma correction section 30. Further, in the embodiment, the three-dimensional noise reduction unit 50 performs three-dimensional noise reduction on the video signal in the three primary color format, rather than the video signal in the luminance/color difference format. Hereinafter, the configuration of the three-dimensional noise reduction unit 50 according to the embodiment will be described in detail. In the following description, it is assumed that video signals of three primary colors in a Bayer array are input from the image sensor 2.

FIG. 3 is a diagram showing a Bayer array. In the Bayer array, 4 (2×2) pixels forming one unit are arranged at a ratio of R:G:B=1:2:1. The human retina is more sensitive to green wavelengths, so green is arranged more often than red and blue. In the example shown in FIG. 3, the Bayer array is realized by color filters Cf. Bayer array color filters Cf are installed on a plurality of pixels included in the imaging region 2a of the image sensor 2.

A red filter transmits light in the red wavelength range and blocks light in other wavelength ranges. A green filter transmits light in the green wavelength range and blocks light in other wavelength ranges. A blue filter transmits light in the blue wavelength range and blocks light in other wavelength ranges. In the Bayer array, two green filters are placed on a diagonal, and a red filter and a blue filter are placed on another diagonal.

The light receiving element arranged in each pixel converts the light transmitted through the color filter Cf into an electrical signal. Note that three types of RGB light receiving elements may be arranged in a Bayer arrangement within the imaging area 2a without providing the color filter Cf. The image sensor 2 outputs a red component signal R0, a green component signal G0, a green component signal G1, and a blue component signal B1 from one unit of Bayer array.

In each pixel, two color component signals that do not exist are generated by interpolating a plurality of adjacent pixels. For example, a red component signal of a green pixel is generated by interpolating red component signals of a plurality of adjacent red pixels. A blue component signal of a green pixel is generated by interpolating blue component signals of a plurality of adjacent blue pixels. This interpolation process is executed in the luminance/color difference conversion section 40.

FIG. 4 is a diagram showing a configuration example of the three-dimensional noise reduction unit 50 according to the embodiment. A red component signal R0, a green component signal G0, a green component signal G1, and a blue component signal B1 of four pixels constituting the Bayer array are simultaneously input to the three-dimensional noise reduction unit 50.

The three-dimensional noise reduction unit 50 includes a first frame difference signal generation unit 51a, a second frame difference signal generation unit 51b, a third frame difference signal generation unit 51c, a fourth frame difference signal generation unit 51d, and a first noise extraction unit 52a. , second noise extraction section 52b, third noise extraction section 52c, fourth noise extraction section 52d, mixing section 53, first comparison section 54a, fourth comparison section 54d, first amplitude conversion section 55a, fourth amplitude conversion section 55d, first coefficient multiplication section 56a, second coefficient multiplication section 56b, third coefficient multiplication section 56c, fourth coefficient multiplication section 56d, first noise removal section 57a, second noise removal section 57b, third noise removal section 57c , a fourth noise removing section 57d, and a frame memory 58.

The frame memory 58 temporarily stores the red component signal R0, green component signal G0, green component signal G1, and blue component signal B1 after three-dimensional noise reduction by the three-dimensional noise reduction unit 50.

The first frame difference signal generation section 51a, the second frame difference signal generation section 51b, the third frame difference signal generation section 51c, and the fourth frame difference signal generation section 51d generate a red component signal R0 and a green component signal G0 of the current frame image. , green component signal G1, blue component signal B1, and the red component signal R0, green component signal G0, green component signal G1, and blue component signal B1 of the previous frame image delayed by one frame by the frame memory 58, respectively. generate.

In the configuration example shown in FIG. 4, the first frame difference signal generation section 51a, the second frame difference signal generation section 51b, the third frame difference signal generation section 51c, and the fourth frame difference signal generation section 51d generate the red The red component signal R0, green component signal G0, green component signal G1, and blue component signal B1 of the current frame image are subtracted from the component signal R0, green component signal G0, green component signal G1, and blue component signal B1, respectively, to obtain the red component. A frame difference signal r0d, a green component frame difference signal g0d, a green component frame difference signal g1d, and a blue component frame difference signal b1d are generated.

The red component frame difference signal r0d, the green component frame difference signal g0d, the green component frame difference signal g1d, and the blue component frame difference signal b1d each include a noise component and a motion component. Note that if the subject is stationary, no motion component is included.

The first noise extraction unit 52a, the second noise extraction unit 52b, the third noise extraction unit 52c, and the fourth noise extraction unit 52d each include a red component frame difference signal r0d, a green component frame difference signal g0d, a green component frame difference signal g1d, A red component noise extraction signal r0n, a green component noise extraction signal g0n, a green component noise extraction signal g1n, and a blue component noise extraction signal b1n are generated by removing the motion component and extracting the noise component from the blue component frame difference signal b1d, respectively. .

For example, the first noise extraction section 52a, the second noise extraction section 52b, the third noise extraction section 52c, and the fourth noise extraction section 52d each have a red component frame difference signal r0d, a green component frame difference signal g0d, a green component frame difference signal The amplitude levels of the frame difference signal g1d and the blue component frame difference signal b1d are limited to a predetermined range. Normally, the amplitude of the noise component is small and the amplitude of the motion component is large, so by limiting the amplitude, the noise component can be extracted effectively.

Moreover, the first noise extraction section 52a, the second noise extraction section 52b, the third noise extraction section 52c, and the fourth noise extraction section 52d each have a red component frame difference signal r0d, a green component frame difference signal g0d, a green component frame difference signal The frequency bands of the component frame difference signal g1d and the blue component frame difference signal b1d may be limited to a predetermined value or less. Normally, the frequency of noise components is high and the frequency of motion components is low, so by limiting the frequency band, the noise components can be extracted effectively.

In this specification, the mixing section 53, the first comparison section 54a, the fourth comparison section 54d, the first amplitude conversion section 55a, and the fourth amplitude conversion section 55d are collectively referred to as a correction section. The correction unit can correct the amplitude level of the red component noise extraction signal r0n and the amplitude level of the blue component noise extraction signal b1n.

The mixing unit 53 generates a reference green component noise extraction signal by mixing the green component noise extraction signal g0n and the green component noise extraction signal g1n. In the configuration example shown in FIG. 4, the mixing unit 53 generates the reference green component noise extraction signal ave(g0n, g1n) by simply averaging the green component noise extraction signal g0n and the green component noise extraction signal g1n. . Note that the mixing unit 53 may perform weighted addition of the green component noise extraction signal g0n and the green component noise extraction signal g1n to generate the reference green component noise extraction signal.

The first comparison unit 54a compares the red component noise extraction signal r0n and the reference green component noise extraction signal ave(g0n, g1n). Specifically, the first comparison unit 54a calculates the absolute value abs(dif_rg) of the difference dif_rg between the red component noise extraction signal r0n and the reference green component noise extraction signal ave(g0n, g1n). The first comparator 54a compares the absolute value abs(dif_rg) of the difference dif_rg with the first threshold r0n_th, and supplies the comparison result to the first amplitude converter 55a.

When the absolute value abs(dif_rg) of the difference dif_rg is less than the first threshold value r0n_th, the first amplitude converter 55a directly converts the red component noise extraction signal r0n into the red component noise extraction signal r0n_lim after amplitude range limitation without correcting it. Output as .

When the absolute value abs(dif_rg) of the difference dif_rg is equal to or greater than the first threshold value r0n_th, the first amplitude converter 55a converts the level of the red component noise extraction signal r0n into the green component noise extraction signal ave(g0n, g1n) for reference. Make corrections to bring it closer to the level of .

Specifically, when the absolute value abs(dif_rg) of the difference dif_rg is equal to or greater than the first threshold value r0n_th and the difference dif_rg is equal to or greater than zero, the first amplitude conversion unit 55a converts the reference green component noise extraction signal ave(g0n , g1n) and the first threshold value r0n_th is output as the red component noise extraction signal r0n_lim after amplitude range limitation.

When the absolute value abs(dif_rg) of the difference dif_rg is greater than or equal to the first threshold value r0n_th and the difference dif_rg is less than zero, the first amplitude converter 55a converts the reference green component noise extraction signal ave(g0n, g1n) into The signal obtained by subtracting one threshold value r0n_th is output as the red component noise extraction signal r0n_lim after amplitude range limitation.

FIG. 5 is a diagram showing a specific image of the amplitude range limiting process performed by the first amplitude converter 55a. Centering on the reference green component noise extraction signal ave (g0n, g1n), the value obtained by adding the first threshold value r0n_th to the reference green component noise extraction signal ave (g0n, g1n) is set as the upper limit value r0n_lim_upper of the amplitude range. The value obtained by subtracting the first threshold value r0n_th from the reference green component noise extraction signal ave (g0n, g1n) is set as the lower limit value r0n_lim_under of the amplitude range.

The first amplitude converter 55a limits the red component noise extraction signal r0n that deviates from the amplitude range to a signal within the amplitude range. The first amplitude conversion unit 55a corrects the red component noise extraction signal r0n, which exceeds the upper limit value r0n_lim_upper of the amplitude range, to the upper limit value r0n_lim_upper. Note that the first amplitude conversion unit 55a may correct the red component noise extraction signal r0n to a value closer to the reference green component noise extraction signal ave(g0n, g1n) than the upper limit value r0n_lim_upper.

Further, the first amplitude conversion unit 55a corrects the red component noise extraction signal r0n, which is below the lower limit value r0n_lim_under of the amplitude range, to the lower limit value r0n_lim_under. Note that the first amplitude converter 55a may correct the red component noise extraction signal r0n to a value closer to the reference green component noise extraction signal ave(g0n, g1n) than the lower limit value r0n_lim_under.

Return to Figure 4. The fourth comparison unit 54d compares the blue component noise extraction signal b1n and the reference green component noise extraction signal ave(g0n, g1n). Specifically, the fourth comparison unit 54d calculates the absolute value abs(dif_bg) of the difference dif_bg between the blue component noise extraction signal b1n and the reference green component noise extraction signal ave(g0n, g1n). The fourth comparison section 54d compares the absolute value abs(dif_bg) of the difference dif_bg with the second threshold b1n_th, and supplies the comparison result to the fourth amplitude conversion section 55d.

When the absolute value abs(dif_bg) of the difference dif_bg is less than the second threshold b1n_th, the fourth amplitude converter 55d converts the blue component noise extraction signal b1n into the blue component noise extraction signal b1n_lim after amplitude range limitation without correcting it. Output as .

When the absolute value abs(dif_bg) of the difference dif_bg is greater than or equal to the second threshold b1n_th, the fourth amplitude converter 55d converts the level of the blue component noise extraction signal b1n into the green component noise extraction signal ave(g0n, g1n) for reference. Make corrections to bring it closer to the level of .

Specifically, when the absolute value abs(dif_bg) of the difference dif_bg is equal to or greater than the second threshold value b1n_th and the difference dif_bg is equal to or greater than zero, the fourth amplitude conversion unit 55d converts the reference green component noise extraction signal ave(g0n , g1n) and the second threshold value b1n_th is output as the blue component noise extraction signal b1n_lim after amplitude range limitation.

When the absolute value abs(dif_bg) of the difference dif_bg is greater than or equal to the second threshold b1n_th and the difference dif_bg is less than zero, the fourth amplitude converter 55d converts the reference green component noise extraction signal ave(g0n, g1n) into The signal obtained by subtracting the two threshold values b1n_th is output as the blue component noise extraction signal b1n_lim after amplitude range limitation.

The first threshold r0n_th and the second threshold b1n_th are each set by the designer based on at least one of the designer's knowledge, experimental results, and simulation results. The smaller the values of the first threshold r0n_th and the second threshold b1n_th are set, the closer the levels of the noise extraction signals of the red component, green component, and blue component can be.

The first coefficient multiplier 56a multiplies the amplitude range limited red component noise extraction signal r0n_lim output from the first amplitude converter 55a by a first cyclic coefficient kr0 (0<kr0<1). The second coefficient multiplier 56b multiplies the green component noise extraction signal g0n extracted by the second noise extractor 52b by a second cyclic coefficient kg0 (0<kg0<1). The third coefficient multiplier 56c multiplies the green component noise extraction signal g1n extracted by the third noise extractor 52c by a third cyclic coefficient kg1 (0<kg1<1). The fourth coefficient multiplier 56d multiplies the amplitude range limited blue component noise extraction signal b1n_lim output from the fourth amplitude converter 55d by a fourth cyclic coefficient kb1 (0<kb1<1).

In this embodiment, fixed values are set for each of the first cyclic coefficient kr0, the second cyclic coefficient kg0, the third cyclic coefficient kg1, and the fourth cyclic coefficient kb1. You can also set different values for each color to adjust the noise removal sensitivity for each color. Note that fluctuation values that adaptively change depending on the video analysis result may be set for each of the first cyclic coefficient kr0, the second cyclic coefficient kg0, the third cyclic coefficient kg1, and the fourth cyclic coefficient kb1.

The first noise remover 57a, the second noise remover 57b, the third noise remover 57c, and the fourth noise remover 57d each output a red component signal R0, a green component signal G0, a green component signal G1, and a blue component of the current frame image. The red component noise extraction signal r0n_lim, the green component noise extraction signal g0n, the green component noise extraction signal g1n, and the blue component noise extraction signal b1n_lim after multiplication by the cyclic coefficient are removed from the signal B1.

In the configuration example shown in FIG. 4, the first frame difference signal generation section 51a, the second frame difference signal generation section 51b, the third frame difference signal generation section 51c, and the fourth frame difference signal generation section 51d generate the red The red component signal R0, green component signal G0, green component signal G1, and blue component signal B1 of the current frame image are subtracted from the component signal R0, green component signal G0, green component signal G1, and blue component signal B1, respectively. In this case, the first noise removal section 57a, the second noise removal section 57b, the third noise removal section 57c, and the fourth noise removal section 57d each output a red component signal R0, a green component signal G0, and a green component signal G1 of the current frame image. , the red component noise extraction signal r0n_lim after multiplication by the cyclic coefficient, the green component noise extraction signal g0n, the green component noise extraction signal g1n, and the blue component noise extraction signal b1n_lim are added to the blue component signal B1, respectively.

On the contrary, the first frame difference signal generation section 51a, the second frame difference signal generation section 51b, the third frame difference signal generation section 51c, and the fourth frame difference signal generation section 51d generate the red component signal R0 and the green component signal of the current frame image. When the red component signal R0, green component signal G0, green component signal G1, and blue component signal B1 of the previous frame image are subtracted from the component signal G0, green component signal G1, and blue component signal B1, the first noise removal unit 57a, The second noise removal unit 57b, the third noise removal unit 57c, and the fourth noise removal unit 57d calculate the cyclic coefficient from the red component signal R0, green component signal G0, green component signal G1, and blue component signal B1 of the current frame image. The multiplied red component noise extraction signal r0n_lim, green component noise extraction signal g0n, green component noise extraction signal g1n, and blue component noise extraction signal b1n_lim are each subtracted.

As explained above, according to this embodiment, when the difference between the green component noise extraction signal and the red component noise extraction signal or the blue component noise extraction signal is large, the red component noise extraction signal or the blue component noise extraction signal is Perform correction to bring it closer to the green component noise extraction signal. Thereby, it is possible to suppress the occurrence of hue shifts and false colors in areas where movement occurs in the video.

Furthermore, in this embodiment, the green component noise extraction signal is excluded from the amplitude conversion target. As a result, the green component signal, which makes a large contribution to the brightness, is not subjected to amplitude conversion more than necessary, and the influence on the noise removal effect on the brightness signal can be reduced.

Further, in this embodiment, a three-dimensional noise reduction section 50 is arranged before the gamma correction section 30. Thereby, three-dimensional noise reduction can be performed before noise in dark areas is amplified by gamma correction. By performing gamma correction after noise removal, noise in dark areas can also be effectively reduced.

The present invention has been described above based on examples. Those skilled in the art will understand that this example is merely an example, and that various modifications can be made to the combinations of these components and processing processes, and that such modifications are also within the scope of the present invention. . Furthermore, the expressions of the present invention replaced with methods, devices, systems, recording media, computer programs, etc. are also effective as aspects of the present invention.

In the embodiments described above, an example has been described in which the average value of the green component noise extraction signals of two pixels forming the Bayer array is used as the reference green component noise extraction signal. In this regard, the reference green component noise extraction signal may be generated from the green component noise extraction signal of one pixel.

FIG. 6 is a diagram showing an RGB-IR type color filter Cf. The image sensor 2 using this color filter Cf can also receive infrared light. In this example, the green component noise extraction signal g0n is used as it is as the reference green component noise extraction signal. In this way, the red noise extraction signal of a red pixel or the blue noise extraction signal of a blue pixel is a reference green component noise extraction signal generated based on the noise extraction signal of one or more adjacent green pixels. It only needs to be compared with the signal.

1 Imaging System, 2 Imaging Element, 3 Video Signal Processing Device, 10 HDR Combining Unit, 20 White Balance Adjustment Unit, 30 Gamma Correction Unit, 40 Luminance/Color Difference Conversion Unit, 50 Three-dimensional Noise Reduction Unit, 51a, 51b, 51c, 51d frame difference signal generation section, 52a, 52b, 52c, 52d noise extraction section, 53 mixing section, 54a, 54d comparison section, 55a, 55d amplitude conversion section, 56a, 56b, 56c, 56d coefficient multiplication section, 57a, 57b, 57c, 57d noise removal section, 58 frame memory, 60 video output section.

Claims (5)

A video signal processing device that processes three primary color video signals input from an image sensor,
a frame difference signal generation unit that generates a frame difference signal between the current frame image and the previous frame image;
a noise extraction unit that extracts a noise component from the frame difference signal;
a correction unit for correcting the level of the red component noise extraction signal and the level of the blue component noise extraction signal extracted by the noise extraction unit;
a coefficient multiplication unit that multiplies each of the green component noise extraction signal extracted by the noise extraction unit, the red component noise extraction signal output from the correction unit, and the blue component noise extraction signal output from the correction unit by a cyclic coefficient; and,
a noise removal unit that removes a red component noise extraction signal, a green component noise extraction signal, and a blue component noise extraction signal after multiplication by the cyclic coefficient from the red component signal, green component signal, and blue component signal of the current frame image; , comprising:
The correction unit is
When the absolute value of the difference between the level of the red component noise extraction signal and the green component noise extraction signal is greater than or equal to a first threshold, performing correction to bring the level of the red component noise extraction signal closer to the level of the green component noise extraction signal;
When the absolute value of the difference between the level of the blue component noise extraction signal and the green component noise extraction signal is greater than or equal to a second threshold, performing correction to bring the level of the blue component noise extraction signal closer to the level of the green component noise extraction signal;
Video signal processing device. The video signal processing device receives a video signal of three primary colors in a Bayer array from the image sensor, and
The correction unit is
Generate a reference green component noise extraction signal by mixing two green component noise extraction signals constituting one Bayer array,
When the absolute value of the difference between the level of the red component noise extraction signal constituting the Bayer array and the reference green component noise extraction signal is greater than or equal to the first threshold, the level of the red component noise extraction signal is set to the reference green component noise extraction signal. Performs correction to bring the level of the green component noise extraction signal close to that of the green component noise extraction signal, outputs the corrected red component noise extraction signal, and when the absolute value of the difference is less than the first threshold, corrects the level of the red component noise extraction signal. output without
When the absolute value of the difference between the level of the blue component noise extraction signal constituting the Bayer array and the reference green component noise extraction signal is greater than or equal to the second threshold, the level of the blue component noise extraction signal is set to the reference green component noise extraction signal. Performs correction to bring the level of the green component noise extraction signal close to that of the green component noise extraction signal, outputs the corrected blue component noise extraction signal, and when the absolute value of the difference is less than the second threshold, corrects the level of the blue component noise extraction signal. output without
The video signal processing device according to claim 1. The correction unit is
When the absolute value of the difference between the level of the red component noise extraction signal constituting the Bayer array and the reference green component noise extraction signal is greater than or equal to the first threshold, and when the difference is greater than or equal to zero, the reference A signal obtained by adding the first threshold to the green component noise extraction signal is output as a corrected red component noise extraction signal, and when the difference is less than zero, the first threshold is added to the reference green component noise extraction signal The signal after subtracting is output as the corrected red component noise extraction signal,
When the absolute value of the difference between the level of the blue component noise extraction signal constituting the Bayer array and the reference green component noise extraction signal is greater than or equal to the second threshold, and when the difference is greater than or equal to zero, the reference A signal obtained by adding the second threshold to the green component noise extraction signal of is output as a corrected blue component noise extraction signal, and when the difference is less than zero, the second threshold is added to the reference green component noise extraction signal. The signal after subtracting is output as the corrected blue component noise extraction signal,
The video signal processing device according to claim 2. further comprising a gamma correction section installed after the noise removal section and correcting the gradation of the three primary color video signals;
The video signal processing device according to any one of claims 1 to 3. A video signal processing method for processing three primary color video signals input from an image sensor, the method comprising:
Generate a frame difference signal between the current frame image and the previous frame image,
extracting a noise component from the frame difference signal;
When the absolute value of the difference between the level of the red component noise extraction signal and the green component noise extraction signal is greater than or equal to a first threshold, performing correction to bring the level of the red component noise extraction signal closer to the level of the green component noise extraction signal;
When the absolute value of the difference between the level of the blue component noise extraction signal and the green component noise extraction signal is greater than or equal to a second threshold, performing correction to bring the level of the blue component noise extraction signal closer to the level of the green component noise extraction signal;
Multiply each of the green component noise extraction signal, red component noise extraction signal, and blue component noise extraction signal by a cyclic coefficient,
removing a red component noise extraction signal, a green component noise extraction signal, and a blue component noise extraction signal after multiplication by the cyclic coefficient from the red component signal, green component signal, and blue component signal of the current frame image, respectively;
Video signal processing method.

Images