JP3967453B2 - Image encoding method and image encoding apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention encodes and compresses image data obtained from an image sensor.Image codingThe present invention relates to a method and an apparatus.
[0002]
[Prior art]
FIG. 22 is a block diagram of a conventional image signal compression coding apparatus disclosed in, for example, Japanese Patent Laid-Open No. 6-54308. The image signal input in FIG. 22 is converted into a frequency component by using the discrete cosine transform (DCT) in the orthogonal transform unit 37 and then input to the noise removal unit 38. The noise removing unit 38 removes noise included in the image signal using the characteristics of the frequency component, and the DCT coefficient after the noise removal is input to the bit allocation unit 39 and the quantization unit 40. The quantization unit 40 quantizes the DCT coefficient according to the number of quantization bits output from the bit allocation unit 39. The quantized DCT coefficient is encoded by the encoding unit 41 and output as an encoded signal.
[0003]
[Problems to be solved by the invention]
This type of conventional apparatus has been configured as described above. By the way, in order to realize high-speed image recording such as a continuous shooting function in a still image pickup device, a method of reducing the data transfer and recording speed by compressing the image data from the image pickup device and reducing the data size is effective. It is. In the image encoding method used in such an imaging apparatus, it is necessary to transfer image data at a constant rate, and therefore the code data length needs to be fixed. Further, since image data is encoded from the image sensor, it is necessary to minimize image quality degradation.
[0004]
However, since image data from the image sensor includes noise, it is necessary to remove the noise in order to obtain an appropriate quantization level in the encoding process. For example, in the image signal compression coding apparatus disclosed in Japanese Patent Laid-Open No. 6-54308, noise removal processing is performed in coding processing, but noise removal is performed using frequency components, so bit allocation after noise removal is performed. Depending on the case, the image quality such as mosquito noise may be deteriorated in the decoded image. In addition, in order to perform high-speed image recording such as a continuous shooting function, simpler and faster image encoding processing and noise removal processing are necessary.
[0005]
The present invention has been made in order to solve the above-described problems, and realizes encoding processing for performing noise removal processing in image encoding processing with a simpler and faster algorithm, and at the same time, visual image quality degradation is reduced. It is an object of the present invention to provide a method and apparatus for image coding that can obtain a small number of decoded images.
[0007]
[Means for Solving the Problems]
  In view of the above object, the present invention provides a block dividing step of dividing image data obtained from an image sensor into M × N (M and N are natural numbers) blocks, and a maximum value Lmax and a minimum pixel value in the block. A signal level range calculating step for calculating the value Lmin, a quantization level calculating step for determining a quantization level of the pixel value from the obtained Lmax and Lmin, and each pixel value in the block from the obtained quantization level. From the quantization process to quantize, Lmax, Lmin and the quantized value of each pixel valueCalculate the absolute value of the convolution integral value using the pattern matrix, and based on the magnitude of the obtained absolute valueNoise detection process for detecting noise and pixels detected as noiseAbout before quantizationThe image encoding method is characterized by comprising a noise removing step of correcting the pixel value of the above and a block encoding step of performing encoding in the block after noise removal.
  Also, the image data obtained from the image sensor is M × N. ( M and N are natural numbers ) A block dividing step for dividing the block into blocks, a signal level range calculating step for calculating the maximum value Lmax and the minimum value Lmin of the pixel value in the block, and a quantum for determining the quantization level of the pixel value from the obtained Lmax and Lmin A quantization level calculation step, a quantization step for quantizing each pixel value in the block from the obtained quantization level, a result of counting the number of pixels included in each quantization level in the block, and a threshold value A noise detection step for detecting noise by the relationship, a noise removal step for correcting a pixel value before quantization for a pixel detected as noise, and a block coding step for performing coding in the block after noise removal, An image encoding method characterized by comprising:
[0008]
Further, according to the present invention, in the noise detection step, a pixel number counting step for counting the number of pixels included in each quantization level in the block, and a pixel included in a level smaller than a threshold value as a result of the counting is determined as noise. And a threshold value determining step.
[0009]
Further, according to the present invention, in the noise detection step, a pixel count step for counting the number of pixels included in each quantization level in the block, and a pixel for selecting a pixel included in a level smaller than a threshold value as a result of the counting The image encoding method includes a selection step and a noise determination step of determining, as noise, a pixel in which a pixel having a predetermined relationship with a neighboring pixel does not exist.
[0010]
Further, according to the present invention, in the noise detection step, a pixel count step for counting the number of pixels included in each quantization level in the block, and a pixel for selecting a pixel included in a level smaller than a threshold value as a result of the counting The image encoding method includes a selection step and a noise determination step of determining, as noise, a pixel in which a pixel having a pixel value close to a neighboring pixel does not exist in the selected pixel.
[0011]
Further, according to the present invention, in the noise detection step, a pixel count step for counting the number of pixels included in each quantization level in the block, and a pixel for selecting a pixel included in a level smaller than a threshold value as a result of the counting An image coding characterized by comprising a selection step, and a noise determination step for determining, as noise, a pixel in which a pixel having a monotonously increasing or monotonic decreasing relationship with a pixel value of a target pixel does not exist among neighboring pixels in the selected pixel Is in the way.
[0012]
Further, according to the present invention, in the noise detection step, a pixel count step for counting the number of pixels included in each quantization level in the block, and a pixel for selecting a pixel included in a level smaller than a threshold value as a result of the counting In the selection step, a pixel in which there is no pixel that has a monotonous increase or decrease in the pixel value of the pixel of interest in the neighboring pixel and a pixel that does not have a pixel value close to the neighboring pixel is determined as noise. The image coding method is characterized by comprising a noise determination step.
[0013]
Further, according to the present invention, the block coding step includes a second signal level range calculating step for recalculating Lmax and Lmin in the block after noise removal, and a second quantization for calculating a quantization level from the obtained Lmax and Lmin. An image encoding method comprising: a level calculation step, and an encoding step of encoding after quantizing each pixel value in the block from the obtained quantization level.
[0015]
  The present invention also calculates block dividing means for dividing the image data obtained from the image sensor into M × N (M and N are natural numbers) blocks, and a maximum value Lmax and a minimum value Lmin of pixel values in the block. Signal level range calculation means, quantization level calculation means for determining the quantization level of the pixel value from the obtained Lmax and Lmin, and quantization for quantizing each pixel value in the block from the obtained quantization level Means, Lmax, Lmin and the quantized value of each pixel valueCalculate the absolute value of the convolution integral value using the pattern matrix, and based on the magnitude of the obtained absolute valueNoise detection means for detecting noise and pixels detected as noiseAbout before quantizationThe image coding apparatus is characterized by comprising noise removing means for correcting the pixel value of the first block and block coding means for performing coding in the block after noise removal.
  Also, the image data obtained from the image sensor is M × N. ( M and N are natural numbers ) Block dividing means for dividing the block into blocks, signal level range calculating means for calculating the maximum value Lmax and minimum value Lmin of the pixel value in the block, and a quantum for determining the quantization level of the pixel value from the obtained Lmax and Lmin Quantization level calculation means, quantization means for quantizing each pixel value in the block from the obtained quantization level, a result of counting the number of pixels included in each quantization level in the block, and a threshold value A noise detection unit that detects noise according to a relationship; a noise removal unit that corrects a pixel value before quantization for a pixel detected as noise; a block coding unit that performs coding in the block after noise removal; An image encoding apparatus characterized by comprising:
[0016]
Further, according to the present invention, the noise detecting means counts the number of pixels included in each quantization level in the block, and determines that pixels included in a level smaller than a certain threshold as noise are counted as noise. The image coding apparatus is characterized by comprising a threshold value determination unit.
[0017]
Further, according to the present invention, the noise detecting means is a pixel number counting unit for counting the number of pixels included in each quantization level in the block, and a pixel for selecting a pixel included in a level smaller than a threshold value as a result of counting. An image encoding apparatus comprising: a selection unit; and a noise determination unit that determines, as noise, a pixel in which a pixel having a predetermined relationship with a neighboring pixel does not exist.
[0018]
Further, according to the present invention, the noise detecting means is a pixel number counting unit for counting the number of pixels included in each quantization level in the block, and a pixel for selecting a pixel included in a level smaller than a threshold value as a result of counting. The image coding apparatus includes a selection unit and a noise determination unit that determines, as noise, a pixel in which a pixel having a pixel value close to a neighboring pixel does not exist.
[0019]
Further, according to the present invention, the noise detecting means is a pixel number counting unit for counting the number of pixels included in each quantization level in the block, and a pixel for selecting a pixel included in a level smaller than a threshold value as a result of counting. An image coding comprising: a selection unit; and a noise determination unit configured to determine, as noise, a pixel in which a pixel having a monotonously increasing or monotonic decreasing relationship with a pixel value of a target pixel does not exist in a neighboring pixel. In the device.
[0020]
Further, according to the present invention, the noise detecting means is a pixel number counting unit for counting the number of pixels included in each quantization level in the block, and a pixel for selecting a pixel included in a level smaller than a threshold value as a result of counting. The selection unit determines, as noise, a pixel in which a pixel that does not have a monotonically increasing or decreasing relationship with the pixel value of the pixel of interest in the selected pixel and a pixel that does not have a pixel value close to the neighboring pixel do not exist. The image coding apparatus is characterized by comprising a noise determination unit.
[0021]
Also, the present invention provides a second signal level range calculation unit in which the block encoding means recalculates Lmax and Lmin in a block after noise removal, and a second quantization in which a quantization level is calculated from the obtained Lmax and Lmin. An image encoding apparatus comprising: a level calculation unit; and an encoding unit that performs encoding after quantizing each pixel value in the block from the obtained quantization level.
[0023]
DETAILED DESCRIPTION OF THE INVENTION
Embodiment 1 FIG.
FIG. 1 is a diagram showing a processing flow of an image encoding method according to an embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of the image coding apparatus according to this embodiment. Each functional block in FIG. 2 is assigned with the reference numerals of the processes in FIG. The apparatus according to the present invention is basically constituted by a computer that performs processing according to a program, and this is the same in the following embodiments.
[0024]
In FIG. 2, the image coding apparatus includes a block dividing unit 1, a signal level range calculating unit (1, 2), a quantization level calculating unit (4, 5, 6), a quantizing unit 7a, a noise detecting unit 7b, and a noise removing unit. Means 8 and block encoding means 9;
[0025]
In this embodiment, in step 1 of FIG. 1, the image data is divided into 4 × 4 blocks as shown in FIG. Next, each block divided in step 2 of FIG. 1 is read one by one, and processing is performed for each block. In the processing of each block, as shown in step 3 of FIG. 1, first, the maximum value Lmax and the minimum value Lmin related to the pixel value of the pixels included in the block are calculated. Next, P1 and P2 are calculated in step 4 of FIG. 1 from the obtained Lmax and Lmin. Here, the calculation of P1 and P2 is performed using equations (1) and (2).
[0026]
P1 = (Lmax + 3Lmin) / 4 (1)
P2 = (3Lmax + Lmin) / 4 (2)
[0027]
Next, Q1 and Q4 are calculated using the obtained P1 and P2 as shown in Step 5 of FIG. Here, Q1 is an average value of pixel values of pixels having a pixel value smaller than P1 in the block, and Q4 is an average value of pixel values of pixels having a pixel value larger than P2 in the block.
[0028]
Next, in step 6 of FIG. 1, La and Ld are calculated from the obtained Q1 and Q4. Here, La is a representative value of the pixel value in the block, and is calculated using Expression (3). Ld is a range of pixel values in the block, and is calculated using Expression (4).
[0029]
La = (Q1 + Q4) / 2 (3)
Ld = Q4-Q1 (4)
[0030]
Next, noise included in the image is detected in step 7 of FIG. Noise detection is performed as follows. First, the quantization value of each pixel in the block is calculated using P1, La, and P2 as threshold values. For example, when the pixel value of the target pixel is larger than P2, the quantization value of the target pixel is (Lmax + P2) / 2, and when the pixel value of the target pixel is larger than La and smaller than or equal to P2, the quantization value of the target pixel To (P2 + La) / 2. When the pixel value of the target pixel is greater than P1 and equal to or less than La, the quantization value of the target pixel is (La + P1) / 2, and when the pixel value of the target pixel is equal to or less than P1, the target pixel is quantized. The value is set to (P1 + Lmin) / 2.
[0031]
Next, using the pattern matrix as shown in FIG. 4, the absolute value of the convolution integral value with the matrix of FIG. 4 is calculated for the quantized pixel value. Note that the pattern matrix in FIG. 4 is centered on the pixel of interest. Next, when the obtained absolute value is equal to or greater than a predetermined threshold value, the pixel is determined as noise. Here, the threshold value is a value determined by the noise level included in the image, and generally differs depending on the image sensor.
[0032]
For the pixels determined as noise in step 7 of FIG. 1, noise removal processing (noise correction processing) is performed in step 8 of FIG. The noise removal process is performed by replacing the pixel value of the pixel with an average value regarding the neighboring pixels of the pixel. The neighboring pixels referred to in this embodiment are as shown in FIG. In FIG. 5, the pixel of interest is in the center of the matrix, and the shaded pixel is a neighboring pixel that is referenced. Further, in the case of referring to neighboring pixels in FIGS. 4 and 5 of this embodiment, when referring to beyond the boundary of the block, necessary pixel data is read out appropriately to refer to the pixel value. For example, by performing processing using a line buffer of 4 lines, it is possible to easily refer to the left and right pixels of the matrix beyond the block boundary. With respect to the upper and lower pixels, the same method using a line buffer can be used to easily cross the block boundary. When data reference beyond the image boundary occurs, exception processing such as setting the data at the position exceeding the image boundary to 0 may be performed.
[0033]
Next, in step 9 of FIG. 1, a block is encoded by assigning a corresponding code from the quantized value of each pixel to each pixel. In this embodiment, since quantization is performed using three threshold values P1, La, and P2, the code corresponding to the quantized value of each pixel may be 2 bits. For example, when the pixel value of the pixel of interest is larger than P2, the code of the pixel of interest is 11 in binary, and when the pixel value of the pixel of interest is larger than La and less than or equal to P2, the code of the pixel of interest is 10 in binary. To. Further, when the pixel value of the target pixel is greater than P1 and equal to or less than La, the sign of the target pixel is 01 in binary, and when the pixel value of the target pixel is P1 or less, the sign of the target pixel is 00. .
[0034]
Next, in step 10 in FIG. 1, it is checked whether or not there is an unprocessed block in the image. If there is an unprocessed block, the process is performed on the unprocessed block from step 2 in FIG. By performing the processing as described above, it is possible to perform noise removal processing and encoding processing of each block.
[0035]
In this embodiment, a block size of 4 × 4 is used, but other block sizes may be used. When the input image signal is color, the same effect can be obtained by selecting a block size or a pixel reference method according to the color format. In this embodiment, a method of calculating P1, P2, La, etc. from Lmax and Lmin and quantizing the pixel value of the pixel in the block is used. However, the pixel value in the block is quantized from Lmax and Lmin. As long as it is a technique to be used, another technique or other number of quantization levels may be used.
[0036]
In this embodiment, the noise determination process is performed using a pattern matrix as shown in FIG. 4. However, a good result can be obtained by using an appropriate matrix depending on the noise characteristics of the image sensor. Can do. Further, although the noise detection process using the pattern matrix of FIG. 4 has been shown, other noise detection methods may be used as long as they can be applied to quantized pixel values. Further, the pattern matrix to be used is not necessarily the size, shape, and coefficient as shown in FIG. 4, and may be, for example, a 5 × 5 matrix or a 6 × 4 matrix. Further, a pattern matrix having a shape other than a rectangle may be used.
[0037]
In this embodiment, the average value of the pixel values related to the neighboring pixels is calculated by referring to the neighboring pixels as shown in FIG. 5, but other positions may be referred to. In addition to the noise removal processing (correction processing) using the neighboring pixels shown in this embodiment, a processing method of subtracting or adding only the noise component from the pixel value is also conceivable. These methods are particularly effective when the noise level of the image sensor is known in advance.
[0038]
Embodiment 2. FIG.
FIG. 6 is a diagram showing a processing flow of noise detection of the image coding method according to another embodiment of the present invention. FIG. 7 is a block diagram showing the configuration of the noise detection means of the image coding apparatus according to this embodiment, and each functional block is assigned a processing code in FIG. Since other parts of this embodiment are the same as those of the above embodiment, the description thereof is omitted.
[0039]
In FIG. 7, the noise detection means 7 b includes a pixel number counting unit 11 and a threshold value determination unit 12.
[0040]
In this embodiment, in step 11 in FIG. 6, the number of pixels included in the quantization level in the block is counted. For example, when dividing into four quantization levels using three threshold values of P1, La, and P2 as shown in the above embodiment, the pixels included in each level are counted, and four count values are set. Will get.
[0041]
Next, in step 12 of FIG. 6, the obtained count value of each level is compared with a predetermined threshold value. If the count value is smaller than the threshold value, pixels included in the level are determined as noise. In this embodiment, the case of four quantization levels has been described. However, other quantization levels may be used.
[0042]
Embodiment 3 FIG.
FIG. 8 is a diagram showing the noise detection processing flow of the image coding method according to another embodiment of the present invention. FIG. 9 is a block diagram showing the configuration of the noise detection means of the image coding apparatus according to this embodiment, and each functional block is assigned the processing code in FIG. Since other parts of this embodiment are the same as those of the above embodiment, the description thereof is omitted.
[0043]
In FIG. 9, the noise detection means 7 b includes a pixel number counting unit 13, a pixel selection unit 14, and a noise determination unit 15.
[0044]
In this embodiment, first, in step 13 of FIG. 8, the number of pixels included in the quantization level in the block is counted. For example, when dividing into four quantization levels using three threshold values of P1, La, and P2 as shown in the above embodiment, the pixels included in each level are counted, and four count values are set. Will get.
[0045]
Next, in step 14 of FIG. 8, the obtained count value of each level is compared with a predetermined threshold value, and if the count value is smaller than the threshold value, the level is selected. Next, in step 15 in FIG. 8, when a pixel included in the selected level does not have a predetermined relationship between the neighboring pixel and the pixel, the pixel is determined as noise. Here, for example, as shown in FIG. 10, the predetermined relationship is such that the pixel value of the pixel near the right is smaller and the pixel value of the pixel near the left is larger than the pixel value of the pixel. Therefore, an appropriate one is used depending on the characteristics of noise and the characteristics of the image obtained from the image sensor.
[0046]
In this embodiment, the case of four quantization levels has been described. However, other quantization levels may be used. In addition, as a predetermined relationship regarding neighboring pixels, the pixel value of the pixel in the right neighborhood is smaller and the pixel value of the pixel in the left neighborhood is larger than the pixel value of the pixel. It may be.
[0047]
Embodiment 4 FIG.
FIG. 11 is a diagram showing the noise detection processing flow of the image coding method according to another embodiment of the present invention. Further, the configuration of the noise detection means of the image coding apparatus in this embodiment is the same as that shown in FIG. 9, and each functional block is assigned the processing code in FIG. That is, the noise detection means 7 b includes a pixel count unit 16, a pixel selection unit 17, and a noise determination unit 18. Since other parts of this embodiment are the same as those of the above embodiment, the description thereof is omitted.
[0048]
In this embodiment, first, in step 16 of FIG. 11, the number of pixels included in the quantization level in the block is counted. For example, when dividing into four quantization levels using three threshold values of P1, La, and P2 as shown in the above embodiment, the pixels included in each level are counted, and four count values are set. Will get.
[0049]
Next, in step 17 of FIG. 11, the obtained count value of each level is compared with a predetermined threshold value, and if the count value is smaller than the threshold value, the level is selected. Next, in step 18 of FIG. 11, when a pixel included in the selected level does not have a pixel having a pixel value close to the pixel value of the pixel, the pixel is determined as noise. Here, whether or not the pixel value of the target pixel is close to the pixel values of neighboring pixels is determined by comparing the absolute value of the difference between the two pixel values with a predetermined threshold value. An appropriate threshold is used depending on the characteristics of noise and the characteristics of an image obtained from the image sensor.
[0050]
In this embodiment, the case of four quantization levels has been described. However, other quantization levels may be used. In addition, the absolute value related to the difference between the two pixel values is used to determine whether the pixel value of the target pixel is close to the pixel values of the neighboring pixels, but other determination methods may be used.
[0051]
Embodiment 5 FIG.
FIG. 12 is a diagram showing the noise detection processing flow of the image coding method according to another embodiment of the present invention. Further, the configuration of the noise detection means of the image coding apparatus in this embodiment is the same as that shown in FIG. 9, and each functional block is assigned the processing code in FIG. That is, the noise detection means 7 b includes a pixel count unit 19, a pixel selection unit 20, and a noise determination unit 21. Since other parts of this embodiment are the same as those of the above embodiment, the description thereof is omitted.
[0052]
In this embodiment, first, in step 19 of FIG. 12, the number of pixels included in the quantization level in the block is counted. For example, when dividing into four quantization levels using three threshold values of P1, La, and P2 as shown in the above embodiment, the pixels included in each level are counted, and four count values are set. Will get.
[0053]
Next, in step 20 of FIG. 12, the count value of each level obtained is compared with a predetermined threshold value, and if the count value is smaller than the threshold value, the level is selected. Next, in step 21 of FIG. 12, when there is no pixel included in the selected level that has a monotonously increasing or monotonically decreasing relationship with the target pixel, the pixel is determined as noise.
[0054]
Here, the neighboring pixels referred to for the pixels included in the selected level are the four pixel pairs shown in FIG. The pixel pairs shown in FIG. 13 are the upper and lower pixels centered on the target pixel, the left and right pixels centered on the target pixel, the pixels in the diagonal direction from the upper left to the lower right centered on the target pixel, It is a pixel in an oblique direction from the upper right to the lower left with the pixel at the center. In FIG. 13, the center pixel is the target pixel.
[0055]
In this embodiment, with respect to each neighboring pixel pair shown in FIG. 13, it is determined whether or not there is a relationship between the pixel value of the target pixel and a monotone increase or a monotone decrease. Here, the relationship of monotonic increase is the relationship shown in FIG. 14, and the relationship of monotonous decrease is the relationship shown in FIG.
[0056]
In this embodiment, the case of four quantization levels has been described. However, other quantization levels may be used. Further, the pixel pair shown in FIG. 13 is used to determine the relationship between the monotonic increase or the monotonic decrease with respect to the pixel value of the target pixel and the pixel value of the neighboring pixel. A determination may be made.
[0057]
Embodiment 6 FIG.
FIG. 16 is a diagram showing the noise detection processing flow of the image coding method according to another embodiment of the present invention. Further, the configuration of the noise detection means of the image encoding apparatus in this embodiment is the same as that shown in FIG. 9, and each functional block is assigned the processing code in FIG. That is, the noise detecting means 7b includes a pixel number counting unit 22, a pixel selecting unit 23, and noise determining units (24, 25, 26). Since other parts of this embodiment are the same as those of the above embodiment, the description thereof is omitted.
[0058]
In this embodiment, first, in step 22 in FIG. 16, the number of pixels included in the quantization level in the block is counted. For example, when dividing into four quantization levels using three threshold values of P1, La, and P2 as shown in the above embodiment, the pixels included in each level are counted, and four count values are set. Will get.
[0059]
Next, in step 23 of FIG. 16, the obtained count value of each level is compared with a predetermined threshold value, and if the count value is smaller than the threshold value, the level is selected. Next, in step 24 of FIG. 16, a pixel that is included in the selected level and that does not have a pixel that has a monotonously increasing or monotonically decreasing relationship with the target pixel is selected. Here, the determination of monotonic increase or monotonic decrease for the neighboring pixels is performed using the method as shown in the fifth embodiment.
[0060]
Further, in step 25 of FIG. 16, a pixel that is included in the selected level and has no pixel having a pixel value close to the pixel value of the pixel in the vicinity pixel is selected. Here, the determination regarding the neighboring pixels is performed using the method as described in the fourth embodiment.
[0061]
Next, in step 26 of FIG. 16, pixels that satisfy both the conditions of steps 24 and 25 in the selected pixels are determined as noise. In this embodiment, the case of four quantization levels has been described, but other quantization levels may be used.
[0062]
Embodiment 7 FIG.
FIG. 17 is a diagram showing a processing flow of block encoding processing of an image encoding method according to another embodiment of the present invention. FIG. 18 shows the configuration of the block encoding means of the image encoding apparatus in this embodiment. Each functional block is denoted by a reference numeral for processing in FIG. That is, the block encoding unit 9 includes a second signal level range calculation unit 27, a second quantization level calculation unit (28, 29, 30), and an encoding unit 31. Since other parts of this embodiment are the same as those of the above embodiment, the description thereof is omitted.
[0063]
In this embodiment, first, Lmax and Lmin are recalculated for the pixels in the block from which noise has been removed in step 27 of FIG. In step 28 of FIG. 17, P1 and P2 are calculated using the expressions (1) and (2) shown in the first embodiment, and Q1 and Q4 are calculated in step 29 of FIG. In addition, the method shown in Embodiment 1 is used for the calculation method of Lmax, Lmin, P1, P2, Q1, and Q4. In step 30 of FIG. 17, La and Ld are calculated from the equations (3) and (4) shown in the first embodiment.
[0064]
Next, each pixel in the block is quantized using P1, La, and P2 obtained as threshold values, and a predetermined code is assigned to the pixels included in each level. For example, in this embodiment, since three threshold values of P1, La, and P2 are used, each pixel value is quantized to four levels, and a code for each level is a 2-bit code such as 00, 01, 10, 11 Encoding is performed by making it a value.
[0065]
In this embodiment, a method of calculating P1, P2, La, etc. from Lmax and Lmin and quantizing the pixel values of the pixels in the block is used. However, the pixel values in the block are quantized from Lmax and Lmin. Any other method may be used as long as it is a method. In this embodiment, a method of encoding each pixel with 2 bits is shown, but other quantization numbers may be used. The number of bits to be assigned is selected according to the required post-decoding image quality and compression rate.
[0066]
Reference example.
  FIG. 19 shows the present invention.Reference exampleFIG. 6A is a diagram illustrating a processing flow of the image decoding method in FIG. 5A, FIG. 5A illustrates a detailed flow of the decoding processing and pixel correction processing, and FIG. Figure 20 shows thisReference exampleIt is a block diagram which shows the structure of the image decoding apparatus in FIG. Each functional block in FIG. 20 is given the reference numerals for the processes in FIG.
[0067]
20, the image decoding apparatus includes decoding means (32, 33, 34) and pixel correction means 36 including a matrix selection unit 36a, a matching unit 36b, a noise determination unit 36c, and a pixel correction unit 36d.
[0068]
  thisReference exampleThen, decoding processing and noise removal processing are performed every four lines using a four-line buffer. (A) in FIG. 19 shows thisReference exampleIt is a figure showing the processing flow of the decoding process and noise removal process which are performed for every 4 line buffer in FIG. Accordingly, the decoding process and the noise removal process for the entire image are performed by repeating the flow of FIG. 19A every four lines.
[0069]
In the process performed every four lines, first, the code data of each block is read from the code data of the image in step 32 of FIG. 19A, and further the quantum data is read from the code data of the block read in step 33 of FIG. The representative value of the conversion level is calculated. For example, when La and Ld are added to the code data of each block in the encoding process, the representative value of each quantization level is expressed as shown in equations (5), (6), (7), and (8). calculate. Here, representative values in the case of using four quantization levels are shown.
[0070]
La-Ld / 2 (5)
La-Ld / 6 (6)
La + Ld / 6 (7)
La + Ld / 2 (8)
[0071]
Further, in step 34 in FIG. 19A, the obtained representative value of each quantization level is assigned to the code corresponding to each pixel, thereby decoding the pixel value. The obtained decoded value is held in the line buffer in units of blocks. Next, in step 35 of FIG. 19A, it is checked whether there is a block for which a decoded value of each pixel to be held in the 4-line buffer is not obtained. If there is an unprocessed block, the process related to the block is performed from step 32. If there is no unprocessed block, it is determined that decoded values of each pixel for four lines have been obtained, and noise removal processing is performed on the decoded values on the line buffer in step 36 in FIG. Pixel correction for correcting the pixel value of the pixel detected as noise is performed. This pixel correction processing is for removing noise that has not been removed in the noise removal processing at the time of encoding as shown in the first embodiment.
[0072]
FIG. 19B shows a detailed flow of the pixel correction process. In the noise removal process, first, one pattern matrix is selected from a plurality of pattern matrices prepared in advance in step 36a. As a pattern matrix selection method, a method using an external control signal or the like, a method of associating from an average value of decoded values, or the like is used. Here, a 5 × 5 matrix centered on the pixel of interest shown in FIG. 21 is used as the pattern matrix selected by the matrix selection unit 36a.
[0073]
Next, in step 36b, pattern matching between the selected pattern matrix and the decoded value on the line buffer is performed. Pattern matching is performed by calculating the absolute value of the convolution integral relating to the pattern of FIG. 21 and the decoded values of the pixel of interest and its neighboring pixels. Here, reference to pixels beyond the boundary of the 4-line buffer may occur. In this case, convolution integration is performed with the decoded value located outside the 4-line buffer as 0.
[0074]
Next, in step 36c, it is determined whether or not the pixel of interest is noise by comparing the absolute value of the convolution integral that is the obtained matching result with a predetermined threshold value. If the matching result related to the target pixel is equal to or greater than the threshold value, it is determined as noise, and the decoded value of the target pixel is corrected in step 36d. The correction process related to the decoded value of the target pixel is the same as the noise removal process performed in step 8 of FIG. Each pixel value after noise removal, that is, pixel correction processing is output from the line buffer, and the next four lines are processed. If there are no four lines to be processed next, the entire image processing is terminated.
[0075]
  thisReference exampleShows a method of performing processing every 4 lines using a 4-line buffer, but this is for facilitating reference of a decoded value across a block boundary when performing pattern matching. The method may be used to refer to a decoded value that exceeds a block boundary. Further, it may be realized using a buffer of four lines or more. BookReference exampleThen, when calculating the quantization level from the code data of the block, La and Ld are used, but each quantization level may be calculated using other methods.
[0076]
Further, the pattern matrix used in the noise removal process does not necessarily have the size, shape, and coefficients shown in FIG. 21, and may be, for example, a 5 × 5 matrix or a 6 × 4 matrix. Further, a pattern matrix having a shape other than a rectangle may be used. In addition, although the method of comparing the absolute value of the value obtained by using convolution integration with the threshold value in the matching processing and noise determination processing using the pattern matrix was shown, the pixel of interest and its surrounding pixels using the pattern matrix Any other method may be used as long as it is a method for determining whether or not the pattern matches the pattern.
[0077]
【The invention's effect】
As described above, in the first image encoding method and apparatus according to the present invention, the image data obtained from the image sensor is divided into M × N (M and N are natural numbers) blocks. Further, the maximum value Lmax and the minimum value Lmin of the pixel value in the block are calculated, and the quantization level of the pixel value is determined from the obtained Lmax and Lmin. Next, each pixel value in the block is quantized from the obtained quantization level, and noise is detected from Lmax, Lmin, and the quantized value of each pixel value. Further, the pixel value of the pixel detected as noise is corrected, and encoding is performed in the block after noise removal. By performing such image coding processing, noise contained in the image obtained from the image sensor can be reduced, and at the same time, block coding of the image can be performed at an appropriate quantization level. By removing, there is an effect that a decoded image having good image quality is guaranteed.
[0078]
In the second image coding method and apparatus according to the present invention, the number of pixels included in each quantization level in the block is counted particularly in noise detection processing, and the counted result is included in a level smaller than a certain threshold value. The detected pixel is determined as noise. By performing such noise detection processing, noise detection can be easily performed in the image encoding processing, and noise detection utilizing the quantization level calculation processing at the time of encoding can be performed. There is an effect that can be done.
[0079]
In the third image coding method and apparatus according to the present invention, the number of pixels included in each quantization level in the block is counted particularly in noise detection processing, and the counted result is included in a level smaller than a certain threshold value. Select the pixel to be displayed. Further, a pixel in which a pixel having a predetermined relationship with neighboring pixels does not exist in the selected pixel is determined as noise. By performing such noise detection processing, noise misjudgment can be reduced, and in particular, the number of pixels at each level obtained from the quantization level calculation processing at the time of encoding, and the relationship between those pixels and neighboring pixels. By using the noise detection process, noise detection with few erroneous determinations can be performed.
[0080]
In the fourth image encoding method and apparatus according to the present invention, pixels that do not include pixels having pixel values close to neighboring pixels among the pixels selected in the noise determination processing of the noise detection processing are determined as noise. By performing such noise detection processing, noise detection processing is performed using the number of pixels at each level obtained from the quantization level calculation processing at the time of encoding and the pixel values of those pixels and neighboring pixels. Noise detection with few misjudgments can be performed.
[0081]
In the fifth image encoding method and apparatus according to the present invention, in particular, a pixel selected in the noise determination process of the noise detection process includes a pixel that is in a monotonically increasing or decreasing relationship with the pixel value of the pixel of interest in the neighboring pixels. A pixel that does not exist is determined as noise. By performing such noise detection processing, noise detection processing is performed by using the number of pixels at each level obtained from the quantization level calculation processing at the time of encoding and the relationship between these pixels and neighboring pixels. Can be reduced, and in particular, erroneous determination near the edge can be reduced.
[0082]
In the sixth image coding method and apparatus according to the present invention, in particular, a pixel selected in the noise determination process of the noise detection process includes a pixel that is in a monotonically increasing or monotonic decreasing relationship with the pixel value of the target pixel in the neighboring pixels. Pixels that do not exist and pixels that do not have pixels with pixel values close to neighboring pixels are determined as noise. By performing such noise detection processing, noise detection processing is performed by using the number of pixels at each level obtained from the quantization level calculation processing at the time of encoding and the relationship between these pixels and neighboring pixels. Can be reduced, and in particular, erroneous determination in the vicinity of an edge can be reduced.
[0083]
In the seventh image encoding method and apparatus according to the present invention, Lmax and Lmin are recalculated in the block after noise removal in the block encoding process, and the quantization level is calculated from the obtained Lmax and Lmin. Do. By performing such an encoding process, an image can be encoded based on an appropriate quantization level after noise removal, and there is an effect that a decoded image having a good image quality can be obtained.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a processing flow of an image encoding method according to Embodiment 1 of the present invention.
FIG. 2 is a block diagram showing a configuration of an image coding apparatus according to Embodiment 1 of the present invention.
FIG. 3 is a diagram showing blocks to be divided in the encoding process according to the first embodiment of the present invention.
FIG. 4 is a diagram showing a pattern matrix used in encoding processing according to Embodiment 1 of the present invention.
FIG. 5 is a diagram showing neighboring pixels to be referred to in the encoding process according to Embodiment 1 of the present invention.
FIG. 6 is a diagram illustrating a processing flow of noise detection processing of an image encoding method according to Embodiment 2 of the present invention.
FIG. 7 is a block diagram showing a configuration of noise detection means of an image coding apparatus according to Embodiment 2 of the present invention.
FIG. 8 is a diagram illustrating a processing flow of noise detection processing of an image encoding method according to Embodiment 3 of the present invention.
FIG. 9 is a block diagram showing a configuration of noise detection means of an image coding apparatus according to Embodiments 3 to 6 of the present invention.
FIG. 10 is a diagram showing the relationship between neighboring pixels used in noise detection processing according to Embodiment 3 of the present invention.
FIG. 11 is a diagram illustrating a processing flow of noise detection processing of an image encoding method according to Embodiment 4 of the present invention.
FIG. 12 is a diagram illustrating a processing flow of noise detection processing of an image encoding method according to Embodiment 5 of the present invention.
FIG. 13 is a diagram illustrating a pair of neighboring pixels referred to in the noise detection process according to the fifth embodiment of the present invention.
FIG. 14 is a diagram illustrating the relationship between neighboring pixels used in noise detection processing according to the fifth embodiment of the present invention.
FIG. 15 is a diagram illustrating a relationship between neighboring pixels used in noise detection processing according to the fifth embodiment of the present invention.
FIG. 16 is a diagram illustrating a processing flow of noise detection processing of an image encoding method according to Embodiment 6 of the present invention.
FIG. 17 is a diagram illustrating a processing flow of block encoding processing of an image encoding method according to Embodiment 7 of the present invention.
FIG. 18 is a block diagram showing a configuration of block encoding means of an image encoding device according to Embodiment 7 of the present invention.
FIG. 19 shows the present invention.Reference exampleIt is a figure showing the processing flow of the image decoding method in.
FIG. 20 shows the present invention.Reference exampleIt is a block diagram which shows the structure of the image decoding apparatus in FIG.
FIG. 21 shows the present invention.Reference exampleIt is a figure showing the noise pattern used by the decoding process of.
FIG. 22 is a block diagram of a conventional image encoding device.
[Explanation of symbols]
  1 block division means, 2, 3 signal level range calculation means, 4, 5, 6 quantization level calculation means, 7a quantization means, 7b noise detection means, 8 noise removal means, 9 block coding means, 11, 13, 16, 19, 22 Pixel count unit, 14, 17, 20, 23 Pixel selection unit, 15, 18, 21, 24, 25, 26 Noise determination unit, 27 Second signal level range calculation unit, 28, 29, 30 2nd quantization level calculation part, 31 encoding part, 32, 33, 34 decoding means, 36 pixel correction means, 36a matrix selection part, 36b matching part, 36c noise determination part, 36d pixel correction part.

Claims (16)

A block dividing step of dividing image data obtained from the image sensor into M × N (M and N are natural numbers) blocks;
A signal level range calculating step for calculating a maximum value Lmax and a minimum value Lmin of the pixel values in the block;
A quantization level calculation step of determining a quantization level of the pixel value from the obtained Lmax and Lmin;
A quantization step of quantizing each pixel value in the block from the obtained quantization level;
A noise detection step of calculating an absolute value of a convolution integral value using a pattern matrix from Lmax, Lmin and a quantized value of each pixel value, and detecting noise based on the magnitude of the obtained absolute value ;
For a pixel detected as noise, a noise removal step of correcting the pixel value before quantization ;
A block encoding step for encoding in the block after noise removal;
An image encoding method comprising: M × N image data obtained from the image sensor ( M and N are natural numbers ) A block dividing step of dividing the block into
  A signal level range calculating step for calculating a maximum value Lmax and a minimum value Lmin of the pixel values in the block;
  A quantization level calculation step of determining a quantization level of the pixel value from the obtained Lmax and Lmin;
  A quantization step of quantizing each pixel value in the block from the obtained quantization level;
A noise detection step of detecting noise based on a relationship between a result of counting the number of pixels included in each quantization level in the block and a threshold;
For a pixel detected as noise, a noise removal step of correcting the pixel value before quantization;
A block encoding step for encoding in the block after noise removal;
  An image encoding method comprising: The noise detection step counts the number of pixels included in each quantization level in the block, and threshold determination for determining pixels included in a level smaller than a certain threshold as noise. The image encoding method according to claim 2 , comprising steps. The noise detection step is selected by a pixel number counting step for counting the number of pixels included in each quantization level in the block, and a pixel selection step for selecting pixels included in a level smaller than a threshold value as a result of the counting. The image encoding method according to claim 2 , further comprising: a noise determination step of determining, as noise, a pixel in which a pixel having a predetermined relationship with neighboring pixels does not exist. The noise detection step is selected by a pixel number counting step for counting the number of pixels included in each quantization level in the block, and a pixel selection step for selecting pixels included in a level smaller than a threshold value as a result of the counting. The image encoding method according to claim 2 , further comprising: a noise determination step of determining, as noise, a pixel in which a pixel having a pixel value close to a neighboring pixel does not exist. The noise detection step is selected by a pixel number counting step for counting the number of pixels included in each quantization level in the block, and a pixel selection step for selecting pixels included in a level smaller than a threshold value as a result of the counting. The image code according to claim 2 , further comprising: a noise determination step of determining, as noise, a pixel in which no neighboring pixel has a pixel value that is monotonously increasing or decreasing monotonously with a pixel value of a target pixel. Method. The noise detection step is selected by a pixel number counting step for counting the number of pixels included in each quantization level in the block, and a pixel selection step for selecting pixels included in a level smaller than a threshold value as a result of the counting. A noise determination step of determining, as a noise, a pixel in which a pixel having a relationship of monotonously increasing or monotonously decreasing with a pixel value of a target pixel in a neighboring pixel and a pixel having a pixel value close to a neighboring pixel are not present, The image encoding method according to claim 2 , comprising: The block encoding step includes a second signal level range calculating step for recalculating Lmax and Lmin in the block after noise removal, a second quantization level calculating step for calculating a quantization level from the obtained Lmax and Lmin, The image encoding method according to claim 1 , further comprising: an encoding step of encoding after quantizing each pixel value in the block from the quantized level. Block dividing means for dividing image data obtained from the image sensor into M × N (M and N are natural numbers) blocks;
Signal level range calculating means for calculating the maximum value Lmax and the minimum value Lmin of the pixel values in the block;
A quantization level calculating means for determining a quantization level of the pixel value from the obtained Lmax and Lmin;
Quantization means for quantizing each pixel value in the block from the obtained quantization level;
Noise detection means for calculating an absolute value of a convolution integral value using a pattern matrix from Lmax, Lmin and the quantized value of each pixel value, and detecting noise based on the magnitude of the obtained absolute value ;
Noise removal means for correcting pixel values before quantization for pixels detected as noise,
Block encoding means for performing encoding in the block after noise removal;
An image encoding apparatus comprising: M × N image data obtained from the image sensor ( M and N are natural numbers ) Block dividing means for dividing the block into
  Signal level range calculating means for calculating the maximum value Lmax and the minimum value Lmin of the pixel values in the block;
  A quantization level calculating means for determining a quantization level of the pixel value from the obtained Lmax and Lmin;
  Quantization means for quantizing each pixel value in the block from the obtained quantization level;
A noise detecting means for detecting noise based on a relationship between a threshold value and a result of counting the number of pixels included in each quantization level in the block;
Noise removal means for correcting pixel values before quantization for pixels detected as noise,
Block encoding means for performing encoding in the block after noise removal;
  An image encoding apparatus comprising: The above-mentioned noise detecting means counts the number of pixels included in each quantization level in the block, and threshold determination for determining pixels included in a level lower than a certain threshold as noise as a result of counting The image encoding apparatus according to claim 10 , further comprising: The noise detection means is selected by a pixel number counting unit that counts the number of pixels included in each quantization level in the block, and a pixel selection unit that selects pixels included in a level smaller than a threshold value as a result of counting. The image coding apparatus according to claim 10 , further comprising: a noise determination unit that determines, as noise, a pixel in which a pixel having a predetermined relationship among neighboring pixels does not exist. The noise detecting means is selected by a pixel number counting unit that counts the number of pixels included in each quantization level in the block, and a pixel selection unit that selects pixels included in a level that is smaller than a threshold value as a result of counting. The image coding apparatus according to claim 10 , further comprising: a noise determination unit that determines, as noise, a pixel in which a pixel having a pixel value close to a neighboring pixel does not exist. The noise detecting means is selected by a pixel number counting unit that counts the number of pixels included in each quantization level in the block, and a pixel selection unit that selects pixels included in a level that is smaller than a threshold value as a result of counting. The image code according to claim 10 , further comprising: a noise determination unit that determines, as noise, a pixel in which a pixel having a monotonously increasing or monotonic decreasing relationship with a pixel value of a target pixel does not exist in a neighboring pixel. Device. The noise detecting means is selected by a pixel number counting unit that counts the number of pixels included in each quantization level in the block, and a pixel selection unit that selects pixels included in a level that is smaller than a threshold value as a result of counting. A noise determination unit that determines, as a noise, a pixel in which there is no pixel that is in a monotonically increasing or monotonic decreasing relationship with a pixel value of a target pixel in a neighboring pixel and a pixel that does not have a pixel having a pixel value close to the neighboring pixel; The image encoding device according to claim 10 , comprising: A second signal level range calculating unit for recalculating Lmax and Lmin in the block after noise removal; a second quantization level calculating unit for calculating a quantization level from the obtained Lmax and Lmin; The image encoding apparatus according to claim 9 or 10 , further comprising: an encoding unit that performs encoding after quantizing each pixel value in the block from the quantized level.

Images