CN111583162A - Image enhancement method based on histogram equalization - Google Patents
Image enhancement method based on histogram equalization Download PDFInfo
- Publication number
- CN111583162A CN111583162A CN202010378155.4A CN202010378155A CN111583162A CN 111583162 A CN111583162 A CN 111583162A CN 202010378155 A CN202010378155 A CN 202010378155A CN 111583162 A CN111583162 A CN 111583162A
- Authority
- CN
- China
- Prior art keywords
- histogram
- image
- global
- sub
- local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Facsimile Image Signal Circuits (AREA)
Abstract
The invention provides an image enhancement method based on histogram equalization, which comprises the following steps: an image input unit inputs an image; the global histogram equalization unit performs global equalization processing on the image to obtain a first image; the local histogram equalization unit performs local equalization processing on the image to obtain a second image; the scene similarity unit calculates the weight occupied by the first image and the weight occupied by the second image; and the image output unit fuses the first image and the second image according to the weight occupied by the first image and the weight occupied by the second image to obtain an enhanced image. According to the invention, the input graphics are processed, so that the contrast of the image is enhanced, the image details can be maintained, and meanwhile, the image has good overall brightness.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image enhancement method based on histogram equalization.
Background
The image histogram is an image which is obtained by counting the size and the number of each pixel level in the image based on a statistical concept, and can well show the distribution condition of the pixel levels of the image. Histograms of images are often applied in image processing methods, in particular image enhancement methods. Histogram equalization is a very representative method in image contrast enhancement methods, and aims to convert a brightness interval in a relatively concentrated image histogram into uniform distribution in a large range, thereby effectively enhancing the image contrast. Due to a series of advantages of simple histogram equalization principle, low calculation complexity and the like, the method is widely applied to the fields of high-dynamic images, infrared images, underwater image enhancement methods and the like.
Conventional histogram equalization methods are classified into a global histogram equalization method and a local histogram equalization method. Global histogram equalization aims at mapping all pixels at the same pixel level in the whole image to another pixel level, and performs contrast enhancement on the image in a global range, but in some cases, the processing has the problems of regional oversaturation and the like, the contrast of the image is not obvious enough, and the details of the image cannot be maintained. The local histogram equalization divides the original image into sub image blocks, and performs histogram equalization processing on the sub image blocks respectively, so that the processing can effectively improve the definition, the contrast enhancement effect is better than that of the global histogram equalization, but the problems of blocking effect, amplification of dark noise and the like sometimes exist.
Disclosure of Invention
The invention aims to provide an image enhancement method based on histogram equalization, which can enhance the contrast of an image, keep the details of the image and provide the overall brightness of the image by processing an input image.
In order to achieve the above object, the present invention provides an image enhancement method based on histogram equalization, comprising the steps of:
an image input unit inputs an image;
the global histogram equalization unit performs global equalization processing on the image to obtain a first image;
the local histogram equalization unit performs local equalization processing on the image to obtain a second image;
the scene similarity unit calculates the weight occupied by the first image and the weight occupied by the second image;
and the image output unit fuses the first image and the second image according to the weight occupied by the first image and the weight occupied by the second image to obtain an enhanced image.
Optionally, in the histogram equalization-based image enhancement method, the method for obtaining the first image by performing global equalization on the image by using the global histogram equalization unit includes:
the global histogram calculation module counts the pixel distribution of the whole image and forms a global histogram;
the histogram segmentation module segments the global histogram into a global high-order sub-histogram and a global low-order sub-histogram;
the bidirectional limited calculation module respectively calculates the dynamic range for limiting the global high-order sub-histogram and the dynamic range for limiting the global low-order sub-histogram;
and the bidirectional limited equalization module equalizes the global high-order sub-histogram according to the dynamic range of the global high-order sub-histogram, equalizes the global low-order sub-histogram according to the dynamic range of the global low-order sub-histogram, fuses the equalized global high-order sub-histogram and the equalized global low-order sub-histogram, and applies the fused global high-order sub-histogram and the equalized global low-order sub-histogram to the input image to obtain a first image.
Optionally, in the histogram equalization-based image enhancement method, the method for forming a global histogram by a global histogram calculation module counting pixel distributions of the entire image includes:
calculating the total number of pixel points of the whole image;
counting the number of pixels corresponding to the pixel level according to the pixel level of the image;
and forming a global histogram by using the pixel grade and the ratio of the number of pixels corresponding to the pixel grade to the total number of the pixels.
Optionally, in the histogram equalization-based image enhancement method, the method for dividing the global histogram into the global upper sub-histogram and the global lower sub-histogram by the histogram dividing module includes:
calculating the threshold values of the separation points of the global high-order sub-histogram and the global low-order sub-histogram according to the pixel level and the pixel level ratio of the global histogram;
the threshold serves as a reference point that separates the global histogram into a global upper sub-histogram and a global lower sub-histogram.
Optionally, in the histogram equalization-based image enhancement method, the method for limiting the dynamic range of the global high-order sub-histogram and the method for limiting the dynamic range of the global low-order sub-histogram by the bidirectional limited computation module respectively include:
and obtaining the dynamic range for limiting the global high-order sub-histogram and the dynamic range for limiting the global low-order sub-histogram by using the threshold, the highest pixel level of the image, the lowest pixel level of the image and the number of pixel levels of the coding format corresponding to the image.
Optionally, in the histogram equalization-based image enhancement method, a formula that limits a dynamic range of the global high-order sub-histogram to meet is calculated as follows:
wherein αH1、αH2For a bi-directional limited intensity parameter configured by the user, the input range is [0,3 ]](ii) a n is the pixel grade number of the coding format corresponding to the image; th of spaced pointsA threshold value; histMinHTo limit the lower limit of the dynamic range of the global high sub-histogram; histMaxHTo limit the upper limit of the dynamic range of the global high sub-histogram; histMin is the lowest pixel level of the image; histMax is the highest pixel level of the image.
Optionally, in the histogram equalization-based image enhancement method, a formula that limits a dynamic range of the global lower sub-histogram to meet is calculated as follows:
wherein αL1、αL2For a bi-directional limited intensity parameter configured by the user, the input range is [0,3 ]](ii) a n is the pixel grade number of the coding format corresponding to the image; th is a threshold value of the separation point; histMinLTo limit the lower limit of the dynamic range of the global lower sub-histogram; histMaxLTo limit the upper limit of the dynamic range of the global lower sub-histogram; histMin is the lowest pixel level of the image; histMax is the highest pixel level of the image.
Optionally, in the histogram equalization-based image enhancement method, the method for equalizing the global upper sub-histogram according to the dynamic range of the global upper sub-histogram and equalizing the global lower sub-histogram according to the dynamic range of the global lower sub-histogram by using a bidirectional limited equalization module includes:
expanding the global high-order sub-histogram according to the dynamic range of the global high-order sub-histogram, and expanding the global low-order sub-histogram according to the dynamic range of the global low-order sub-histogram;
counting the proportion values corresponding to all pixel levels in the global low-order sub-histogram, cutting the proportion values which are larger than a first preset value, and averagely adding the accumulated cut values to each proportion value of the expanded global low-order sub-histogram;
and counting the proportion values corresponding to all the pixel levels in the global high-order sub-histogram, clipping the proportion values which are larger than a second preset value, and averagely adding the accumulated clipped values to each of the expanded proportion values in the global high-order sub-histogram.
Optionally, in the histogram equalization-based image enhancement method, the method of fusing the equalized global upper sub-histogram and the equalized global lower sub-histogram, and then applying the fused global upper sub-histogram and the equalized global lower sub-histogram to the input image to obtain the first image includes:
obtaining an accumulated histogram of the expanded global high-order sub-histogram and an accumulated histogram of the expanded global low-order sub-histogram;
fusing the cumulative histogram of the expanded global high-order sub-histogram with the cumulative histogram of the expanded global low-order sub-histogram;
and mapping the fused cumulative histogram as a mapping curve to the input image to obtain a first image.
Optionally, in the histogram equalization-based image enhancement method, the method for obtaining the second image by performing local equalization on the image by using the local histogram equalization unit includes:
the local image segmentation module equally divides the image into a plurality of sub-images;
the local histogram calculation module counts pixels of each sub-image;
the local histogram segmentation module divides the sub-image into a local high-order sub-image and a local low-order sub-image;
the local bidirectional limited calculation module respectively calculates the dynamic range for limiting the local high-order sub-image and the dynamic range for limiting the local low-order sub-image;
the local bidirectional limited equalization module equalizes the local high-order sub-image according to the dynamic range of the local high-order sub-image, equalizes the local low-order sub-image according to the dynamic range of the local low-order sub-image, fuses the equalized local high-order sub-histogram and the equalized local low-order sub-histogram, and then applies the fused local high-order sub-histogram and the equalized local low-order sub-histogram to the sub-image to obtain a sub-image subjected to local equalization processing;
and the local image fusion module performs linear interpolation processing on the sub-images subjected to the local equalization processing and fuses the sub-images subjected to the linear interpolation processing.
Optionally, in the histogram equalization-based image enhancement method, the method for performing linear interpolation processing on the sub-image subjected to the local equalization processing by the local image fusion module includes:
dividing the sub-image subjected to the local equalization processing into an edge area, a four-corner area and a middle area;
and performing interpolation operation on the pixel points in the edge area, the pixel points in the four corner areas and the pixel points in the middle area respectively.
Optionally, in the image enhancement method based on histogram equalization, the method for fusing the first image and the second image by the scene similarity unit includes:
the scene similarity calculation module acquires an information window with the current pixel point as the center according to window parameters configured by a user and calculates a scene similarity value;
the similarity weight calculation module judges whether the current pixel is in a uniform pixel area according to the scene similarity value;
if the histogram is in the uniform pixel region, the weight of the global histogram is larger; the local histogram is weighted more if in a non-uniform pixel region.
Optionally, in the image enhancement method based on histogram equalization, the method for obtaining the scene similarity value includes:
calculating the absolute difference value of the pixel grades of all the pixel points and the current pixel point in the information window;
and acquiring the weight of the first image and the weight of the second image according to the absolute difference value.
Optionally, in the image enhancement method based on histogram equalization, the weight of the first image is proportional to the value of the scene similarity.
In the image enhancement method based on histogram equalization provided by the invention, global histogram equalization processing is firstly carried out on an input image, meanwhile, local histogram equalization processing is carried out on the input image, and then the image after the global histogram equalization processing and the image after the local histogram equalization processing are combined according to the value of scene similarity, an enhanced image is output, and the contrast of the image is enhanced; the image details can be kept; while allowing the image to have good overall brightness.
Furthermore, in the local histogram equalization processing of the present invention, the local high-order sub-image and the local low-order sub-image are equalized by performing bidirectional limitation on the local high-order sub-image and the local low-order sub-image, and the problems of supersaturation, amplification of noise in a dark area, and the like can be suppressed, thereby further improving the overall effect of the image.
Drawings
FIG. 1 is a flow chart of an image enhancement method based on histogram equalization according to an embodiment of the present invention;
fig. 2 to 4 are process diagrams of histogram variations of a method of enhancing image contrast of histogram equalization according to an embodiment of the present invention;
FIG. 5 is a cumulative histogram mapping curve for a first image;
FIG. 6 is a diagram of a 5x5 information window with D12 as the current pixel according to an embodiment of the present invention;
in the figure: 1-first curve, 2-second curve, 3-third curve, 4-fourth curve, 5-fifth curve, 6-sixth curve, 7-seventh curve, 8-eighth curve.
Detailed Description
The following describes in more detail embodiments of the present invention with reference to the schematic drawings. The advantages and features of the present invention will become more apparent from the following description. It is to be noted that the drawings are in a very simplified form and are not to precise scale, which is merely for the purpose of facilitating and distinctly claiming the embodiments of the present invention.
In the following, the terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances. Similarly, if the method described herein comprises a series of steps, the order in which these steps are presented herein is not necessarily the only order in which these steps may be performed, and some of the described steps may be omitted and/or some other steps not described herein may be added to the method.
Referring to fig. 1, the present invention provides an image enhancement method based on histogram equalization, including:
s11: an image input unit inputs an image;
s12: the global histogram equalization unit performs global equalization on the image to obtain a first image;
s13: the local histogram equalization unit performs local equalization processing on the image to obtain a second image;
s14: the scene similarity unit calculates the weight occupied by the first image and the weight occupied by the second image;
s15: and the image output unit fuses the first image and the second image according to the weight occupied by the first image to obtain an enhanced image.
Here, the first image refers to an image after global equalization processing, and the second image refers to an image after global equalization processing.
In order to suppress the problems of supersaturation, dark space noise amplification and the like and further improve the overall effect of an image, the invention provides an image enhancement method based on histogram equalization, which can combine the results of the global histogram and the local histogram equalization processing according to scene similarity information, enhance the contrast of the image, keep the image details and simultaneously enable the image to have good overall brightness, and can suppress the problems of supersaturation, dark space noise amplification and the like and further improve the overall effect of the image.
Further, referring to fig. 2 to 5, the method for performing global equalization processing on the image by the global equalization unit to obtain the first image includes:
the global histogram calculation module counts the pixel distribution of the whole image and forms a global histogram;
the histogram segmentation module segments the global histogram into a global high-order sub-histogram and a global low-order sub-histogram;
the bidirectional limited calculation module respectively calculates the dynamic range for limiting the global high-order sub-histogram and the dynamic range for limiting the global low-order sub-histogram;
and the bidirectional limited equalization module equalizes the global high-order sub-histogram according to the dynamic range of the global high-order sub-histogram, equalizes the global low-order sub-histogram according to the dynamic range of the global low-order sub-histogram, and fuses the equalized global high-order sub-histogram and the equalized global low-order sub-histogram to obtain the first image.
Further, the method for forming the global histogram by the global histogram calculation module counting the pixel distribution of the whole image comprises the following steps:
calculating the total number of pixel points of the whole image;
counting the number of pixels corresponding to the pixel level according to the pixel level of the image;
and forming a global histogram by taking the pixel grade as a ratio of the number of pixels corresponding to the pixel grade to the total number of the pixels.
Specifically, the input image has a pixel level, for example, an 8-bit format image is input, the pixel level may be any one value from 0 to 255, and the same pixel level may have a plurality of pixel points, and of course, a situation where there is no pixel point in a certain pixel level or some pixel levels may also occur. Each pixel level occupies a certain proportion, and histograms can be formed by the pixel level and the proportion value of the pixel level, and the trend of the histograms can form a curve, for example, the histogram curve 1 of the input image in fig. 2, the abscissa is the pixel level, and the ordinate is the proportion value of the pixel level (the proportion value of the number of the pixel levels to the number of the pixel levels of the whole image). However, since the histogram curve 1 of the input image has a single peak or multiple peaks, which may affect the equalization of the global histogram, the global histogram needs to be equalized, and the first step of the equalization processing is to divide the global histogram into a global high-order sub-histogram and a global low-order sub-histogram, and divide the entire histogram into two global sub-histograms with an appropriate pixel level as a boundary.
Further, the method for the histogram segmentation module to segment the global histogram into a global upper sub-histogram and a global lower sub-histogram includes:
calculating the threshold values of the separation points of the global high-order sub-histogram and the global low-order sub-histogram according to the pixel level and the pixel level ratio of the global histogram;
the threshold serves as a reference point that separates the global histogram into a global upper sub-histogram and a global lower sub-histogram.
Specifically, the threshold for obtaining the separation point satisfies the following formula:
wherein: where the round function returns a rounded integer value; x is the pixel level of the image; pdf _ ori (x) is the fractional value of the pixel level of the image; and m is the highest pixel level of the coding format corresponding to the image.
For example, an 8-bit format image is input in the embodiment of the present invention, the pixel level of the 8-bit format image should be a value of 0 to 255, the highest pixel level is 255, that is, the value of m is 255, and then 256 levels are total. The pixel level values of the input image are all within 0-255, but each level of 0-255 is not necessarily available, for example, there are no pixel points with a pixel level of 0 and a pixel level of 1, and there are pixel points with a pixel level of 3 and a pixel level of 4. And obtaining the product of the ratio of the pixel level 3 to the pixel level 3, then obtaining the product of the ratio of the pixel level 4 to the pixel level 4, and adding the products to form an integer, namely the threshold value of the separation point. If in other embodiments of the present invention, the input image may be an image of other format, such as an image of 64-bit format, and the pixel points have more pixel levels, the same method is used to sequentially obtain the product of the ratio of each pixel level to the pixel level, and finally, all the products are summed, and the integer is the threshold of the separation point. Then, the maximum pixel level and the minimum pixel level of the image are obtained, as shown in fig. 2, the first curve 1 is a histogram of the input image in the histogram, and its corresponding minimum value and maximum value on the abscissa, the histogram in which the curve from the minimum pixel level to the threshold of the separation point is located is taken as the global lower sub-histogram, and the histogram in which the curve from the threshold of the separation point to the maximum pixel level is located is taken as the global upper sub-histogram. The abscissa of the histogram covers all levels from 0 to 255, but the abscissa corresponding to the curve does not necessarily cover all levels, and therefore the minimum pixel level of the image, i.e., the minimum value of the abscissa corresponding to the curve, is not the minimum value of the abscissa of the histogram, and similarly, the maximum pixel level of the image is also the same. As shown in fig. 3, the second curve 2 is a curve of the global lower sub-histogram, and the third curve 3 is a curve of the global upper sub-histogram.
In order to prevent the image from generating the problems of over-bright/over-dark and abnormal enhancement of noise in a dark area, the dynamic range of the segmented global high sub-histogram and the segmented global low sub-histogram after transformation needs to be limited, namely, the two-way limited equalization of the global high sub-histogram and the global low sub-histogram is carried out, and the overlapping area of the global high sub-histogram and the global low sub-histogram after the two-way limited equalization is combined according to a linear weighting method.
Further, the method for the bidirectional limited computation module to separately compute the dynamic range limiting the global upper sub-histogram and the dynamic range limiting the global lower sub-histogram includes:
and respectively obtaining the dynamic range for limiting the global high-order sub-histogram and the dynamic range for limiting the global low-order sub-histogram by utilizing the threshold of the separation point, the highest pixel level of the image, the lowest pixel level of the image and the number of pixel levels of the coding format corresponding to the image.
Specifically, the formula for calculating and limiting the dynamic range of the global high-order sub-histogram to satisfy is as follows:
wherein αH1、αH2For a bi-directional limited intensity parameter configured by the user, the input range is [0,3 ]](ii) a n is the number of pixel levels of the coding format corresponding to the image, for example, the value of n in the embodiment of the present invention may be 256, n-1 is 255, and 0 to 255 are also the value range of the pixel levels in the embodiment of the present invention; th is a threshold value of the separation point; histMinHTo limit the lower limit of the dynamic range of the global high sub-histogram; histMaxHTo limit the upper limit of the dynamic range of the global high sub-histogram; histMin is the lowest pixel level of the image; histMax is the highest pixel level of the image.
The formula that the calculation limits the dynamic range satisfaction of the global lower sub-histogram is as follows:
wherein αL1、αL2For a bi-directional limited intensity parameter configured by the user, the input range is [0,3 ]]The specific value is determined by the user; n is the number of pixel levels of the coding format corresponding to the image, for example, the value of n in the invention can be 256, n-1 is 255, and 0-255 is also the value range of the pixel levels in the embodiment of the invention; th is a threshold value of the separation point; histMinLTo limit the lower limit of the dynamic range of the global lower sub-histogram; histMaxLTo limit the upper limit of the dynamic range of the global lower sub-histogram; histMin is the lowest pixel level of the image; histMax is the highest pixel level of the image.
The lower limit of the dynamic range of the global lower sub-histogram, the upper limit of the dynamic range of the global lower sub-histogram, the lower limit of the dynamic range of the global upper sub-histogram and the upper limit of the dynamic range of the global upper sub-histogram range from 0 to the maximum pixel level of the coding format corresponding to the image, i.e. within the limits of the embodiments of the present invention [0,255 ]. That is, the value is 255 if the calculated dynamic range exceeds 255, and 0 if the calculated dynamic range is less than 0. And when the bidirectional limited intensity parameter input by the user is 0, the dynamic range is not expanded, and the larger the bidirectional limited intensity parameter input by the user is, the more the dynamic range is expanded, the larger the bright-dark contrast of the image is enhanced.
Further, the method for equalizing the global high sub-histogram by the bi-directional limited equalization module according to the dynamic range of the global high sub-histogram and equalizing the global low sub-histogram according to the dynamic range of the global low sub-histogram includes:
expanding the global high-order sub-histogram according to the dynamic range of the global high-order sub-histogram, and expanding the global low-order sub-histogram according to the dynamic range of the global low-order sub-histogram;
counting the occupation ratio values corresponding to the pixel levels of the global low-order sub-histogram in [ histMin, Th ], clipping the occupation ratio values larger than a first preset value, and averagely adding the accumulated clipped values to each occupation ratio value of the expanded global low-order sub-histogram in [ histMinL, histMaxL ];
and counting the occupation ratio values corresponding to the pixel levels of the global high-order sub-histogram in [ Th, histMax ], clipping the occupation ratio values which are larger than a second preset value, and averagely adding the accumulated clipped values to each occupation ratio value of the expanded global high-order sub-histogram in [ histMinH, histMaxH ]. And taking the lower limit of the dynamic range of the global high-order sub-histogram and the upper limit of the dynamic range of the global high-order sub-histogram as the minimum level and the maximum level of the curve in the expanded global high-order sub-histogram after equalization processing, and taking the lower limit of the dynamic range of the global low-order sub-histogram and the upper limit of the dynamic range of the global low-order sub-histogram as the minimum level and the maximum level of the curve in the expanded global low-order sub-histogram after equalization processing. The first preset value is a clipping value configured by a user, the default is 0.01, if the proportion value corresponding to each pixel level in [ histMin, Th ] of the global low-order sub-histogram exceeds 0.01, the global low-order sub-histogram is clipped, for example, if the value corresponding to a certain pixel level is 0.015, the clipped value is 0.005, namely, the clipped value is obtained by subtracting 0.01 from 0.015. If the ratio values corresponding to a plurality of levels need to be clipped, the sum of the clipped values is averagely added to the ratio value corresponding to each pixel level in the extended global low sub-histogram [ histMinL, histMaxL ]. As shown in fig. 4, the fourth curve 4 is an equalized global low sub-histogram, and the fifth curve 5 is an equalized global high sub-histogram.
Further, the method for fusing the equalized global upper sub-histogram and the equalized global lower sub-histogram includes:
respectively obtaining an accumulated histogram of the expanded global high-order sub-histogram and an accumulated histogram of the expanded global low-order sub-histogram;
and fusing the cumulative histogram of the expanded global upper sub-histogram and the cumulative histogram of the expanded global lower sub-histogram.
Specifically, the expression for obtaining the cumulative histogram of the expanded global high-order sub-histogram is as follows:
wherein: x represents the pixel level of the histogram, the range is [0,255], cdf _ H (x) is the pixel ratio of the cumulative histogram of the expanded global high-order sub-histogram, pdf _ H (i) is the value of the pixel level of the high-order sub-histogram, namely, the interpretation of the formula is that when i takes a certain pixel level, the sum of the pixel levels of all corresponding high-order sub-histograms of the pixel levels starting from 0 to i is taken as the pixel ratio of the cumulative histogram of the expanded global high-order sub-histogram, and the value of cdf _ H (x) is constantly changed along with the change of the value of i, so that the cumulative histogram of the global high-order sub-histogram is formed.
The expression of the cumulative histogram of the extended global lower sub-histogram is as follows:
wherein: x represents the pixel level of the histogram, the range is [0,255], cdf _ L (x) is the pixel ratio of the cumulative histogram of the expanded global lower sub-histogram, pdf _ L (i) is the value of the pixel level of the lower sub-histogram, that is, the interpretation of the formula is that when i takes a certain pixel level, the sum of the pixel levels of all corresponding lower sub-histograms from 0 to i is taken as the pixel ratio of the cumulative histogram of the expanded global upper sub-histogram, and as the value of i is changed, the value of cdf _ L (x) is also changed continuously, thus forming the cumulative histogram of the global lower sub-histogram.
According to the expression, when x is less than histMinLThen, the expression for solving the fused cumulative histogram is as follows:
cdf_M(x)＝x，
wherein: x is the pixel level; cdf _ M (x) is the fused cumulative histogram;
when histMinL≤x＜histMinHThen, the expression for solving the fused cumulative histogram is as follows:
cdf_M(x)＝cdf_L(x)，
wherein: cdf _ l (x) is the pixel fraction value of the cumulative histogram of the extended global lower sub-histogram; cdf _ M (x) is the fused cumulative histogram;
when histMinH≤x＜histMaxLThen, the expression for solving the fused cumulative histogram is as follows:
wherein: cdf _ l (x) is the pixel fraction value of the cumulative histogram of the extended global lower sub-histogram; histMinHBeing global high order sub-histogramsThe lower limit of the dynamic range; histMaxLTo limit the upper limit of the dynamic range of the global lower sub-histogram; histMinHIs the lower limit of the dynamic range of the global high sub-histogram; histMaxLIs the upper limit of the dynamic range of the global low sub-histogram; cdf _ h (x) is the pixel fraction value of the cumulative histogram of the extended global high sub-histogram; cdf _ M (x) is the fused cumulative histogram;
when histMaxL≤x＜histMaxHThen, the expression for solving the fused cumulative histogram is as follows:
cdf_M(x)＝cdf_H(x)，
wherein: cdf _ h (x) is the pixel fraction value of the cumulative histogram of the extended global high sub-histogram; cdf _ M (x) is the fused cumulative histogram;
when x is more than or equal to histMaxHThen, the expression for solving the fused cumulative histogram is as follows:
cdf_M(x)＝x，
wherein: x is the pixel level; cdf _ m (x) is the fused cumulative histogram.
The fused image is divided into a plurality of segments, the mapping expressions of the segments are different, peaks are eliminated, the curve of the histogram of the image is enabled to be more uniform, and the fused cumulative histogram cdf _ M (x) is mapped to the input image as a mapping curve to obtain a first image. Fig. 5 is a cumulative histogram mapping curve of the first image, the abscissa is the pixel level, the ordinate is the mapped pixel level of the cumulative histogram, the sixth curve 6 is a mapping curve of the cumulative histogram of the expanded global lower sub-histogram, the seventh curve 7 is a mapping curve of the cumulative histogram of the expanded global upper sub-histogram, and the eighth curve 8 is a mapping curve of the fused cumulative histogram.
Further, the method for performing local equalization processing on the image by the local histogram equalization unit to obtain the second image includes:
the local image segmentation module equally divides the image into a plurality of sub-images;
the local histogram calculation module counts pixels of the sub-images;
the local histogram segmentation module divides the sub-image into a local high-order sub-image and a local low-order sub-image;
the local bidirectional limited calculation module respectively calculates and limits the dynamic range of the local high-order sub-image and the dynamic range of the local low-order sub-image;
the local bidirectional limited equalization module equalizes the local high-order sub-image according to the dynamic range of the local high-order sub-image, equalizes the local low-order sub-image according to the dynamic range of the local low-order sub-image, fuses the equalized local high-order sub-histogram and the equalized local low-order sub-histogram, and applies the fused local high-order sub-histogram and the equalized local low-order sub-histogram to the sub-image to obtain a sub-image after local equalization processing;
and the local image fusion module performs linear interpolation processing on the sub-images subjected to the local equalization processing and fuses the sub-images subjected to the linear interpolation processing.
Since the image contrast is enhanced globally during the global histogram equalization process, some local features are not enhanced as desired. In order to further enhance the details of the image and enhance the local contrast of the image, it is necessary to further perform local histogram equalization processing on the input image. The local equalization processing method needs to divide an input image into a plurality of local modules, and then performs equalization processing on each local module independently. The length and width of the input image can be equally divided into 8 parts according to the length and width value of the input image, and 8 rows and 8 columns can obtain 64 partial images with the aspect ratio of the input image maintained. The local histogram calculation module counts the pixels of each sub-image; the local histogram segmentation module divides the sub-image into a local high-order sub-image and a local low-order sub-image; the local bidirectional limited calculation module respectively calculates and limits the dynamic ranges of the local high-order sub-image and the local low-order sub-image; the method for equalizing the local high-order sub-image according to the dynamic range of the local high-order sub-image and the method for equalizing the local low-order sub-image according to the dynamic range of the local low-order sub-image are the same as the global histogram equalization method, and therefore, the details are not repeated here.
Further, the method for performing linear interpolation processing on the sub-image after the local equalization processing by the local image fusion module includes: dividing the sub-image subjected to the local equalization processing into an edge area, a four-corner area and a middle area; and performing interpolation operation on the pixel points in the edge area, the pixel points in the four corner areas and the pixel points in the middle area respectively.
Because the local equalization processing method is to process the single local image, and a very obvious blocking effect can remain between the sub-images, the linear interpolation processing between the sub-images needs to be carried out, so that the value after each pixel point transformation is obtained by the bilinear interpolation of 4 sub-images around the pixel point, and the influence of the blocking effect is eliminated. The pixel points in the four corner regions refer to the pixel points at the four corners in the pixel block schematic diagram of the sub-image, and for the pixel points in the four corner regions, the calculated pixel points are still the pixel points. The pixel point of the edge area refers to a pixel point at the edge in the pixel block schematic diagram of the sub-image, and for the pixel point of the edge area, the calculated value is the interpolation value of the pixel point and the adjacent edge pixel of the adjacent local image. The pixel point in the middle area refers to a pixel point in the middle of the pixel block schematic diagram of the sub-image. And for the pixel points in the middle area, the calculated pixel points are bilinear interpolation of the mapping curves of the current sub-image and the three sub-images nearest to the current sub-image. And after the pixel point processing of each sub-image is finished, fusing the sub-images subjected to the linear interpolation processing together to obtain a second image.
Further, the method for fusing the first image and the second image by the scene similarity unit comprises the following steps:
the scene similarity calculation module acquires an information window with the current pixel point as the center according to window parameters configured by a user and calculates a scene similarity value;
the similarity weight calculation module judges whether the current pixel is in a uniform pixel area according to the scene similarity value;
the global histogram is weighted more if in the homogeneous pixel region and the local histogram is weighted more if in the heterogeneous pixel region.
Further, the method for obtaining the scene similarity value includes:
calculating absolute difference values of all pixel points and the current pixel point in the information window;
and respectively acquiring the weight of the first image and the weight of the second image according to the absolute difference value.
Specifically, the method for obtaining the information window centered on the current point according to the window parameter configured by the user comprises the following steps: taking the window parameter wSize ═ 5 as an example, as shown in fig. 6, a 5 × 5 information window centered on the current pixel point D12 calculates a scene similarity value according to the pixel levels of the current pixel point and its neighboring pixel points. The formula for calculating the absolute difference value of the pixel grades of all the pixel points and the current pixel point in the information window is as follows:
AbsDiff(x)＝|D(x)-D(12)|，
wherein: absdiff (x) is the absolute difference of the pixel levels of all the pixel points and the current pixel point in the information window; d (x) is the pixel grade of the adjacent pixel point of the current pixel point; d (12) is the pixel grade of the current pixel point.
Then the value of the scene similarity corresponding to the current pixel point D12 is calculated as follows:
wherein: the global standard deviation of the image, that is, the standard deviation of the pixel level of the whole image, can be calculated by the prior art, and is not described herein again; sim is the value of scene similarity; absdiff (x) is the absolute difference between the pixel levels of all the pixel points in the information window and the current pixel point.
Further, the weight of the first image is proportional to the value of the scene similarity. If the scene similarity value is large, it indicates that the current pixel point is in a uniform region, and it is expected that good brightness can be maintained, and more data of the image after global histogram equalization processing should be combined, so that the given weight is large. If the scene similarity value of the current pixel point is small, the current pixel point is in a non-uniform area (detail area), and the data of the image after the local histogram equalization processing is mainly combined, so that the given weight is small.
More specifically, a calculation formula for calculating the weight of the first image from the scene similarity value is as follows:
w _ sim ═ p × sim, where p is a user-configured weighting factor, with a default value of 0.4; w _ sim is the weight of the first image; sim is the value of scene similarity.
Finally, the image output unit fuses the first image and the second image according to the weight occupied by the first image and the weight occupied by the second image to obtain an enhanced image, and the formula is as follows:
i _ enhance ═ W _ sim × I _ global + (1-W _ sim) × I _ local, where: i _ enhance is an enhanced image; w _ sim is the weight of the first image; i _ global is the first picture and I _ local is the second picture.
In summary, in the image enhancement method based on histogram equalization provided in the embodiment of the present invention, the global histogram equalization processing is performed on the input image, meanwhile, the local histogram equalization processing is performed on the input image, and then the image after the global histogram equalization processing and the image after the local histogram equalization processing are combined according to the value of the scene similarity, so as to output the enhanced image, thereby enhancing the contrast of the image; the image details can be kept; while allowing the image to have good overall brightness. Furthermore, in the local histogram equalization processing of the present invention, the local high-order sub-image and the local low-order sub-image are equalized by performing bidirectional limitation on the local high-order sub-image and the local low-order sub-image, and the problems of supersaturation, amplification of noise in a dark area, and the like can be suppressed, thereby further improving the overall effect of the image.
The above description is only a preferred embodiment of the present invention, and does not limit the present invention in any way. It will be understood by those skilled in the art that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (14)
1. An image enhancement method based on histogram equalization, characterized in that the image enhancement method comprises the following steps:
an image input unit inputs an image;
the global histogram equalization unit performs global equalization processing on the image to obtain a first image;
the local histogram equalization unit performs local equalization processing on the image to obtain a second image;
the scene similarity unit calculates the weight occupied by the first image and the weight occupied by the second image;
and the image output unit fuses the first image and the second image according to the weight occupied by the first image and the weight occupied by the second image to obtain an enhanced image.
2. The image enhancement method based on histogram equalization as claimed in claim 1, wherein the global histogram equalization unit performs global equalization processing on the image to obtain the first image, and the method comprises:
the global histogram calculation module counts the pixel distribution of the whole image and forms a global histogram;
the histogram segmentation module segments the global histogram into a global high-order sub-histogram and a global low-order sub-histogram;
the bidirectional limited calculation module respectively calculates the dynamic range for limiting the global high-order sub-histogram and the dynamic range for limiting the global low-order sub-histogram;
and the bidirectional limited equalization module equalizes the global high-order sub-histogram according to the dynamic range of the global high-order sub-histogram, equalizes the global low-order sub-histogram according to the dynamic range of the global low-order sub-histogram, fuses the equalized global high-order sub-histogram and the equalized global low-order sub-histogram, and applies the fused global high-order sub-histogram and the equalized global low-order sub-histogram to the input image to obtain a first image.
3. The histogram equalization-based image enhancement method as claimed in claim 2, wherein the method for forming a global histogram by counting the pixel distribution of the whole image by the global histogram calculation module comprises:
calculating the total number of pixel points of the whole image;
counting the number of pixels corresponding to the pixel level according to the pixel level of the image;
and forming a global histogram by using the pixel grade and the ratio of the number of pixels corresponding to the pixel grade to the total number of the pixels.
4. The histogram equalization-based image enhancement method as claimed in claim 3, wherein the method for partitioning the global histogram into the global upper sub-histogram and the global lower sub-histogram by the histogram partitioning module comprises:
calculating the threshold values of the separation points of the global high-order sub-histogram and the global low-order sub-histogram according to the pixel level and the pixel level ratio of the global histogram;
the threshold serves as a reference point that separates the global histogram into a global upper sub-histogram and a global lower sub-histogram.
5. The histogram equalization-based image enhancement method as claimed in claim 4, wherein the method for respectively calculating and limiting the dynamic range of the global upper sub-histogram and the dynamic range of the global lower sub-histogram by the bidirectional limited calculation module comprises:
and obtaining the dynamic range for limiting the global high-order sub-histogram and the dynamic range for limiting the global low-order sub-histogram by using the threshold, the highest pixel level of the image, the lowest pixel level of the image and the number of pixel levels of the coding format corresponding to the image.
6. The histogram equalization-based image enhancement method according to claim 5, wherein a formula that limits the dynamic range of the global high-order sub-histogram to satisfy is calculated as follows:
wherein αH1、αH2For a bi-directional limited intensity parameter configured by the user, the input range is [0,3 ]](ii) a n is the pixel grade number of the coding format corresponding to the image; th is a threshold value of the separation point; histMinHTo limit the lower limit of the dynamic range of the global high sub-histogram; histMaxHTo limit the upper limit of the dynamic range of the global high sub-histogram; histMin is the lowest pixel level of the image; histMax is the highest pixel level of the image.
7. The histogram equalization-based image enhancement method according to claim 6, wherein a formula that limits the dynamic range of the global lower sub-histogram to be satisfied is calculated as follows:
wherein αL1、αL2For a bi-directional limited intensity parameter configured by the user, the input range is [0,3 ]](ii) a n is the pixel grade number of the coding format corresponding to the image; th is a threshold value of the separation point; histMinLTo limit the lower limit of the dynamic range of the global lower sub-histogram; histMaxLTo limit the upper limit of the dynamic range of the global lower sub-histogram; histMin is the lowest pixel level of the image; histMax is the highest pixel level of the image.
8. The histogram equalization-based image enhancement method of claim 7, wherein the method for equalizing the global upper sub-histogram according to the dynamic range of the global upper sub-histogram and equalizing the global lower sub-histogram according to the dynamic range of the global lower sub-histogram by a bi-directional limited equalization module comprises:
expanding the global high-order sub-histogram according to the dynamic range of the global high-order sub-histogram, and expanding the global low-order sub-histogram according to the dynamic range of the global low-order sub-histogram;
counting the proportion values corresponding to all pixel levels in the global low-order sub-histogram, cutting the proportion values which are larger than a first preset value, and averagely adding the accumulated cut values to each proportion value of the expanded global low-order sub-histogram;
and counting the proportion values corresponding to all the pixel levels in the global high-order sub-histogram, clipping the proportion values which are larger than a second preset value, and averagely adding the accumulated clipped values to each of the expanded proportion values in the global high-order sub-histogram.
9. A method for enhancing images based on histogram equalization as defined in claim 8, wherein fusing the equalized global upper sub-histogram and the equalized global lower sub-histogram, and applying the fused global upper sub-histogram and equalized global lower sub-histogram to the inputted image to obtain the first image comprises:
obtaining an accumulated histogram of the expanded global high-order sub-histogram and an accumulated histogram of the expanded global low-order sub-histogram;
fusing the cumulative histogram of the expanded global high-order sub-histogram with the cumulative histogram of the expanded global low-order sub-histogram;
and mapping the fused cumulative histogram as a mapping curve to the input image to obtain a first image.
10. The image enhancement method based on histogram equalization as claimed in claim 1, wherein the local histogram equalization unit performs local equalization processing on the image to obtain the second image, and the method comprises:
the local image segmentation module equally divides the image into a plurality of sub-images;
the local histogram calculation module counts pixels of each sub-image;
the local histogram segmentation module divides the sub-image into a local high-order sub-image and a local low-order sub-image;
the local bidirectional limited calculation module respectively calculates the dynamic range for limiting the local high-order sub-image and the dynamic range for limiting the local low-order sub-image;
the local bidirectional limited equalization module equalizes the local high-order sub-image according to the dynamic range of the local high-order sub-image, equalizes the local low-order sub-image according to the dynamic range of the local low-order sub-image, fuses the equalized local high-order sub-histogram and the equalized local low-order sub-histogram, and then applies the fused local high-order sub-histogram and the equalized local low-order sub-histogram to the sub-image to obtain a sub-image subjected to local equalization processing;
and the local image fusion module performs linear interpolation processing on the sub-images subjected to the local equalization processing and fuses the sub-images subjected to the linear interpolation processing.
11. The histogram equalization-based image enhancement method as claimed in claim 10, wherein the method for performing linear interpolation processing on the sub-image after the local equalization processing by the local image fusion module comprises:
dividing the sub-image subjected to the local equalization processing into an edge area, a four-corner area and a middle area;
and performing interpolation operation on the pixel points in the edge area, the pixel points in the four corner areas and the pixel points in the middle area respectively.
12. The histogram equalization-based image enhancement method as claimed in claim 1, wherein the method for fusing the first image and the second image by the scene similarity unit comprises:
the scene similarity calculation module acquires an information window with the current pixel point as the center according to window parameters configured by a user and calculates a scene similarity value;
the similarity weight calculation module judges whether the current pixel is in a uniform pixel area according to the scene similarity value;
if the histogram is in the uniform pixel region, the weight of the global histogram is larger; the local histogram is weighted more if in a non-uniform pixel region.
13. The histogram equalization based image enhancement method according to claim 12, wherein the method of obtaining the scene similarity value comprises:
calculating the absolute difference value of the pixel grades of all the pixel points and the current pixel point in the information window;
and acquiring the weight of the first image and the weight of the second image according to the absolute difference value.
14. The histogram equalization based image enhancement method of claim 13, wherein the weight of the first image is proportional to the value of the scene similarity.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010378155.4A CN111583162B (en) | 2020-05-07 | 2020-05-07 | Image Enhancement Method Based on Histogram Equalization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010378155.4A CN111583162B (en) | 2020-05-07 | 2020-05-07 | Image Enhancement Method Based on Histogram Equalization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111583162A true CN111583162A (en) | 2020-08-25 |
| CN111583162B CN111583162B (en) | 2023-09-01 |
Family
ID=72112049
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010378155.4A Active CN111583162B (en) | 2020-05-07 | 2020-05-07 | Image Enhancement Method Based on Histogram Equalization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111583162B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113902635A (en) * | 2021-09-29 | 2022-01-07 | 浙江双视红外科技股份有限公司 | A kind of infrared thermal imager image processing method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110206280A1 (en) * | 2007-05-03 | 2011-08-25 | Ho-Young Lee | Image brightness controlling apparatus and method thereof |
| CN105654438A (en) * | 2015-12-27 | 2016-06-08 | 西南技术物理研究所 | Gray scale image fitting enhancement method based on local histogram equalization |
| CN107301635A (en) * | 2017-06-28 | 2017-10-27 | 武汉格物优信科技有限公司 | A kind of infrared image detail enhancing method and device |
-
2020
- 2020-05-07 CN CN202010378155.4A patent/CN111583162B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110206280A1 (en) * | 2007-05-03 | 2011-08-25 | Ho-Young Lee | Image brightness controlling apparatus and method thereof |
| CN105654438A (en) * | 2015-12-27 | 2016-06-08 | 西南技术物理研究所 | Gray scale image fitting enhancement method based on local histogram equalization |
| CN107301635A (en) * | 2017-06-28 | 2017-10-27 | 武汉格物优信科技有限公司 | A kind of infrared image detail enhancing method and device |
Non-Patent Citations (2)
| Title |
|---|
| 江巨浪;张佑生;薛峰;胡敏;: "保持图像亮度的局部直方图均衡算法" * |
| 潘强;印鉴;: "基于权重约束决策的图像增强算法" * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113902635A (en) * | 2021-09-29 | 2022-01-07 | 浙江双视红外科技股份有限公司 | A kind of infrared thermal imager image processing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111583162B (en) | 2023-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bora | Importance of image enhancement techniques in color image segmentation: a comprehensive and comparative study | |
| US10339643B2 (en) | Algorithm and device for image processing | |
| DE602004002618T2 (en) | Method and system for edge enhancement | |
| Lin et al. | Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation | |
| US9984445B2 (en) | Tone mapping | |
| US8831346B2 (en) | Image processing apparatus and method, and program | |
| US6289110B1 (en) | Object extracting method using motion picture | |
| US9147238B1 (en) | Adaptive histogram-based video contrast enhancement | |
| CN101473347B (en) | Picture enhancing increasing precision smooth profiles | |
| US9875523B2 (en) | Image processing apparatus and image processing method | |
| US9025903B2 (en) | Image processing device and image processing method | |
| JP2012226769A (en) | Image identifier extraction device | |
| Tsai | Adaptive local power-law transformation for color image enhancement | |
| US20110116713A1 (en) | Image contrast enhancement apparatus and method thereof | |
| JPWO2008105222A1 (en) | Noise reduction device, noise reduction method, and noise reduction program | |
| CN103312963A (en) | Image processing device and image processing method | |
| CN105335933A (en) | Image contrast enhancement method and apparatus | |
| CN106204551A (en) | A kind of image significance detection method and device | |
| US8213736B2 (en) | Image processing device and image processing method | |
| Das et al. | Histogram equalization techniques for contrast enhancement: a review | |
| CN111583162A (en) | Image enhancement method based on histogram equalization | |
| CN109615599B (en) | Bilateral filtering algorithm based on classification optimization and hyperspectral image space spectrum feature extraction method | |
| US9589324B1 (en) | Overshoot protection of upscaled images | |
| US9154671B2 (en) | Image processing apparatus, image processing method, and program | |
| CN110163874B (en) | Bilateral filtering algorithm based on homogeneous region segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |