CN116913197B - Pixel positioning method, pixel brightness extraction method and system - Google Patents
Pixel positioning method, pixel brightness extraction method and system Download PDFInfo
- Publication number
- CN116913197B CN116913197B CN202311187897.9A CN202311187897A CN116913197B CN 116913197 B CN116913197 B CN 116913197B CN 202311187897 A CN202311187897 A CN 202311187897A CN 116913197 B CN116913197 B CN 116913197B
- Authority
- CN
- China
- Prior art keywords
- sub
- pixel
- pixels
- brightness
- display panel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000000605 extraction Methods 0.000 title claims abstract description 39
- 238000009826 distribution Methods 0.000 claims abstract description 6
- 239000003086 colorant Substances 0.000 claims description 27
- 238000012937 correction Methods 0.000 claims description 6
- 238000005286 illumination Methods 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 13
- 239000011324 bead Substances 0.000 description 9
- 238000003860 storage Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000004590 computer program Methods 0.000 description 6
- 230000007547 defect Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000008439 repair process Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 241001270131 Agaricus moelleri Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 239000012769 display material Substances 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
- G09G3/2007—Display of intermediate tones
- G09G3/2074—Display of intermediate tones using sub-pixels
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
- G09G3/22—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources
- G09G3/30—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources using electroluminescent panels
- G09G3/32—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources using electroluminescent panels semiconductive, e.g. using light-emitting diodes [LED]
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2320/00—Control of display operating conditions
- G09G2320/02—Improving the quality of display appearance
- G09G2320/0271—Adjustment of the gradation levels within the range of the gradation scale, e.g. by redistribution or clipping
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Control Of Indicators Other Than Cathode Ray Tubes (AREA)
Abstract
The invention provides a pixel positioning method, a pixel brightness extraction method and a pixel brightness extraction system, which belong to the field of image processing and are applied to a display panel, wherein the display panel comprises a plurality of pixels, each pixel comprises at least one color sub-pixel, and the method comprises the following steps: acquiring a sub-pixel brightness map of a preset color when sub-pixels are lighted by a display panel at intervals; the interval lighting is that the same-color sub-pixels which are not lighted exist between two adjacent lighted sub-pixels with the same color; positioning the lighted sub-pixel positions based on the sub-pixel brightness map; and predicting the unlit sub-pixel positions according to the pixel distribution condition of the display panel and the positioned sub-pixel positions to obtain the positions of all the sub-pixels. The invention lights the sub-pixels at intervals instead of all the sub-pixels when the sub-pixels are positioned and the brightness of the sub-pixels is extracted, can reduce the inter-pixel crosstalk and improve the positioning precision and the brightness extraction precision of the sub-pixels.
Description
Technical Field
The invention belongs to the field of image processing, and particularly relates to a pixel positioning method, a pixel brightness extraction method and a pixel brightness extraction system.
Background
With the continuous increase of social demands, the development of display technology is rapid. Organic Light-Emitting Diode (OLED) and Micro Light-Emitting Diode (Micro Light Emitting Diode Display, micro LED) are important display materials for small-area display panels of mobile phones, tablet computers, televisions, and the like; OLED or Micro LED is used as a bead on a display panel, and the finished product of the display panel needs to ensure that no defect or defect can flow into the market within an allowable range, and the occurrence of Mura defect is unavoidable due to the complexity and difficulty of the production process of the display panel, so that Mura defect repair is necessary, that is, the luminance uniformity correction (DeMura) technology is applied.
In the DeMura process, sub-pixel positioning and sub-pixel brightness extraction are used as key steps, positioning accuracy and brightness map extraction accuracy directly influence the final repairing effect, the sub-pixel positioning is specifically used for determining the accurate sub-pixel position corresponding to the center of each lamp bead on a shot image of a display panel, and the sub-pixel brightness extraction is specifically used for extracting the brightness of each sub-pixel after the display panel is lightened. However, as the area of the display panel is smaller and smaller, the distance between the lamp beads is smaller and smaller, and accordingly the pixel interval (Pitch) of the sub-pixels on the photographed image is very small, so that the inter-pixel crosstalk is very serious, the positioning accuracy of the sub-pixels and the brightness extraction accuracy of the sub-pixels are affected, and the DeMura repairing effect is poor.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a pixel positioning method, a pixel brightness extraction method and a system, and aims to solve the problems that the pixel interval of sub-pixels on the existing display panel is small, the inter-pixel crosstalk is serious, the sub-pixel positioning and the brightness extraction are affected, and the Demura restoration effect is poor.
To achieve the above object, in a first aspect, the present invention provides a pixel positioning method applied to a display panel including a plurality of pixels each including a sub-pixel of at least one color, the method comprising:
acquiring a sub-pixel brightness map of a preset color when sub-pixels are lighted by a display panel at intervals; the interval lighting is that the same-color sub-pixels which are not lighted exist between two adjacent lighted sub-pixels with the same color;
positioning the lighted sub-pixel positions based on the sub-pixel brightness map;
and predicting the unlit sub-pixel positions according to the pixel distribution condition of the display panel and the positioned sub-pixel positions to obtain the positions of all the sub-pixels.
The invention can light the sub-pixels at intervals, avoid crosstalk caused by too close distance between the lightened sub-pixels, and improve the positioning accuracy of the sub-pixels. In addition, the invention predicts the position of the non-lighting sub-pixel by using the positioning of the lighting sub-pixel, avoids the position prediction of all the lighting sub-pixels, greatly reduces the calculation time, predicts the position of the non-lighting sub-pixel by using the lighting sub-pixel with relatively accurate positioning, ensures that the position prediction of the non-lighting sub-pixel is relatively accurate, and improves the positioning precision and the positioning time of the sub-pixel.
It will be appreciated that the positioning of the bright sub-pixels can be seen in the related art. For example, the sub-pixels can be positioned by the display gray scale of the sub-pixels, and when the display gray scale of the sub-pixels is larger than a preset value, the corresponding position is represented as the center position of the lamp beads corresponding to the sub-pixels. Of course, those skilled in the art may select other positioning manners to position the bright sub-pixels according to actual needs, and the present invention does not limit or protect the specific positioning manners.
In one possible implementation, obtaining a subpixel luminance map when subpixels of a preset color are illuminated at intervals includes:
if each pixel comprises a sub-pixel with one color, controlling the display panel to light the sub-pixels at intervals, and shooting the display panel to obtain a corresponding sub-pixel brightness map;
or if each pixel comprises sub-pixels with multiple colors, controlling the display panel to independently light the sub-pixels with preset colors at intervals, and taking the image of the display panel to obtain a corresponding sub-pixel brightness map;
or if each pixel comprises a plurality of color sub-pixels, respectively and independently controlling the display panel to light the preset color sub-pixels at intervals for each color sub-pixel, and taking the image of the display panel to obtain a sub-pixel brightness map of each color;
or if each pixel comprises the sub-pixels with multiple colors, the display panel is controlled to synchronously light the sub-pixels with all the colors at intervals, and the display panel is taken to obtain a corresponding sub-pixel brightness map.
Note that, for a single-color or multi-color display panel, the manner of lighting the positioning sub-pixels at intervals is applicable.
Further, for a multi-color display panel, the sub-pixels of the various colors may be respectively lighted at intervals to obtain luminance maps of the sub-pixels of the various colors, or the sub-pixel luminance maps of the various colors may be respectively extracted after synchronously lighting at intervals.
For example, when a black-and-white camera is used to capture an image of a display panel, sub-pixels of different colors may be respectively lighted at intervals to extract sub-pixel luminance maps of the respective colors. When a black-and-white camera shoots, the pixels with different colors display gray scales or display gray scale values.
For example, when a color camera is used for shooting an image of a display panel, sub-pixels with different colors can be respectively lightened at intervals so as to extract sub-pixel brightness maps with various colors; the sub-pixels of the plurality of colors may be simultaneously lighted at intervals, and then the sub-pixel luminance maps of the respective color channels may be extracted from the captured image.
In one possible implementation, the subpixel luminance map is deblurred based on a point spread function (Point Spread Function, PSF) prior to locating the position of the bright subpixels.
It should be noted that, if the sub-pixels are positioned directly based on the sub-pixel luminance map, since the sub-pixels are turned on at intervals, there may be a case where the luminance of the single sub-pixel is blurred, and at this time, there may be some errors in positioning. Therefore, PSF can be adopted to deblur the sub-pixel brightness map, and then positioning is carried out, so that the positioning accuracy of the sub-pixels is further improved.
In one possible implementation, deblurring a subpixel luminance map based on a point spread function includes:
predicting a corresponding point spread function according to the real image of the interval lighting sub-pixels of the display panel and the taken image of the sub-pixel brightness map;
deconvolution is carried out on the picked-up sub-pixel brightness map and the predicted point spread function to obtain a deblurred sub-pixel brightness map.
It will be appreciated that the actual image of the display panel interval lighting sub-pixels may be generated by a code, i.e. according to preset lighting logic.
In one possible implementation, the interval lighting method includes: the same color subpixels are illuminated at intervals of at least one row and/or at intervals of at least one column.
In one possible implementation, predicting unlit subpixel positions includes:
and combining the arrangement condition of the sub-pixels on the display panel, and interpolating based on the positions of the positioned sub-pixels to obtain the positions of the non-lighted sub-pixels.
It should be noted that, in general, the sub-pixels are regularly arranged on the display panel, so that the position of the non-lit sub-pixel can be predicted by interpolation.
Specifically, interpolation algorithms such as linear interpolation, spline interpolation, etc. may be used to predict the location of the unlit sub-pixels.
In one possible implementation, the display panel is a Micro LED display.
In a second aspect, the present invention provides a pixel brightness extraction method, including:
determining the position of each sub-pixel on the display panel based on the pixel positioning method described in the first aspect or any one of the possible implementation manners of the first aspect;
controlling the display panel to light sub-pixels of a preset color at intervals for multiple times so as to light all the sub-pixels of the preset color in multiple times and obtain a multi-Zhang Zixiang-pixel brightness map of the preset color;
obtaining sub-pixel brightness of corresponding positions according to a preset color multi-Zhang Zixiang-pixel brightness map and the determined positions of all sub-pixels on the display panel;
correcting the brightness of the sub-pixels extracted from each sub-pixel brightness map according to a preset brightness coefficient to obtain the final brightness of the position of each sub-pixel with a preset color; the brightness coefficient is used for ensuring that the sub-pixel brightness map has preset consistency when the multi-Zhang Zixiang-pixel brightness map is combined and then is lightened together with all sub-pixels of preset colors.
If the sub-pixels are turned on by interval lighting during the sub-pixel brightness extraction process, crosstalk between the sub-pixels can be avoided greatly during sub-pixel brightness extraction, and the brightness extraction accuracy of the sub-pixels can be improved. However, the load is different between the full screen lighting and the interval lighting, and there is a problem of unbalanced load; the load here refers to the total number of lamp beads. When the loads are different, the currents of the lamp beads are different under the same voltage, and the corresponding display brightness is different, so that the brightness extracted when the single sub-pixels are lighted at intervals is different from the brightness extracted when the single sub-pixels are lighted in a full screen.
It can be understood that the brightness of the sub-pixels extracted during interval lighting is corrected through the preset brightness coefficient, and load balancing compensation is performed, so that the inconsistency of brightness graphs of the sub-pixels obtained during interval lighting and when the sub-pixels are combined and then are lighted together with all the sub-pixels is avoided.
In one possible implementation, the luminance coefficients corresponding to the sub-pixels of different regions are different.
Specifically, the subpixels of different regions refer to subpixels of different positions, or all subpixels that are lit at certain intervals. In general, the luminance coefficients of all the sub-pixels that are lighted at the same sub-interval are uniform, and the luminance coefficients of the sub-pixels that are lighted at different sub-intervals are different. But it is also possible that the luminance coefficients differ between sub-pixels that are lit at the same interval. The person skilled in the art can set the corresponding brightness coefficient according to the actual situation, and the present invention is not limited thereto.
In a third aspect, the present invention provides a pixel brightness extraction system, comprising:
a sub-pixel positioning unit, configured to determine a position of each sub-pixel on the display panel based on the pixel positioning method described in the first aspect or any one of the possible implementation manners of the first aspect;
the sub-pixel lighting unit is used for controlling the display panel to light sub-pixels of a preset color at intervals for a plurality of times so as to light all the sub-pixels of the preset color for a plurality of times and obtain a multi-Zhang Zixiang-pixel brightness map of the preset color;
a sub-pixel brightness extraction unit, configured to obtain sub-pixel brightness at a corresponding position according to a multi-Zhang Zixiang-pixel brightness map of a preset color and the determined positions of the sub-pixels on the display panel;
the sub-pixel brightness correction unit is used for correcting the brightness of the sub-pixels extracted from each sub-pixel brightness map according to a preset brightness coefficient to obtain the final brightness of the position of each sub-pixel with the preset color; the brightness coefficient is used for ensuring that the sub-pixel brightness map has preset consistency when the multi-Zhang Zixiang-pixel brightness map is combined and then is lightened together with all sub-pixels of preset colors.
In a fourth aspect, the present invention provides an electronic device comprising: at least one memory for storing a program; at least one processor configured to execute a program stored in a memory, the processor configured to perform the method described in the first aspect, any one of the possible implementations of the first aspect, the second aspect, or any one of the possible implementations of the second aspect when the program stored in the memory is executed.
In a fifth aspect, the present invention provides a computer readable storage medium storing a computer program which, when run on a processor, causes the processor to perform the method described in the first aspect or any one of the possible implementations of the first aspect.
In a sixth aspect, the invention provides a computer program product which, when run on a processor, causes the processor to perform the method described in the first aspect, any one of the possible implementations of the first aspect, the second aspect or any one of the possible implementations of the second aspect.
It will be appreciated that the advantages of the third to sixth aspects may be found in the relevant description of the first and second aspects, and are not described here again.
In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:
the invention provides a pixel positioning method, a pixel brightness extraction method and a pixel brightness extraction system, which avoid crosstalk caused by too close distance between lighted sub-pixels by controlling the interval lighting of the sub-pixels, thereby improving the positioning accuracy of the sub-pixels. In addition, the invention predicts the position of the non-lighted sub-pixel by utilizing the positioning of the lighted sub-pixel, avoids the position prediction of all the lighted sub-pixels, greatly reduces the calculation time, and predicts the position of the non-lighted sub-pixel by utilizing the lighted sub-pixel with relatively accurate positioning, so that the position prediction of the non-lighted sub-pixel is relatively accurate, and the positioning precision and the positioning time of the sub-pixel are improved.
The invention provides a pixel positioning method, a pixel brightness extraction method and a system, wherein the sub-pixels are lightened in a spaced-lightening mode in the process of sub-pixel brightness extraction, so that crosstalk between the sub-pixels can be avoided greatly during sub-pixel brightness extraction, and the brightness extraction precision of the sub-pixels is improved. The brightness of the sub-pixels extracted during interval lighting is corrected through the preset brightness coefficient, and load balancing compensation is carried out, so that the situation that the brightness of the sub-pixels obtained during interval lighting is inconsistent with the brightness map when all the sub-pixels are lighted together after being combined is avoided.
The invention provides a pixel positioning method, a pixel brightness extraction method and a pixel brightness extraction system, which improve the positioning precision and the brightness extraction precision of sub-pixels and can further improve the repairing effect of a display panel. The pixel positioning method and the brightness extraction method provided by the invention can be automatically adapted to different screen types, and manual operation is not needed, so that usability and reusability are improved.
Drawings
FIG. 1 is a flowchart of a pixel positioning method according to an embodiment of the present invention;
FIG. 2 (a) is a schematic diagram illustrating an example of sub-pixel interval lighting provided in an embodiment of the present invention;
FIG. 2 (b) is another exemplary schematic diagram of sub-pixel interval lighting provided by an embodiment of the present invention;
FIG. 2 (c) is a schematic diagram illustrating still another example of sub-pixel interval lighting provided by an embodiment of the present invention;
FIG. 3 is a schematic view illustrating an image capturing of a sub-pixel of a display panel according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of deblurring a captured image according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating the positioning of the illuminated sub-pixel locations according to an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating the positioning of unlit sub-pixel locations provided by an embodiment of the present invention;
FIG. 7 is a flowchart of a method for extracting pixel brightness according to an embodiment of the present invention;
FIG. 8 is a block diagram of a pixel location system according to an embodiment of the present invention;
fig. 9 is a schematic diagram of a pixel brightness extraction system according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The term "and/or" herein is an association relationship describing an associated object, and means that there may be three relationships, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. The symbol "/" herein indicates that the associated object is or is a relationship, e.g., A/B indicates A or B.
In embodiments of the invention, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.
First, technical terms involved in the embodiments of the present invention will be described.
The display panel consists of an array of individual light emitting elements called "pixels". A pixel refers to an individual light emitting element, consisting of one or more light emitting points. The sub-pixel refers to a light emitting point that emits a color, such as Red (Red, R), green (G), or Blue (Blue, B).
The protection scope of the technical scheme provided by the invention is not limited to specific colors of the sub-pixels, and the following embodiments are exemplified by three colors of R, G, B of the sub-pixels, and do not limit the protection scope of the invention.
Next, the technical scheme provided in the embodiment of the present invention is described.
FIG. 1 is a flowchart of a sub-pixel positioning method according to an embodiment of the present invention; as shown in fig. 1, the method comprises the following steps:
s101, obtaining a sub-pixel brightness map of a sub-pixel with a preset color when the sub-pixel is lighted by a display panel at intervals; the interval lighting is that the same-color sub-pixels which are not lighted exist between two adjacent lighted sub-pixels with the same color;
specifically, the interval lighting may be lighting at intervals, i.e., pixels are lighted at fixed line numbers and column numbers intervals. The invention does not limit the row and column number of the pixel interval, and the row number and the column number of the interval need to be more than or equal to 1.
FIGS. 2 (a) -2 (c) are three exemplary schematic diagrams of sub-pixel interval lighting provided by embodiments of the present invention; fig. 2 (a) illustrates the row-by-column lighting sub-pixels at intervals, fig. 2 (b) illustrates the row-by-column lighting sub-pixels at intervals, and fig. 2 (c) illustrates the row-by-column lighting sub-pixels at intervals.
It can be understood that the interval lighting method provided by the embodiment of the invention can be used for positioning a monochrome display panel and a color display panel, is not limited to a specific type of display panel, and can be applied to the method provided by the invention as long as the pixel interval lighting can be performed.
S102, positioning the lighted sub-pixel positions based on the sub-pixel brightness map;
fig. 3 is an image capturing schematic diagram of a sub-pixel of a display panel according to an embodiment of the present invention.
Further, when the lamp beads of the display panel are shot, optical blurring caused by the optical imaging system exists, so that focusing inconsistency and optical diffusion exist at different positions of the panel, and further, positioning precision errors become large. In order to solve the problems, prior to positioning, original image deblurring is performed based on PSF prediction, so that the accuracy of a positioning algorithm and the robustness to image capturing blurring are improved to a large extent.
In one specific example, the prediction model for the PSF is:. Where f is the actual display panel image, h is the actual acquired display panel image, and PSF is the predicted point spread function.
The specific deblurring operation modeling is as follows, and the actual image h and the PSF can obtain a deblurred image through deconvolution deconv operation:
fig. 4 is a schematic diagram of deblurring a captured image according to an embodiment of the present invention, where (a) in fig. 4 is a luminance map of a single pixel before deblurring, and (b) is a luminance map of a single pixel after deblurring.
Further, after the deblurring operation, the lit sub-pixels may be located based on the deblurred sub-pixel luminance map. The specific positioning results are shown in fig. 5. It can be seen in fig. 5 that the position of each lighting sub-pixel can be obtained.
S103, predicting the unlit sub-pixel positions according to the pixel distribution condition of the display panel and the positioned sub-pixel positions to obtain the positions of all the sub-pixels.
Illustratively, the unlit position may be predicted based on the lit bead positioning results, as shown in FIG. 6. Specifically, for pixels with different pixel arrangements, the unlit position can be predicted by interpolation based on different arrangements and spatial positions of the screen, and the interpolation method can be various, such as linear interpolation, spline interpolation, and other interpolation algorithms.
In a specific embodiment, the pixel positioning method provided by the invention can obtain the corresponding sub-pixel coordinates of each original lamp bead on the image. For example, a display panel of 4K (resolution: 2160×3840) is photographed using a 151MB color camera (bayer resolution: 10640× 14144). Numbering each sub-pixelP ij (i=1:2160, j=1:3840), and obtaining corresponding pixel coordinates for each sub-pixel location isQ ij （i=1,2...,10640, j=1,2...,14144）。
As can be seen from the description of the foregoing embodiments, the dot screen patterns do not need to be designed separately for different screen bodies, so that the usability is greatly improved, the adaptation to different screen bodies is simpler, and the adaptation period of different screen bodies is greatly shortened. Meanwhile, as only one dot screen pattern is used, different screen types can be applied, and the complexity and stability of the algorithm are greatly improved.
FIG. 7 is a flowchart of a method for extracting pixel brightness according to an embodiment of the present invention; as shown in fig. 7, the method comprises the following steps:
s701, determining the positions of all sub-pixels on the display panel based on the pixel positioning method shown in FIG. 1 and the specific example thereof;
it will be appreciated that for a sub-pixel of one color, the location of all sub-pixels in that color can be obtained by locating the sub-pixel luminance map obtained by one interval illumination. The sub-pixels can be positioned for multiple times through the sub-pixel brightness map which is lighted for multiple times at intervals, and then the average value of the positioning results is calculated for the multiple times.
S702, controlling the display panel to light sub-pixels of a preset color at intervals for multiple times so as to light all the sub-pixels of the preset color for multiple times, and obtaining a multi-Zhang Zixiang-pixel brightness map of the preset color;
s703, obtaining the sub-pixel brightness of the corresponding position according to the Zhang Zixiang pixel brightness map with preset colors and the determined positions of the sub-pixels on the display panel;
s704, correcting the brightness of the sub-pixels extracted from each sub-pixel brightness map according to a preset brightness coefficient to obtain the final brightness of the position of each sub-pixel with a preset color; the brightness coefficient is used for ensuring that the sub-pixel brightness map has preset consistency when the multi-Zhang Zixiang-pixel brightness map is combined and then is lightened together with all sub-pixels of preset colors.
Specifically, the luminance coefficients corresponding to the sub-pixels of different regions are different. The subpixels of different regions refer to subpixels of different positions, or all subpixels that are lit at certain intervals. In general, the luminance coefficients of all the sub-pixels that are lighted at the same sub-interval are uniform, and the luminance coefficients of the sub-pixels that are lighted at different sub-intervals are different. But it is also possible that the luminance coefficients differ between sub-pixels that are lit at the same interval. The person skilled in the art can set the corresponding brightness coefficient according to the actual situation, and the present invention is not limited thereto.
For example, after the positioning result of the sub-pixels is obtained, in the process of extracting the brightness of different positions of the display panel, because the sub-pixels are lighted at intervals, there may be load imbalance compared with the original full-screen lighting, so that the problem of inconsistent brightness between the sub-pixels lighted at intervals after the brightness of the sub-pixels lighted at intervals is combined is caused. Based on the problem, brightness fusion operation can be performed on brightness of different phases so as to avoid the problem of unbalanced load.
For example, if the sub-pixels are lit one by one at intervals based on the pixel brightness extracted by the interval lighting, the sub-pixels need to be lit four times at intervals to light all the sub-pixels. At this time, the brightness is 4 equal, and the brightness coefficient correction operation can be performed on different phases to solve the problem of uneven load. Different phases here refer to different sub-interval lighting.
It should be noted that luminance fusion does not actually occur during the process of sub-pixel luminance extraction. According to the invention, the brightness of the extracted sub-pixels is corrected by configuring the brightness coefficient, so that the final brightness of the sub-pixels is obtained, and the problem that the brightness map of all the sub-pixels obtained by directly fusing the brightness of the sub-pixels extracted before correction is inconsistent with the brightness map obtained by lighting all the sub-pixels together is avoided.
It can be appreciated that the preset manner of the luminance coefficient may be: all the sub-pixels are lit up in advance to obtain a luminance map, from which the luminance of each sub-pixel is extracted. And then the sub-pixels are lighted at intervals, the brightness of the corresponding sub-pixels is extracted, and then the corresponding brightness coefficients are determined through comparison. The above process can be understood as a calibration process.
The invention provides a pixel positioning method and a pixel brightness extraction method, which adopt an interval lighting mode to position and extract the brightness of sub-pixels, and further carry out OLED Demura or Micro LED Demura repair experiments, and the obtained OLED Demura or Micro LED Demura repair precision is found to be obviously improved compared with the existing pixel positioning and pixel brightness extraction methods, and can be effectively adapted to different screen types.
Fig. 8 is a schematic diagram of a pixel positioning system according to an embodiment of the present invention, as shown in fig. 8, including:
a sub-pixel luminance map obtaining unit 810, configured to obtain a sub-pixel luminance map when sub-pixels of a preset color are illuminated by the display panel at intervals; the interval lighting is that the same-color sub-pixels which are not lighted exist between two adjacent lighted sub-pixels with the same color;
a sub-pixel positioning unit 820 for positioning the lighted sub-pixel position based on the sub-pixel luminance map; and predicting the unlit sub-pixel positions according to the pixel distribution condition of the display panel and the positioned sub-pixel positions to obtain the positions of all the sub-pixels.
Fig. 9 is a schematic diagram of a pixel brightness extraction system according to an embodiment of the present invention, as shown in fig. 9, including:
a subpixel positioning unit 910, configured to determine a position of each subpixel on the display panel based on the pixel positioning method described in the first aspect or any one of the possible implementation manners of the first aspect;
a sub-pixel lighting unit 920, configured to control the display panel to light sub-pixels of a preset color at intervals multiple times, so as to light all the sub-pixels of the preset color multiple times, and obtain a multi-Zhang Zixiang-pixel luminance map of the preset color;
a sub-pixel brightness extraction unit 930, configured to obtain sub-pixel brightness at a corresponding position according to the multi-Zhang Zixiang-pixel brightness map of the preset color and the determined positions of the respective sub-pixels on the display panel;
a sub-pixel brightness correction unit 940, configured to correct the brightness of the sub-pixel extracted from each sub-pixel brightness map according to a preset brightness coefficient, so as to obtain the final brightness of the position where each sub-pixel of the preset color is located; the brightness coefficient is used for ensuring that the sub-pixel brightness map has preset consistency when the multi-Zhang Zixiang-pixel brightness map is combined and then is lightened together with all sub-pixels of preset colors.
It should be noted that the detailed functional implementation of each unit in fig. 8 and fig. 9 may be referred to the description in the foregoing method embodiment, and are not described herein.
It should be understood that, the system is used to execute the method in the foregoing embodiment, and the corresponding program element in the system performs the principle and technical effects similar to those described in the foregoing method, and the working process of the system may refer to the corresponding process in the foregoing method, which is not repeated herein.
Based on the method in the above embodiment, the embodiment of the invention provides an electronic device. The apparatus may include: at least one memory for storing programs and at least one processor for executing the programs stored by the memory. Wherein the processor is adapted to perform the method described in the above embodiments when the program stored in the memory is executed.
Based on the method in the above embodiment, the embodiment of the present invention provides a computer-readable storage medium storing a computer program, which when executed on a processor, causes the processor to perform the method in the above embodiment.
Based on the method in the above embodiments, an embodiment of the present invention provides a computer program product, which when run on a processor causes the processor to perform the method in the above embodiments.
It is to be appreciated that the processor in embodiments of the invention may be a central processing unit (centralprocessing unit, CPU), other general purpose processor, digital signal processor (digital signalprocessor, DSP), application specific integrated circuit (application specific integrated circuit, ASIC), field programmable gate array (field programmable gate array, FPGA) or other programmable logic device, transistor logic device, hardware components, or any combination thereof. The general purpose processor may be a microprocessor, but in the alternative, it may be any conventional processor.
The steps of the method in the embodiment of the present invention may be implemented by hardware, or may be implemented by executing software instructions by a processor. The software instructions may be comprised of corresponding software modules that may be stored in random access memory (random access memory, RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable PROM (EPROM), electrically erasable programmable EPROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted across a computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.
It will be appreciated that the various numerical numbers referred to in the embodiments of the present invention are merely for ease of description and are not intended to limit the scope of the embodiments of the present invention.
It will be readily appreciated by those skilled in the art that the foregoing description is merely a preferred embodiment of the invention and is not intended to limit the invention, but any modifications, equivalents, improvements or alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (8)
1. A pixel brightness extraction method applied to a display panel including a plurality of pixels each including a sub-pixel of at least one color, the method comprising:
acquiring a sub-pixel brightness map of a preset color when sub-pixels are lighted by a display panel at intervals; the interval lighting is that the same-color sub-pixels which are not lighted exist between two adjacent lighted sub-pixels with the same color;
positioning the lighted sub-pixel positions based on the sub-pixel brightness map; predicting the unlit sub-pixel positions according to the pixel distribution condition of the display panel and the positioned sub-pixel positions to obtain the positions of all sub-pixels; predicting unlit subpixel positions includes: combining the arrangement condition of the sub-pixels on the display panel, and interpolating based on the positions of the positioned sub-pixels to obtain the positions of the sub-pixels which are not lightened;
obtaining a multi-Zhang Zixiang-pixel brightness map obtained by lighting all sub-pixels of a preset color at intervals in a dividing way, so as to obtain sub-pixel brightness of a corresponding position according to the determined position of each sub-pixel;
correcting the brightness of the sub-pixels extracted from each sub-pixel brightness map according to a preset brightness coefficient to obtain the final brightness of the position of each sub-pixel with a preset color; the brightness coefficient is used for ensuring that the sub-pixel brightness map has preset consistency when the multi-Zhang Zixiang-pixel brightness map is combined and is lightened together with all sub-pixels of preset colors, so as to compensate for load unbalance among different interval lightening; the preset mode of the brightness coefficient is as follows: and pre-lighting all the sub-pixels to obtain a brightness map, extracting the brightness of each sub-pixel, lighting the sub-pixels at intervals, extracting the brightness of the corresponding sub-pixels, and finally determining the corresponding brightness coefficient through comparison.
2. The method of claim 1, wherein obtaining a subpixel luminance map of a preset color when subpixels of the preset color are illuminated at intervals comprises:
if each pixel comprises a sub-pixel with one color, controlling the display panel to light the sub-pixels at intervals, and shooting the display panel to obtain a corresponding sub-pixel brightness map;
or if each pixel comprises sub-pixels with multiple colors, controlling the display panel to independently light the sub-pixels with preset colors at intervals, and taking the image of the display panel to obtain a corresponding sub-pixel brightness map;
or if each pixel comprises a plurality of color sub-pixels, respectively and independently controlling the display panel to light the preset color sub-pixels at intervals for each color sub-pixel, and taking the image of the display panel to obtain a sub-pixel brightness map of each color;
or if each pixel comprises the sub-pixels with multiple colors, the display panel is controlled to synchronously light the sub-pixels with all the colors at intervals, and the display panel is taken to obtain a corresponding sub-pixel brightness map.
3. The method of claim 1, wherein the sub-pixel intensity map is deblurred based on a point spread function prior to locating the position of the bright sub-pixel.
4. A method according to claim 3, wherein deblurring the sub-pixel luminance map based on a point spread function comprises:
predicting a corresponding point spread function according to the real image of the interval lighting sub-pixels of the display panel and the taken image of the sub-pixel brightness map;
deconvolution is carried out on the picked-up sub-pixel brightness map and the predicted point spread function to obtain a deblurred sub-pixel brightness map.
5. The method of any one of claims 1 to 4, wherein the means for interval illumination comprises: the same color subpixels are illuminated at intervals of at least one row and/or at intervals of at least one column.
6. The method of claim 1, wherein the display panel is a Micro light emitting diode Micro LED display.
7. A method as claimed in any one of claims 1 to 4, wherein the luminance coefficients for the sub-pixels of the different regions are different.
8. A pixel luminance extraction system based on the luminance extraction method according to any one of claims 1 to 6, comprising:
the sub-pixel positioning unit is used for acquiring a sub-pixel brightness map when sub-pixels with preset colors are lighted by the display panel at intervals; the interval lighting is that the same-color sub-pixels which are not lighted exist between two adjacent lighted sub-pixels with the same color; positioning the lighted sub-pixel positions based on the sub-pixel brightness map; predicting the unlit sub-pixel positions according to the pixel distribution condition of the display panel and the positioned sub-pixel positions to obtain the positions of all sub-pixels; predicting unlit subpixel positions includes: combining the arrangement condition of the sub-pixels on the display panel, and interpolating based on the positions of the positioned sub-pixels to obtain the positions of the sub-pixels which are not lightened;
the sub-pixel lighting unit is used for controlling the display panel to light sub-pixels of a preset color at intervals for a plurality of times so as to light all the sub-pixels of the preset color for a plurality of times and obtain a multi-Zhang Zixiang-pixel brightness map of the preset color;
a sub-pixel brightness extraction unit, configured to obtain sub-pixel brightness at a corresponding position according to a multi-Zhang Zixiang-pixel brightness map of a preset color and the determined positions of the sub-pixels on the display panel;
the sub-pixel brightness correction unit is used for correcting the brightness of the sub-pixels extracted from each sub-pixel brightness map according to a preset brightness coefficient to obtain the final brightness of the position of each sub-pixel with the preset color; the brightness coefficient is used for ensuring that the sub-pixel brightness map has preset consistency when the multi-Zhang Zixiang-pixel brightness map is combined and is lightened together with all sub-pixels of preset colors, so as to compensate for load unbalance among different interval lightening; the preset mode of the brightness coefficient is as follows: and pre-lighting all the sub-pixels to obtain a brightness map, extracting the brightness of each sub-pixel, lighting the sub-pixels at intervals, extracting the brightness of the corresponding sub-pixels, and finally determining the corresponding brightness coefficient through comparison.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311187897.9A CN116913197B (en) | 2023-09-15 | 2023-09-15 | Pixel positioning method, pixel brightness extraction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311187897.9A CN116913197B (en) | 2023-09-15 | 2023-09-15 | Pixel positioning method, pixel brightness extraction method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116913197A CN116913197A (en) | 2023-10-20 |
| CN116913197B true CN116913197B (en) | 2024-01-16 |
Family
ID=88353551
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311187897.9A Active CN116913197B (en) | 2023-09-15 | 2023-09-15 | Pixel positioning method, pixel brightness extraction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116913197B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119541393A (en) * | 2025-01-21 | 2025-02-28 | 深圳精智达技术股份有限公司 | Sub-pixel positioning method, device and system of display panel and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103927946A (en) * | 2014-03-25 | 2014-07-16 | 京东方科技集团股份有限公司 | Display method |
| CN104103236A (en) * | 2013-04-10 | 2014-10-15 | 三星显示有限公司 | Display apparatus |
| CN104915922A (en) * | 2015-06-25 | 2015-09-16 | 西安诺瓦电子科技有限公司 | Splicing bright-dark line correction method |
| CN105185315A (en) * | 2015-10-13 | 2015-12-23 | 西安诺瓦电子科技有限公司 | Uniformity correction method for LED display screen |
| CN105185314A (en) * | 2015-10-13 | 2015-12-23 | 西安诺瓦电子科技有限公司 | Uniformity compensation method for LED display screen |
| CN105551428A (en) * | 2016-02-03 | 2016-05-04 | 西安诺瓦电子科技有限公司 | LED display screen alternative point collection correction method |
| CN208903644U (en) * | 2018-10-19 | 2019-05-24 | 南京聚诚电子科技有限公司 | A kind of brightness point by point coloration acquisition correction device for small space distance LED display screen |
| JP2019186736A (en) * | 2018-04-10 | 2019-10-24 | キヤノン株式会社 | Image projection device |
| CN112071246A (en) * | 2020-09-21 | 2020-12-11 | 云谷（固安）科技有限公司 | Method and device for positioning sub-pixels of display panel and compensation method of display panel |
| CN114495803A (en) * | 2022-03-31 | 2022-05-13 | 武汉精立电子技术有限公司 | Mura repairing method of display panel |
| CN115050309A (en) * | 2022-02-24 | 2022-09-13 | 长春希达电子技术有限公司 | LED display screen correction method for adjusting configuration file during collection |
| KR20230087373A (en) * | 2021-12-09 | 2023-06-16 | 엘지디스플레이 주식회사 | Method Of Compensating Display Device |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5752263B2 (en) * | 2011-11-25 | 2015-07-22 | 三菱電機株式会社 | Brightness measuring method and brightness adjusting device for display element |
| CN103424916B (en) * | 2013-08-07 | 2016-01-06 | 京东方科技集团股份有限公司 | A kind of LCDs, its driving method and display device |
| CN103559849B (en) * | 2013-11-15 | 2016-08-17 | 北京京东方光电科技有限公司 | The display packing of display floater |
| JP6834542B2 (en) * | 2017-02-01 | 2021-02-24 | セイコーエプソン株式会社 | Image display device and its adjustment method |
-
2023
- 2023-09-15 CN CN202311187897.9A patent/CN116913197B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104103236A (en) * | 2013-04-10 | 2014-10-15 | 三星显示有限公司 | Display apparatus |
| CN103927946A (en) * | 2014-03-25 | 2014-07-16 | 京东方科技集团股份有限公司 | Display method |
| CN104915922A (en) * | 2015-06-25 | 2015-09-16 | 西安诺瓦电子科技有限公司 | Splicing bright-dark line correction method |
| CN105185315A (en) * | 2015-10-13 | 2015-12-23 | 西安诺瓦电子科技有限公司 | Uniformity correction method for LED display screen |
| CN105185314A (en) * | 2015-10-13 | 2015-12-23 | 西安诺瓦电子科技有限公司 | Uniformity compensation method for LED display screen |
| CN105551428A (en) * | 2016-02-03 | 2016-05-04 | 西安诺瓦电子科技有限公司 | LED display screen alternative point collection correction method |
| JP2019186736A (en) * | 2018-04-10 | 2019-10-24 | キヤノン株式会社 | Image projection device |
| CN208903644U (en) * | 2018-10-19 | 2019-05-24 | 南京聚诚电子科技有限公司 | A kind of brightness point by point coloration acquisition correction device for small space distance LED display screen |
| CN112071246A (en) * | 2020-09-21 | 2020-12-11 | 云谷（固安）科技有限公司 | Method and device for positioning sub-pixels of display panel and compensation method of display panel |
| KR20230087373A (en) * | 2021-12-09 | 2023-06-16 | 엘지디스플레이 주식회사 | Method Of Compensating Display Device |
| CN115050309A (en) * | 2022-02-24 | 2022-09-13 | 长春希达电子技术有限公司 | LED display screen correction method for adjusting configuration file during collection |
| CN114495803A (en) * | 2022-03-31 | 2022-05-13 | 武汉精立电子技术有限公司 | Mura repairing method of display panel |
Non-Patent Citations (3)
| Title |
|---|
| CCD相机全彩LED显示屏亮度检测与校正算法;严利民;潘浩;杜斌;夏明治;;华中科技大学学报(自然科学版)(第08期);第75-79页 * |
| LED显示屏灯点逐点校正系统;唐德荣;杨红雨;;计算机技术与发展(第12期);第138-142页 * |
| 全彩LED显示屏亮度检测与校正算法;严利民;潘浩;杜斌;;光学技术(第02期);第158-165页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116913197A (en) | 2023-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110134353B (en) | Color compensation method, compensation device and display device | |
| CN110085166B (en) | Bright spot compensation method and device for curved screen | |
| CN108492776B (en) | Intelligent external optical compensation method for AMOLED screen brightness unevenness | |
| CN105185314B (en) | LED display uniformity compensation method | |
| CN113191988B (en) | Brightness correction method and device, storage medium and electronic device | |
| CN116913197B (en) | Pixel positioning method, pixel brightness extraction method and system | |
| CN114495803A (en) | Mura repairing method of display panel | |
| CN113724652A (en) | Compensation method and device for Mura of OLED display panel and readable medium | |
| CN108418997B (en) | Method for removing image moire | |
| CN110853105B (en) | Method and device for simultaneously positioning RGB (red, green and blue) sub-pixels and application | |
| CN113160090B (en) | Image correction method and device | |
| JPWO2018235372A1 (en) | Image display device | |
| CN113920917A (en) | Display panel compensation method and compensation device | |
| CN112419950B (en) | Display difference judging method and system of display device and display device | |
| CN115019723A (en) | Screen display method, screen display device, electronic apparatus, program, and medium | |
| CN113870768B (en) | Display compensation method and device | |
| CN106971375B (en) | Image amplification processing method and device | |
| CN113963658A (en) | Brightness compensation method, brightness data determination method, device and chip | |
| CN114283736B (en) | Method, device and equipment for correcting positioning coordinates of sub-pixels and readable storage medium | |
| CN113821185B (en) | Method and device for determining display difference of display panel | |
| CN117475885A (en) | Correction coefficient determination method, apparatus, device, storage medium, and program product | |
| CN112489115B (en) | Light-emitting module positioning method, device, electronic equipment, storage medium and system | |
| CN117392937B (en) | Color point separation and demura method, device and system for white picture of display panel | |
| CN118135936B (en) | Display screen correction method and device, storage medium and electronic equipment | |
| CN115966175B (en) | Display screen brightness compensation method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |