CN114493987B - Image processing method and device and video processing equipment - Google Patents
Image processing method and device and video processing equipment Download PDFInfo
- Publication number
- CN114493987B CN114493987B CN202011175605.6A CN202011175605A CN114493987B CN 114493987 B CN114493987 B CN 114493987B CN 202011175605 A CN202011175605 A CN 202011175605A CN 114493987 B CN114493987 B CN 114493987B
- Authority
- CN
- China
- Prior art keywords
- color space
- image processing
- conversion
- matrix
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 174
- 238000003672 processing method Methods 0.000 title claims abstract description 24
- 238000006243 chemical reaction Methods 0.000 claims abstract description 238
- 239000011159 matrix material Substances 0.000 claims abstract description 208
- 238000000034 method Methods 0.000 claims abstract description 29
- 230000008569 process Effects 0.000 claims abstract description 24
- 238000013507 mapping Methods 0.000 claims abstract description 14
- 230000001939 inductive effect Effects 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 24
- 238000010586 diagram Methods 0.000 description 10
- 238000004364 calculation method Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000000638 solvent extraction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000006698 induction Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention discloses an image processing method and device and video processing equipment. The method comprises the steps of obtaining a bias compensation matrix according to a color space value range type and a color space type of an initial image and a color space value range type and a color space type of a target image, obtaining an interval scaling matrix according to the color space value range type of the initial image and the color space value range type of the target image, obtaining a target conversion matrix according to the color space type of the initial image, the color space type of the target image and linear image processing parameters, providing a bias adding matrix, obtaining a conversion coefficient by combining operation results of the bias compensation matrix, the interval scaling matrix and the target conversion matrix, and performing linear mapping processing on the initial image based on the conversion coefficient to obtain the target image. The embodiment of the invention modularizes the color space conversion process by inducing the color space conversion process, thereby saving resources and improving the image processing precision.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, and a video processing device.
Background
In the case of video processing devices, CSC (Color Space Convert, color space conversion) and linear image processing are both their necessary functions, whereas linear image processing functions mostly require support of the color space conversion function, since they have to be performed on a specific color space.
CSC is the conversion of a representation of one color in a certain color space to another color space, which in the field of video signal processing mainly converts the video data source signal to match the color space of the display device, thus totally involves the conversion of four common color spaces in the field of broadcast video, RGB Limited, YUV Limited, RGB Full and YUV Full, respectively. The RGB represents a color space based on three channels of red (R), green (G) and blue (B), YUV represents a color space based on three channels of brightness (Y) and color cast (U, V), limited is LIMITED RANGE, which is a color space value Range, under 8-bit color representation, the value Range of R, G, B, Y channels is 16-235, the value Range of U and V is 16-240, so that the color space can be called as a channel Limited gray scale value Range, full is Full Range, which is another color space value Range, and under 8-bit color representation, the value Range of all channels is 0-255, so that the color space can be called as a channel Full gray scale value Range. The four color spaces may be converted to each other by a linear mapping relationship, for example, the conversion relationship of RGB Full and YUV Full is as follows:
The four color spaces can be mutually converted by a method similar to the method, and each conversion mode can be composed of a3×3 conversion matrix and a3×1 bias matrix.
The linear image processing module is also composed of a conversion matrix and a bias matrix according to its different functions, but the bias matrix is not necessary. For example, the brightness adjustment on the RGB Full space can be expressed as:
where L represents the proportion of brightness adjustment, no participation of the bias matrix is required.
Whereas brightness adjustment in RGB Limited space can be expressed as:
Which requires a bias matrix to participate.
The existing video processing device performs the process of color space conversion and linear image processing by concatenating these individual modules together by specific rules to achieve the desired functionality. For the single CSC, the corresponding color conversion module is directly used, if image processing is needed, the single CSC is firstly converted into a color space needed by the image processing to be subjected to the image processing and then converted into an output color space. In short, the prior art solution simply superimposes CSC and linear image processing functions, and does not consider building a specific architecture to integrate the functions, so that problems of precision reduction and waste of computing resources are easily caused.
Disclosure of Invention
Accordingly, to overcome at least some of the drawbacks and disadvantages of the prior art, embodiments of the present invention provide an image processing method, an image processing apparatus, and a video processing device.
The image processing method comprises the steps of obtaining a bias compensation matrix according to a color space value range type and a color space type of an initial image and a color space value range type and a color space type of a target image, obtaining an interval scaling matrix according to the color space value range type of the initial image and the color space value range type of the target image, obtaining a target conversion matrix according to the color space type of the initial image, the color space type of the target image and linear image processing parameters, providing a bias addition matrix, obtaining a conversion coefficient by combining operation results of the bias compensation matrix, the interval scaling matrix and the target conversion matrix, and obtaining the target image by performing linear mapping processing on the initial image based on the conversion coefficient.
The image processing method of the embodiment of the invention modularizes the color space conversion process by inducing the color space conversion process, thereby sixteen CSC conversion relations in the prior art can be realized by the modularized steps, saving storage resources, dividing the color space where the linear image processing function is positioned into a plurality of classes based on the color space where the linear image processing function is positioned, reserving the adding positions of the multi-class image processing submodules in the color space conversion architecture, enabling the color space conversion architecture to support the linear image processing function, and simplifying the middle color space conversion process into the color space type conversion through the digital signal and linear unbiased data conversion module (offset compensation and offset addition) and the color space value range type conversion module (interval scaling) in the architecture, thereby reducing the calculation amount on a link and improving the precision.
In one embodiment of the invention, the obtaining the target conversion matrix according to the color space type of the initial image, the color space type of the target image and the linear image processing parameters comprises obtaining a first sub-conversion matrix according to the color space type of the initial image and a first color space type, obtaining a second sub-conversion matrix according to a first linear image processing parameter of the linear image processing parameters, which is used for carrying out image processing in a color space corresponding to the first color space type, obtaining a third sub-conversion matrix according to the first color space type and the second color space type, obtaining a fourth sub-conversion matrix according to a second linear image processing parameter of the linear image processing parameters, which is used for carrying out image processing in a color space corresponding to the second color space type, obtaining a fifth sub-conversion matrix according to the second color space type and the color space type of the target image, and sequentially carrying out operation matrix operation on the first sub-conversion matrix, the second sub-conversion matrix, the third sub-conversion matrix, the fourth sub-conversion matrix and the fifth sub-conversion matrix according to the sequence of the sub-conversion matrix.
In one embodiment of the invention, the obtaining the target conversion matrix according to the color space type of the initial image, the color space type of the target image and the linear image processing parameter comprises obtaining a first sub-conversion matrix according to the color space type of the initial image and the color space type required by the linear image processing parameter, obtaining a second sub-conversion matrix according to the linear image processing parameter, obtaining a third sub-conversion matrix according to the color space type required by the linear image processing parameter and the color space type of the target image, and sequentially performing matrix operation on the first sub-conversion matrix, the second sub-conversion matrix and the third sub-conversion matrix according to the listed sequence to obtain the target conversion matrix.
In one embodiment of the present invention, the target conversion matrix and the interval scaling matrix perform matrix operation prior to the offset adding matrix and the offset compensating matrix, and the interval scaling matrix performs matrix operation prior to or after the target conversion matrix and the offset compensating matrix.
In one embodiment of the present invention, the first color space type is RGB, the first linear image processing parameter is selected from a luminance adjustment parameter, a saturation adjustment parameter, and a color temperature adjustment parameter, the second color space type is YUV, and the second linear image processing parameter is selected from a contrast adjustment parameter and a color temperature adjustment parameter, or the first color space type is YUV, the first linear image processing parameter is selected from a contrast adjustment parameter and a color temperature adjustment parameter, the second color space type is RGB, and the second linear image processing parameter is selected from a luminance adjustment parameter, a saturation adjustment parameter, and a color temperature adjustment parameter.
In addition, another embodiment of the invention provides an image processing method, for example, comprising obtaining a bias compensation matrix according to a color space value range type and a color space type of an initial image and a color space value range type and a color space type of a target image, obtaining a section scaling matrix according to the color space value range type of the initial image and the color space value range type of the target image, obtaining a target conversion matrix according to the color space type of the initial image and the color space type of the target image, providing a bias addition matrix and obtaining a conversion coefficient by combining operation results of the bias compensation matrix, the section scaling matrix and the target conversion matrix, and performing linear mapping processing on the initial image based on the conversion coefficient to obtain the target image.
The image processing method of the embodiment of the invention modularizes the color space conversion process by inducing the color space conversion process, thereby sixteen CSC conversion relations in the prior art can be realized by the modularized steps, saving storage resources, and simplifying the middle color space conversion process into color space type conversion by a module (offset compensation and offset addition) for converting digital signals and linear unbiased data and a color space value range type conversion module (interval scaling) in the architecture, reducing the calculated amount on a link and improving the precision.
Furthermore, the image processing device provided by the embodiment of the invention comprises a bias compensation module, an interval scaling module, a space type conversion and image processing module, a bias adding module and an image processing module, wherein the bias compensation module is used for obtaining a bias compensation matrix according to a color space value range type and a color space type of an initial image and the color space value range type and the color space type of a target image, the interval scaling module is used for obtaining an interval scaling matrix according to the color space value range type of the initial image and the color space value range type of the target image, the space type conversion and image processing module is used for obtaining a target conversion matrix according to the color space type of the initial image, the color space type of the target image and linear image processing parameters, the bias adding module is used for providing a bias adding matrix and combining operation results of the bias compensation matrix, the interval scaling matrix and the target conversion matrix to obtain a conversion coefficient, and the image processing module is used for carrying out linear mapping processing on the initial image based on the conversion coefficient to obtain the target image.
The image processing device of the embodiment of the invention divides the color space conversion process into a plurality of modules through induction of the color space conversion process, so that sixteen types of CSC conversion relations in the prior art can be realized by the modules, storage resources are saved, furthermore, the linear image processing function is divided into a plurality of types based on the color space where the linear image processing function is positioned, the adding position of the multi-type image processing submodule is reserved in the color space conversion framework, so that the color space conversion framework supports the linear image processing function, and in addition, the middle color space conversion process is simplified into color space type conversion through the modules (bias compensation module and bias adding module) for converting digital signals and linear unbiased data in the framework and the color space value range type conversion module (interval scaling module), so that the calculation amount on a link is reduced, and the precision is improved.
In addition, another embodiment of the invention provides an image processing device, for example, comprising a bias compensation module, a section scaling module, a space type conversion module, a bias adding module and an image processing module, wherein the bias compensation module is used for obtaining a bias compensation matrix according to a color space value range type and a color space type of an initial image and the color space value range type and the color space type of a target image, the section scaling module is used for obtaining a section scaling matrix according to the color space value range type of the initial image and the color space value range type of the target image, the space type conversion module is used for obtaining a target conversion matrix according to the color space type of the initial image and the color space type of the target image, the bias adding module is used for providing a bias adding matrix and combining operation results of the bias compensation matrix, the section scaling matrix and the target conversion matrix to obtain a conversion coefficient, and the image processing module is used for carrying out linear mapping processing on the initial image based on the conversion coefficient to obtain the target image.
The image processing device of the embodiment of the invention divides the color space conversion process into a plurality of modules through the induction of the color space conversion process, thereby sixteen CSC conversion relations in the prior art can be realized by the modules, saving storage resources, and furthermore, the middle color space conversion process is simplified into color space type conversion through the modules (the offset compensation module and the offset adding module) for converting digital signals and linear unbiased data and the color space value range type conversion module (the interval scaling module) in the architecture, thereby reducing the calculated amount on a link and improving the precision.
Finally, the video processing device provided by the embodiment of the invention comprises a processor and a programmable logic device, wherein the processor and the programmable logic device are electrically connected with the processor, the processor and the programmable logic device are used for executing the image processing method according to any of the previous embodiments together, and the processor is specifically used for generating the conversion coefficient and transmitting the conversion coefficient to the programmable logic device so that the programmable logic device can perform the linear mapping processing on the initial image based on the conversion coefficient to obtain the target image.
In one embodiment of the present invention, each of the color space range type of the initial image and the color space range type of the target image is RGB or YUV, and each of the color space range type of the initial image and the color space range type of the target image is a channel Limited gray scale range (Limited) or a channel Full gray scale range (Full).
The technical scheme of each embodiment of the invention has one or more advantages that the embodiment of the invention modularizes the color space conversion process, so sixteen CSC conversion relations in the prior art can be realized by the modules, storage resources are saved, furthermore, the linear image processing function is divided into a plurality of classes based on the color space where the linear image processing function is positioned, the adding position of the multi-class image processing submodule is reserved in the color space conversion framework, so that the color space conversion framework supports the linear image processing function to realize, and in addition, the middle color space conversion process is simplified into color space type conversion through the module for converting digital signals and linear unbiased data in the framework and the color space value range type conversion module, so that the calculation amount on a link is reduced, and the precision is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of steps of an image processing method according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a video processing apparatus according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a conversion relationship between RGB/YUV and Limited/Full according to an embodiment of the invention.
Fig. 4 is a schematic diagram showing a specific block configuration of the video processing apparatus shown in fig. 2.
Fig. 5 is a schematic structural diagram of another video processing apparatus according to an embodiment of the present invention.
Fig. 6 is a schematic diagram showing a specific block configuration of the video processing apparatus shown in fig. 5.
Fig. 7 is a flowchart illustrating another image processing method according to an embodiment of the present invention.
FIG. 8 is a schematic diagram of a sub-module of the spatial type conversion and image processing module shown in FIG. 6.
Fig. 9 is a schematic diagram showing another specific block configuration of the video processing apparatus shown in fig. 5.
FIG. 10 is a schematic diagram of another sub-module configuration of the spatial type conversion and image processing module shown in FIG. 6.
FIG. 11 is a schematic diagram of another sub-module configuration of the spatial type conversion and image processing module shown in FIG. 6.
FIG. 12 is a schematic diagram of another sub-module configuration of the spatial type conversion and image processing module of FIG. 6.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The embodiment of the invention designs a CSC architecture which can integrate color space conversion and linear image processing functions, can realize the mutual conversion function of the common color spaces in the four broadcast video fields, and can support the addition of linear image processing sub-modules on the RGB color space and the YUV color space so as to solve the technical problems of low integration level, resource waste and poor precision caused by overlong links in the prior art.
Referring to fig. 1, an image processing method provided in an embodiment of the present invention includes, for example, the following steps S11, S13, S15, S17, and S19.
S11, obtaining a bias compensation matrix according to the color space value range type and the color space type of the initial image and the color space value range type and the color space type of the target image;
S13, obtaining an interval scaling matrix according to the color space value range type of the initial image and the color space value range type of the target image;
S15, obtaining a target conversion matrix according to the color space type of the initial image and the color space type of the target image;
S17, providing a bias adding matrix and combining the operation results of the bias compensation matrix, the interval scaling matrix and the target conversion matrix to obtain conversion coefficients;
And S19, carrying out linear mapping processing on the initial image based on the conversion coefficient to obtain the target image.
In order to more clearly understand the image processing method of the present embodiment, the following will exemplify with reference to fig. 2,3 and 4.
Specifically, as shown in fig. 2, the video processing apparatus 100 includes a processor 101 and a programmable logic device 103 electrically connected to the processor 101, where the processor 101 is, for example, an ARM-based embedded processor, and the programmable logic device 103 is, for example, an FPGA (Field Programmable GATE ARRAY ) device. The processor 101 performs color space conversion according to the input color space type and color space value range type of the initial image and the color space type and color space value range type of the target image to generate conversion coefficients (corresponding to steps S11, S13, S15 and S17), and transmits the conversion coefficients to the programmable logic device 103, so that the programmable logic device 103 performs linear mapping processing (here, pure color space conversion) on the initial image to obtain the target image (corresponding to step S19). The color space of the initial image is, for example, RGB Limited, RGB Full, YUV Limited or YUV Full, and the color space of the target image is, for example, RGB Limited, RGB Full, YUV Limited or YUV Full, whereas in this example, each color space is divided into two physical quantities, one is, for example, RGB and YUV, and the other is, for example, limited and Full, and accordingly, the combination of the color space type and the color space value range type of the initial image input to the processor 101 and the color space type and the color space value range type of the target image may be referred to as a color space conversion mode, so that the processor 101 generates the conversion coefficient according to the input color space conversion mode and transmits the conversion coefficient to the programmable logic device 103.
As shown in fig. 3, the color space conversion is split into two modules, namely a conversion module between the color space types RGB/YUV and a conversion module between the color space value range types Limited/Full, through which the original sixteen conversion relations are split into two four conversion relations. The conversion matrix between RGB/YUV is a 3×3 coefficient matrix, the conversion matrix between Limited/Full is a 3×1 bias matrix and a 3×3 coefficient matrix, and the 3×3 coefficient matrix is used for scaling the three channels respectively, thus being a diagonal matrix. It can be seen from the characteristics of the conversion matrix between Limited/Full that the roles of the 3×3 coefficient matrix and the 3×1 bias matrix can be completely separated, so that the conversion between Limited/Full can be split into two sub-modules, namely, the bias compensation module 1011 for providing a 3×1 bias matrix and the interval scaling module 1013 for providing a 3×3 diagonal matrix. In this way, the interval scaling module 1013 may perform step S13 to obtain an interval scaling matrix, such as a 3×3 diagonal matrix, according to the color space value range type of the initial image and the color space value range type of the target image.
In addition, for the YUV color space type, there is a 128 bias of the digital signal on the U, V channels, so that compensation needs to be performed by using a 3×1 bias matrix [0,128,128] T, where the bias matrix [0,128,128] T can be linearly superimposed with the 3×1 bias matrix in the Limited/Full conversion matrix, and thus is also provided by the bias compensation module 1011, so that the bias compensation module 1011 can perform step S11 to obtain a bias compensation matrix according to the color space value range type and the color space type of the initial image and the color space value range type and the color space type of the target image, so as to obtain a linear superimposed result matrix of the foregoing two 3×1 bias matrices.
As for the spatial type conversion module 1015 and the offset addition module 1017 in fig. 4, they may perform steps S15 and S17, respectively. More specifically, referring to fig. 1 and 4 together, the offset compensation module 1011, the interval scaling module 1013, the spatial type conversion module 1015 and the offset adding matrix are respectively configured to provide the offset compensation matrix, the interval scaling matrix, the target conversion matrix and the offset adding matrix, and in addition, the offset compensation module 1011 provides the offset compensation matrix to the interval scaling module 1013, the spatial type conversion module 1015 is further configured to perform matrix operation on the offset compensation matrix and the interval scaling matrix to obtain a first operation result, and the spatial type conversion module 1015 is further configured to perform matrix operation on the target conversion matrix and the first operation result to obtain a second operation result, and the offset adding module 1017 is further configured to perform matrix operation on the offset adding matrix and the second operation result to obtain the conversion coefficient. The example adds a bias at the end 1017, similar to the original bias compensation 1011, to ensure that the output data is a correct digital signal.
In view of the foregoing, the linear image processing function is generally required to be performed based on linear unbiased data, which means that the data can reasonably and accurately reflect the physical quantity that it represents, while the nature of offset compensation is to convert digital signals into linear unbiased data. The offset compensation module 1011 and the offset adding module 1017 of fig. 4 for providing an offset compensation matrix and an offset adding matrix, respectively, are disposed at two ends of the flow, that is, to ensure that the data of the middle flow is linear unbiased data, so that frequent offset compensation and adding during the color space conversion process before image processing can be avoided.
The section scaling module 1013 provides a diagonal matrix, and since the nature of the matrix is to implement the section size conversion between the color space range types Limited/Full, there are usually only two options, i.e. Limited to Full or Full to Limited, and the two diagonal matrices are inverse matrices. Due to the nature of the diagonal matrix, it can be swapped with other coefficient matrices at will. If there are multiple interval scaling in the whole link, the modules cancel each other out, and finally there are only two cases, a) only one non-cancelled interval scaling module remains, b) all interval scaling modules cancel. Therefore, the color space conversion process before the linear image processing does not need to consider the interval scaling module, and the interval scaling module only needs to perform interval scaling from the initial image color space to the target image color space once in the whole link. In this way, only the color space type conversion submodule needs to be added before the linear image processing submodule.
As shown in fig. 5, this is a case where the processor 101 of the video processing apparatus 100 inputs linear image processing parameters in addition to the color space conversion mode. In fig. 5, the processor 101 performs color space conversion and linear image processing to generate a conversion coefficient according to the input color space type and color space value range type of the initial image, the color space type and color space value range type of the target image, and the linear image processing parameter, and transmits the conversion coefficient to the programmable logic device 103, so that the programmable logic device 103 performs linear mapping (here, color space conversion and linear image processing) on the initial image to obtain the target image. Accordingly, as shown in fig. 6, the processor 101 includes a bias compensation module 1011, an interval scaling module 1013, a spatial type conversion and image processing module 1016, and a bias addition module 1017 connected in this order, and the programmable logic device 103 includes an image processing module 1031 connected to the bias addition module 1017.
In view of the above, the video processing apparatus 100 shown in fig. 5 may perform the image processing method shown in fig. 7, for example, including the steps of:
S71, obtaining a bias compensation matrix according to the color space value range type and the color space type of the initial image and the color space value range type and the color space type of the target image;
s73, obtaining an interval scaling matrix according to the color space value range type of the initial image and the color space value range type of the target image;
S75, obtaining a target conversion matrix according to the color space type of the initial image, the color space type of the target image and linear image processing parameters;
s77, providing a bias adding matrix and combining the operation results of the bias compensation matrix, the interval scaling matrix and the target conversion matrix to obtain conversion coefficients;
and S79, performing linear mapping processing on the initial image based on the conversion coefficient to obtain the target image.
The aforementioned steps S71, S73, S75 and S77 are for example performed in the processor 101, and the aforementioned step S79 is for example performed in the programmable logic device 103. Specifically, the steps S71, S73, S75, and S77 are performed by the offset compensation module 1011, the section scaling module 1013, the spatial type conversion module 1016, and the offset addition module 1017, respectively, shown in fig. 6, for example, and the step S79 is performed by the image processing module 1031 shown in fig. 6. In addition, as can be seen from fig. 6 and 7, the target conversion matrix and the interval scaling matrix perform matrix operation prior to the offset adding matrix and the offset compensating matrix, and the interval scaling matrix performs matrix operation prior to the target conversion matrix and the offset compensating matrix.
Referring to fig. 6 and 8 together, the spatial type conversion and image processing module 1016 includes, for example, a color space type-to-YUV conversion sub-module 10160, a first image processing sub-module 10162, a YUV-to-RGB conversion sub-module 10164, a second image processing sub-module 10166, and a color space type conversion sub-module 10168 of RGB-to-target image. As can be seen from fig. 6 and 8, after offset compensation and interval scaling, the image processing is performed in YUV space by converting the image processing in YUV space, the first image processing sub-module 10162 represents one or more image processing processes performed in YUV space, such as contrast adjustment, tone adjustment, etc., then converting the image processing in RGB space, the second image processing sub-module 10166 represents one or more image processing processes performed in RGB space, such as brightness adjustment, saturation adjustment, color temperature adjustment, etc., finally converting the image processing in YUV space, and then adding the offset output digital signal. All modules in the architectures shown in fig. 6 and 8 support Bypass, and some sub-modules can be skipped, for example, if no image processing is performed, the two middle color space type conversion sub-modules can be skipped, and the type conversion from the color space of the initial image to the color space of the target image can be directly performed.
More specifically, each sub-module shown in fig. 8 may have, for example, functions of 1) a color space type-to-YUV conversion sub-module 10160 for obtaining a first sub-conversion matrix from the color space type and a first color space type of the initial image, 2) the first image processing sub-module 10162 for obtaining a second sub-conversion matrix from a first linear image processing parameter of the linear image processing parameters that performs image processing in a color space corresponding to the first color space type, (3) the YUV-to-RGB conversion sub-module 10164 for obtaining a third sub-conversion matrix from the first color space type and a second color space type, 4) the second image processing sub-module 10166 for obtaining a fourth sub-conversion matrix from a second linear image processing parameter of the linear image processing parameters that performs image processing in a color space corresponding to the second color space type, and 5) the RGB-to-target image color space type conversion sub-module 10168 for obtaining a fifth sub-conversion matrix from the second color space type and the target image. In addition, the first sub-conversion matrix, the second sub-conversion matrix, the third sub-conversion matrix, the fourth sub-conversion matrix and the fifth sub-conversion matrix are sequentially subjected to matrix operation by corresponding sub-modules according to the sequence of columns to obtain the target conversion matrix.
In summary, the embodiment of the invention divides the color space conversion process into four modules (see four modules in the processor 101 in fig. 4 and 6), so that sixteen CSC conversion relations in the prior art are realized by the four modules, storage resources are saved, furthermore, the linear image processing function is divided into two major types based on the color space RGB/YUV where the linear image processing function is located, the added positions of the two types of image processing submodules are reserved in the color space conversion architecture, so that the color space conversion architecture supports the linear image processing function, and furthermore, the digital signal and linear unbiased data conversion modules (the bias compensation module 1011 and the bias adding module 1017) and the Limited and Full color space value range type conversion module (the interval scaling module 1013) in the architecture simplify the middle color space conversion process into RGB and YUV color space type conversion, reduce the calculation amount on links and improve the image processing precision.
Furthermore, it should be noted that, in the overall architecture flow, the location of the section scaling module 1013 is relatively free, and may be located before the spatial type conversion and image processing module 1016 as shown in fig. 9 and after the spatial type conversion and image processing module 1017 as shown in fig. 6. In this case, the target conversion matrix and the section scaling matrix perform matrix operation prior to the bias addition matrix and the bias compensation matrix, and the section scaling matrix performs matrix operation later than the target conversion matrix and the bias compensation matrix.
Furthermore, in the overall architecture flow, the processing sequence of the first and second image processing sub-modules 10162, 10166 is not strictly defined, so that the architecture flow of the embodiment of the present invention may also be designed as shown in fig. 10, that is, the spatial type conversion and image processing module 1016 includes a color space type-to-RGB conversion sub-module 10161, a second image processing sub-module 10166, a RGB-to-YUV conversion sub-module 10163, a first image processing sub-module 10162, and a color space type conversion sub-module 10165 for YUV-to-target image, which are sequentially connected.
In addition, it should be noted that in other embodiments, the step S75 includes, for example, the substeps of i) obtaining a first sub-conversion matrix according to the color space type of the initial image and the color space type required by the linear image processing parameter, ii) obtaining a second sub-conversion matrix according to the linear image processing parameter, iii) obtaining a third sub-conversion matrix according to the color space type required by the linear image processing parameter and the color space type of the target image, and iv) sequentially performing matrix operations on the first sub-conversion matrix, the second sub-conversion matrix, and the third sub-conversion matrix according to the listed order to obtain the target conversion matrix. Accordingly, the foregoing spatial type conversion and image processing module 1016 may employ the color space type-to-YUV conversion sub-module 10160, the first image processing sub-module 10162, and the color space type-to-target image conversion sub-module 10165 of the initial image shown in fig. 11 and sequentially connected, or employ the color space type-to-RGB conversion sub-module 10161, the second image processing sub-module 10166, and the color space type-to-target image conversion sub-module 10168 of the initial image shown in fig. 12 and sequentially connected. Briefly, in this embodiment, the type of color space required for the linear image processing parameters input to the processor 101 is one of YUV and RGB.
It should be noted that, in other embodiments, the offset compensation module 1011, the section scaling module 1013, the spatial type conversion module 1015, the offset adding module 1017, and the image processing module 1031 shown in fig. 4 are used as software modules to form a specific implementation manner of the image processing apparatus according to the embodiment of the present invention, and the offset compensation module 1011, the section scaling module 1013, the spatial type conversion module 1015, the offset adding module 1017, and the image processing module 1031 are sequentially connected in the order listed, where the spatial type conversion module 1015 and the section scaling module 1013 may be located. Alternatively, the offset compensation module 1011, the section scaling module 1013, the spatial type conversion and image processing module 1016, the offset adding module 1017 and the image processing module 1031 shown in fig. 6 and 9 constitute other specific implementations of the image processing apparatus according to the embodiments of the present invention as software modules.
It should be further understood that the image processing method and the image processing apparatus according to the foregoing embodiments of the present invention are not limited to application to the video processing device 100 including the processor 101 and the programmable logic device 103, but may also be applied to one or more other software and hardware devices having a color space conversion function or a color space conversion and linear image processing function, such as a device configured with one or more of CPU, MCU, FPGA, ASIC, ARM or the like.
Still further, other embodiments of the present invention provide an image processing system that includes, for example, a memory and a processor coupled to the memory. The memory may be, for example, a non-volatile memory, on which the computer program is stored. The processor may be, for example, an embedded processor, which when running the computer program performs the image processing method in the previous embodiment. As for the specific operation and technical effects of the graph processing system in the present embodiment, reference is made to the related description of the foregoing embodiments.
In addition, other embodiments of the present invention provide a computer-readable storage medium. The computer readable storage medium is, for example, a non-volatile memory such as magnetic media (e.g., hard disk, floppy disk, and magnetic tape), optical media (e.g., CDROM disk and DVD), magneto-optical media (e.g., optical disk), and hardware devices specially constructed for storing and performing computer executable instructions (e.g., read Only Memory (ROM), random Access Memory (RAM), flash memory, etc.). The computer-readable storage medium has stored thereon computer-executable instructions. The computer-readable storage medium may execute the computer-executable instructions by one or more processors or processing devices to implement the image processing methods in the foregoing embodiments.
In addition, it should be understood that the foregoing embodiments are merely exemplary illustrations of the present invention, and the technical solutions of the embodiments may be arbitrarily combined and matched without conflict in technical features, contradiction in structure, and departure from the purpose of the present invention.
It should be noted that, in several embodiments provided by the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the partitioning of elements is merely a logical functional partitioning, and there may be additional partitioning in actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not implemented. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in hardware plus software functional units.
The integrated units implemented in the form of software functional units described above may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform part of the steps of the methods according to the embodiments of the present invention. The storage medium includes various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory RAM), a magnetic disk, or an optical disk.
It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention, and not for limiting the same, and although the present invention has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiments of the present invention.
Claims (8)
1. An image processing method, comprising:
obtaining a bias compensation matrix according to the color space value range type and the color space type of the initial image and the color space value range type and the color space type of the target image;
obtaining an interval scaling matrix according to the color space value range type of the initial image and the color space value range type of the target image;
obtaining a target conversion matrix according to the color space type of the initial image, the color space type of the target image and linear image processing parameters;
Providing a bias adding matrix and combining the operation results of the bias compensation matrix, the interval scaling matrix and the target conversion matrix to obtain conversion coefficients;
And carrying out linear mapping processing on the initial image based on the conversion coefficient to obtain the target image.
2. The image processing method according to claim 1, wherein the obtaining a target conversion matrix from the color space type of the initial image, the color space type of the target image, and linear image processing parameters comprises:
obtaining a first sub-conversion matrix according to the color space type and a first color space type of the initial image;
Obtaining a second sub-conversion matrix according to a first linear image processing parameter of the linear image processing parameters, wherein the first linear image processing parameter performs image processing in a color space corresponding to the first color space type;
Obtaining a third sub-conversion matrix according to the first color space type and the second color space type;
Obtaining a fourth sub-conversion matrix according to a second linear image processing parameter of the linear image processing parameters, wherein the second linear image processing parameter is used for performing image processing in a color space corresponding to the second color space type;
Obtaining a fifth sub-conversion matrix according to the second color space type and the color space type of the target image;
and sequentially performing matrix operation on the first sub-conversion matrix, the second sub-conversion matrix, the third sub-conversion matrix, the fourth sub-conversion matrix and the fifth sub-conversion matrix according to the sequence of columns to obtain the target conversion matrix.
3. The image processing method according to claim 1, wherein the obtaining a target conversion matrix from the color space type of the initial image, the color space type of the target image, and linear image processing parameters comprises:
Obtaining a first sub-conversion matrix according to the color space type of the initial image and the color space type required by the linear image processing parameters;
Obtaining a second sub-conversion matrix according to the linear image processing parameters;
obtaining a third sub-conversion matrix according to the color space type required by the linear image processing parameters and the color space type of the target image;
And sequentially performing matrix operation on the first sub-conversion matrix, the second sub-conversion matrix and the third sub-conversion matrix according to the sequence of columns to obtain the target conversion matrix.
4. The image processing method according to claim 1, wherein the target conversion matrix and the section scaling matrix perform matrix operation with the bias compensation matrix prior to the bias addition matrix, and the section scaling matrix performs matrix operation with the bias compensation matrix prior to or after the target conversion matrix.
5. The image processing method of claim 2, wherein the first color space type is RGB, the first linear image processing parameter is selected from a brightness adjustment parameter, a saturation adjustment parameter, and a color temperature adjustment parameter, the second color space type is YUV, and the second linear image processing parameter is selected from a contrast adjustment parameter and a hue adjustment parameter;
Or the first color space type is YUV, the first linear image processing parameter is selected from a contrast adjustment parameter and a tone adjustment parameter, the second color space type is RGB, and the second linear image processing parameter is selected from a brightness adjustment parameter, a saturation adjustment parameter and a color temperature adjustment parameter.
6. An image processing apparatus, comprising:
The offset compensation module is used for obtaining an offset compensation matrix according to the color space value range type and the color space type of the initial image and the color space value range type and the color space type of the target image;
The interval scaling module is used for obtaining an interval scaling matrix according to the color space value range type of the initial image and the color space value range type of the target image;
The space type conversion and image processing module is used for obtaining a target conversion matrix according to the color space type of the initial image, the color space type of the target image and linear image processing parameters;
The offset adding module is used for providing an offset adding matrix and combining the operation results of the offset compensating matrix, the interval scaling matrix and the target conversion matrix to obtain conversion coefficients;
and the image processing module is used for carrying out linear mapping processing on the initial image based on the conversion coefficient to obtain the target image.
7. A video processing apparatus, comprising:
Processor, and
A programmable logic device electrically connected to the processor;
The processor and the programmable logic device are configured to perform the image processing method according to any one of claims 1 to 5, and the processor is specifically configured to generate the conversion coefficient and transmit the conversion coefficient to the programmable logic device, so that the programmable logic device performs the linear mapping process on the initial image based on the conversion coefficient to obtain the target image.
8. The video processing device of claim 7, wherein each of the color space range type of the initial image and the color space range type of the target image is RGB or YUV, and each of the color space range type of the initial image and the color space range type of the target image is a channel limited grayscale range or a channel full grayscale range.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011175605.6A CN114493987B (en) | 2020-10-27 | 2020-10-27 | Image processing method and device and video processing equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011175605.6A CN114493987B (en) | 2020-10-27 | 2020-10-27 | Image processing method and device and video processing equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114493987A CN114493987A (en) | 2022-05-13 |
| CN114493987B true CN114493987B (en) | 2024-12-17 |
Family
ID=81491340
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011175605.6A Active CN114493987B (en) | 2020-10-27 | 2020-10-27 | Image processing method and device and video processing equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114493987B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102280096A (en) * | 2011-07-15 | 2011-12-14 | 杭州米加科技有限公司 | Method for combining image scaling and color space switching |
| CN107025880A (en) * | 2017-04-14 | 2017-08-08 | 西安诺瓦电子科技有限公司 | Image display control method and device and display screen control system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1993293A1 (en) * | 2007-05-15 | 2008-11-19 | Via Technologies, Inc. | System and method of image processing |
| US8638342B2 (en) * | 2009-10-20 | 2014-01-28 | Apple Inc. | System and method for demosaicing image data using weighted gradients |
| US9025867B2 (en) * | 2012-05-31 | 2015-05-05 | Apple Inc. | Systems and methods for YCC image processing |
| US20190356891A1 (en) * | 2018-05-16 | 2019-11-21 | Synaptics Incorporated | High dynamic range (hdr) data conversion and color space mapping |
| CN108933933B (en) * | 2018-07-05 | 2019-08-20 | 华为技术有限公司 | A video signal processing method and device |
-
2020
- 2020-10-27 CN CN202011175605.6A patent/CN114493987B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102280096A (en) * | 2011-07-15 | 2011-12-14 | 杭州米加科技有限公司 | Method for combining image scaling and color space switching |
| CN107025880A (en) * | 2017-04-14 | 2017-08-08 | 西安诺瓦电子科技有限公司 | Image display control method and device and display screen control system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114493987A (en) | 2022-05-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5255083A (en) | Digital color correction system and method | |
| RU2737507C2 (en) | Method and device for encoding an image of a high dynamic range, a corresponding decoding method and a decoding device | |
| US20220103720A1 (en) | Method and device for color gamut mapping | |
| US6828982B2 (en) | Apparatus and method for converting of pixels from YUV format to RGB format using color look-up tables | |
| RU2710873C2 (en) | Method and device for colour image decoding | |
| TW201838411A (en) | Image file processing method, apparatus and system, and storage medium | |
| KR20180021747A (en) | Method and device for encoding both a hdr picture and a sdr picture obtained from said hdr picture using color mapping functions | |
| CN113784175B (en) | A kind of HDR video conversion method, device, equipment and computer storage medium | |
| US11962917B2 (en) | Color adjustment method, color adjustment device, electronic device and computer-readable storage medium | |
| CN101742340A (en) | Image optimization editing method and device | |
| EP3453175B1 (en) | Method and apparatus for encoding/decoding a high dynamic range picture into a coded bistream | |
| EP3586503A1 (en) | Method and device for reconstructing an hdr image | |
| US20230050498A1 (en) | Methods for converting an image and corresponding devices | |
| CN113781321B (en) | Information compensation method, device and equipment for image highlight region and storage medium | |
| US20190080674A1 (en) | Systems and methods for combining video and graphic sources for display | |
| JPH05241550A (en) | Color picture converting method | |
| KR20210020103A (en) | Data processing method and device, electronic device and storage medium | |
| CN114493987B (en) | Image processing method and device and video processing equipment | |
| US20190132600A1 (en) | Method and apparatus for encoding/decoding a scalar integer into a parameter representative of a pivot points of a piece-wise linear function | |
| US6825853B2 (en) | Apparatus and method for color space conversion in video images | |
| US10423587B2 (en) | Systems and methods for rendering graphical assets | |
| CN115460461B (en) | Video processing method and device, terminal equipment and computer readable storage medium | |
| CN113194316B (en) | Image encoding and decoding method and device | |
| US20240144426A1 (en) | Super resolution image generation circuit | |
| US20230144286A1 (en) | Method of Generating Target Image Data, Electrical Device and Non-Transitory Computer Readable Medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |