Background
Image scaling refers to the process of resizing a digital image. By adjusting the size of the image, the method can be suitable for different scenes or application occasions, and a better visual effect is obtained. The image resizing is a non-smooth process, and it is the object of the image scaling algorithm to ensure that the image still maintains a good image quality after being enlarged or reduced.
The digital image is enlarged and reduced by adopting an interpolation mode commonly in the existing image processing software. The essence of the method is that a continuous mathematical model is constructed by taking image pixels as sampling points, and the pixels of the amplified dot matrix image are reasonably filled (namely, curved surface interpolation). Some commonly used interpolation algorithms include: bilinear, bicubic, and lanczos.
A typical algorithm uses two one-dimensional curve interpolations in the horizontal and vertical directions. Fig. 1 and 2 are plots for bicubic and lanczos algorithms, respectively. The use of the curves shown in fig. 1 and 2 can bring a sharpening effect to the edges of the image during the image scaling process, and maintain the quality of the image.
In the process of implementing the invention, the inventor finds that the following problems exist in the related art: on the one hand, when the curves with larger curvature shown in fig. 1 and 2 are used, if the gradient angle of the image edge is smaller, the image edge may have a severe aliasing effect during the image scaling process. On the other hand, if a curve with a smaller curvature is simply used, the sharpness of the edge of other images is reduced, and the image quality of the zoomed image cannot be guaranteed.
Disclosure of Invention
The embodiment of the invention provides an image scaling method, an image scaling device and electronic equipment, which are used for solving the problem that the conventional image scaling algorithm easily causes an obvious sawtooth effect when the angle of an image edge is small.
To solve the above technical problem, an embodiment of the present invention provides an image scaling method. The method comprises the following steps: calculating attributes of an image processing window, wherein the attributes comprise gradient and angle of the image processing window, and the image processing window is an image area with a preset number of pixels; determining at least one scaling filter curve having a curvature corresponding to the attribute; in the image processing window, the scaling filter curve is used.
Optionally, the calculating the attribute of the image processing window specifically includes: calculating the horizontal gradient and the vertical gradient of the image processing window through an edge detection operator; and calculating the gradient and the angle according to the horizontal gradient and the vertical gradient.
Optionally, the gradient is calculated by the following equation: g ═ GH | + | GV |; the angle is calculated by the following equation:
wherein G is gradient, Arg is angle, GH is horizontal gradient, and GV is vertical gradient.
Optionally, the method further comprises: when the gradient of the image processing window is larger than a preset threshold value; and determining the image processing window as an image edge.
Optionally, the determining at least one scaling filter curve specifically includes: when the image processing window is an image edge, selecting a scaling filter curve corresponding to the image edge according to a preset judgment condition; the scaling filter curve has a curvature adapted to an angle of an image edge; and when the image processing window is not the edge of the image, selecting a preset scaling filter curve with an initial curvature.
In order to solve the above technical problem, an embodiment of the present invention further provides an image scaling apparatus. The device comprises: the attribute calculation module is used for calculating the attribute of an image processing window, wherein the attribute comprises the gradient and the angle of the image processing window, and the image processing window is an image area with a preset number of pixels; a scaling filter curve selection module to determine at least one scaling filter curve having a curvature corresponding to the attribute; a scaling module for using the scaled filter curve in the image processing window.
Optionally, the attribute calculating module is specifically configured to: calculating the horizontal gradient and the vertical gradient of the image processing window through an edge detection operator; and calculating the gradient and the angle according to the horizontal gradient and the vertical gradient.
Optionally, the gradient is calculated by the gradient by the following equation: g ═ GH | + | GV |; the angle is calculated by the following equation:
wherein G is gradient, Arg is angle, GH is horizontal gradient, and GV is vertical gradient.
Optionally, the apparatus further comprises: an edge detection module; the edge detection module is used for detecting the gradient of the image processing window when the gradient of the image processing window is larger than a preset threshold value; and determining the image processing window as an image edge.
Optionally, the scaling filter curve module is specifically configured to: when the image processing window is an image edge, selecting a scaling filter curve corresponding to the image edge according to a preset judgment condition; the scaling filter curve has a curvature adapted to an angle of an image edge; and when the image processing window is not the edge of the image, selecting a preset scaling filter curve with an initial curvature.
In order to solve the above technical problem, an embodiment of the present invention further provides at least one processor, and a memory communicatively connected to the at least one processor; wherein the memory stores a program of instructions executable by the at least one processor to enable the at least one processor to perform the image scaling method as described above.
The image scaling method and the device thereof provided by the embodiment of the invention can adaptively adjust the filter curve according to the gradient angle of the image and use the filter curve with proper curvature to perform image interpolation, thereby avoiding the obvious sawtooth effect on the image edge caused by overlarge curvature, ensuring the sharpness of the image edge and obtaining the balance between the two.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The aliasing effect is a common artifact in the image scaling process. The interpolation is usually caused by interpolation on different phases in the image amplification process, and the interpolation results have large difference and the transition is not smooth enough.
"image edge" (edge) refers to the most significant portion of the image where the local intensity changes, which means that the image intensity has significant differences on both sides of the discontinuity. Which mainly exists between different objects, objects and background, different areas or different colors of the image. In general, a step change function or a line change function may be used to represent the image edges. Image edges can be described by both angular and gradient properties.
In general, when image scaling is performed using the filter curves shown in fig. 1 and 2, the edge of the image is sharpened, and an image scaling effect that meets the use requirements can be obtained. But the curvature of the scaled filter curve is high. If the included angle between the horizontal gradient and the vertical gradient in the image is small, the interpolation result is greatly different by using the curve, so that the image edge has a remarkable sawtooth effect.
Of course, the use of some curves with lower curvature can alleviate the aliasing effect of the image edge to some extent, but such interpolation will result in the reduced definition of the zoomed image, and the use of the zoomed image is not favored due to the blurring of more image edges.
Fig. 3 is an operation diagram of typical image scaling. As shown in fig. 3, the image 10 to be scaled is a dot matrix image (since the vector image described by the function has no image display effect problems such as distortion and deformation during scaling, therefore, it is not discussed here), and its size is p × q pixels.
In this embodiment, the image 10 to be zoomed is zoomed by using an n × n template, and a zoomed image 20 is obtained. The n × n template represents a minimum processing unit of the image processing algorithm, the number of pixels considered or involved in the scaling algorithm (neighborhood). In the execution process of the image processing algorithm, the image scaling algorithm is repeatedly executed by sequentially using n × n templates for each pixel point of the image 10 to be scaled, so as to obtain a final scaled image 20.
For simplicity, the following detailed description will be given by taking a BiCubic interpolation as an example: the image 10 to be processed is assumed to be a 256-level gray scale image (i.e., each pixel has a value of 0-255), and the size of the scaled image 20 is 2p × 2 q.
A typical image scaling process is actually a filling process of the pixel matrix of the scaled image 20. During the filling process, the values of the new pixel matrix (i.e. interpolation) are determined based on the values of the pixel points of the image 10 to be scaled.
For a certain point (x, y) in the pixel matrix of the scaled image 20, a 4 × 4 neighborhood point (x, y) may be taken in the vicinity thereofi,yi) And i takes 1,2,3 and 4, and carries out interpolation calculation according to the following formula:
wherein W is a BiCubic function.
The interpolation operation is repeatedly executed to determine the value of each pixel point in the pixel matrix of the zoomed image 20, and the zoomed image 20 can be obtained until the pixel matrix of the zoomed image 20 is filled.
As described above, when the image scaling is performed using the above algorithm, a significant jagging effect occurs at the edges of the image having a small angle. In the practical use process, the image zooming method provided by the embodiment of the invention can be used for obtaining a better anti-aliasing effect on the edge of the image while keeping the definition of the zoomed image.
Fig. 4 is a diagram illustrating an image scaling method according to an embodiment of the present invention. As shown in fig. 4, the method may include the steps of:
100. calculating properties of an image processing window, the properties including a gradient and an angle of the image processing window.
Here, the "image processing window" is used to indicate a neighborhood to be considered when performing an operation by an image scaling algorithm such as the above-described template or processing means having a specific size, and is, for example, an image processing window of 4 × 4 or 3 × 3. Such an image processing window is an image area having a predetermined number of pixels, and has a gradient reflecting a change condition of a pixel value of the image processing window and an angle of the gradient in different directions for one image processing window or pixel area having a specific size.
Here, "gradient" is used to mean an overall gradient that reflects the entire image processing window. "Angle" is used to denote the angle between the gradients in different directions. The gradient and angle of the image processing window are used for describing the attribute of the image edge and measuring whether a certain image processing window belongs to the image edge or the property of the image edge. Which may specifically be calculated by any suitable image edge detection operator.
200. Determining at least one scaling filter curve having a curvature corresponding to the property. The attribute calculated by step 100 may be used as a quantization parameter for scaling the image processing window, and step 200 is a process for adaptively adjusting the curvature of the scaling filter curve according to such quantization parameter.
The scaling filter curve may be any suitable type of filter curve, and is used to calculate values of pixel points and fill a pixel matrix to implement scaling operation of an image, for example, a BiCubic function, a lanczos function, and the like. In other embodiments, the scaling filter curve may be obtained by adjusting the curvature through adjusting parameters based on a function curve conventionally used in the prior art, for example, as shown in fig. 8, the scaling filter curve having a more gradual curvature after being adjusted.
In step 200, the basis for determining the scaling filter curve is the curvature of the curve to obtain a smoother interpolation result for the image processing window properties. Of course, the scaling filter curve to be used may be selected or determined by those skilled in the art according to more screening rules according to the actual application requirement.
300. In the image processing window, the scaling filter curve is used. After determining the filter curve corresponding to the image processing window, the pixel value is calculated for the image processing window using a scaled filter curve of curvature adapted to the attribute.
In the image scaling method provided by the embodiment of the invention, the attributes (gradient and angle) of the image processing window are combined as the judgment standard, and the curvature of the used scaling filter curve is adaptively adjusted, so that the problem of serious sawtooth effect caused by overhigh curvature of the curve under the condition that the included angle of the image processing window is smaller is solved. Moreover, since it is an adaptive process, a higher curvature of the scaled filter curve can still be used in other image portions or suitable image edges, with a sharpening effect to ensure the sharpness of these regions.
The above-mentioned mode of adaptively selecting or adjusting the scaling filter curve according to the attribute of the image processing window can give consideration to the global scaling effect of the image, and avoid the obvious sawtooth effect of the image edge caused by over-emphasized sharpening, so that the quality of the scaled image 20 is significantly improved.
The following describes in detail the image processing window attribute calculation process of the image scaling method provided by the embodiment of the present invention, by taking a common edge detection algorithm, namely sobel (sobel) operator, as an example.
As shown in fig. 5, the image processing window attribute calculation process may include:
510: and calculating a horizontal gradient GH in the horizontal direction and a vertical gradient GV in the vertical direction of the image processing window by using a sobel algorithm template.
For a certain pixel point (x, y), the horizontal gradient GH (x, y) and the vertical gradient GV (x, y) are respectively:
GV(x,y)=(I(x-1,y+1)+2×I(x,y+1)+I(x+1,y+1))-(I(x-1,y-1)+2×I(x,y-1)+I(x+1,y-1));
GH(x,y)=(I(x+1,y-1)+2×I(x+1,y)+I(x+1,y+1))-(I(x-1,y-1)+2×I(x-1,y)+I(x-1,y+1))。
520: from the horizontal gradient GH and the vertical gradient GV, the total gradient strength G and the angle Arg are calculated, respectively. Alternatively, it can be specifically calculated by the following equation: g ═ GH | + | GV |;
it will be appreciated by those skilled in the art that in other embodiments, the gradient or angle of the image processing window may also be determined by other suitable operation methods, such as by the following equation:
to calculate and determine the total gradient strength G. In the calculation process, the goal is to determine quantifiable properties such as gradients and angles to describe the image processing window.
With continued reference to fig. 5, after determining the gradient and angle of the image processing window, the filtering scaling curve to be used may be specifically determined as follows.
530: and judging whether the gradient strength G is greater than a preset threshold value. If yes, go to step 540, otherwise go to step 531.
531: the scaling filter curve default with the initial curvature is selected for use.
540: and judging whether the angle Arg is smaller than a preset first angle, if so, executing step 541, and if not, executing step 550.
541: the scaling filter curve1 with the first curvature is selected for use.
550: and judging whether the angle Arg is smaller than a preset second angle, if so, executing a step 551, and if not, executing a step 560.
551: the scaling filter curve2 with the second curvature is selected for use.
560: it is determined whether the angle Arg is smaller than a preset third angle, if so, execute step 561.
In the present embodiment, the angle Arg obtained by the calculation may be an acute angle or an obtuse angle (depending on the directional relationship between the horizontal gradient and the vertical gradient). Therefore, in the process of making the determination, a determination condition adapted to the obtuse included angle needs to be added. That is, for each angle Arg determination step (e.g., steps 440, 450, and 460), it should include the following two determination conditions:
1): whether the absolute value of the angle Arg is smaller than a preset angle argth
2): whether the absolute value of the difference between the angle Arg and pi/2 (i.e. corresponding to an arc of 90 °) is smaller than a preset angle argth.
In the relationship of "or" between the two determination conditions, the angle Arg may be considered to be smaller than a preset angle (e.g., a preset first angle, a preset second angle, or a preset third angle) when any one of the two determination conditions is satisfied.
561: the scaling filter curve3 with the third curvature is selected for use.
……
In this embodiment, in step 510, whether the gradient strength of the image processing window belongs to the image edge is measured according to a preset threshold (similar to the idea of the sobel algorithm). If the image edge is not included, it is generally considered that the initially set scaling filter curve can be directly used without adjusting the scaling filter curve.
After the image processing window is determined to belong to the image edge, the subsequent curvature adaptive adjustment process is further started. The subsequent selection step belongs to the process of selecting the optimal curvature, and is completed in a mode of dividing the angle into a plurality of intervals and enabling each interval to correspond to one zooming filter curve. After the step 560, more or less intervals may be set according to actual needs, and corresponding determination steps may be set. In some embodiments, if the angle Arg is greater than a certain angle, a scaling filter curve with an initial curvature may also be selected. That is, when the angle Arg is judged no in all the angle judgment steps, step 531 is executed.
Alternatively, the above-mentioned optimization goal (determining the optimal curvature) can be achieved by using other different judgment conditions or judgment strategies.
In the embodiment of the present invention, the number of the regions to be divided and the curvature of the scaling filter curve corresponding to each region may be determined after debugging or experiments according to the needs of the actual situation.
Alternatively, the scaling filter curve may be obtained by adjusting a parameter based on the same function curve, for example, for bicubic function curve W (x), a function curve W (x ') of a desired curvature may be obtained by simply making x' ═ mx and adjusting m.
The image scaling process provided by the above method embodiments may be implemented on any suitable electronic computing platform with logical operation capability. Fig. 6 is an electronic device capable of executing the image scaling process according to an embodiment of the present invention.
As shown in fig. 6, the electronic apparatus includes: one or more processors 610, and a memory 620, with one processor 610 being an example in fig. 6. In some embodiments, it may further comprise: an input device and an output device.
The processor, memory, input devices, and output devices may be connected by a bus or other means, as exemplified by the bus connection in fig. 6.
Memory 620, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The processor 610 performs the image scaling method by executing non-volatile software programs, instructions, and modules stored in the memory 620.
The memory 620 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the image scaling method, and the like. Further, the memory 620 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 620 optionally includes memory located remotely from processor 610, examples of which include, but are not limited to, the internet, an intranet, a local area network, a mobile communications network, and combinations thereof.
The input device may receive input numeric or character information and generate key signal inputs related to user settings and function control of the image scaling method. The output device may include a display device such as a display screen. The one or more modules are stored in the memory 620 and, when executed by the one or more processors 610, perform the image scaling method of any of the method embodiments described above.
The embodiment of the invention further provides an image zooming device. Fig. 7 is a functional block diagram of an image scaling apparatus according to an embodiment of the present invention. The image scaling apparatus has a plurality of program functional modules, which can be stored in the memory of the electronic device shown in fig. 6 for being called by the processor to execute the image scaling method disclosed in the above method embodiments.
As shown in fig. 7, the image scaling means may include: an attribute calculation module 100, a scaling filter curve selection module 200, and a scaling module 300.
Wherein, the attribute calculating module 100 is used for calculating the attribute of the image processing window. The properties include a gradient and an angle of the image processing window. The scaling filter curve selection module 200 is configured to determine at least one scaling filter curve having a curvature corresponding to the property. The scaling module 300 is configured to use the scaling filter curve in the image processing window.
Optionally, with continued reference to fig. 7, the image scaling apparatus may further include an edge detection module 400. The edge detection module 400 is configured to determine that the image processing window is an image edge when the gradient of the image processing window is greater than a preset threshold.
In another embodiment, the attribute calculating module 100, the scaling filter curve selecting module 200 and the scaling module 300 may be further configured to implement one or more steps of the image scaling method disclosed in the above method embodiment.
For example, the attribute calculation module 100 may be specifically configured to: calculating the horizontal gradient and the vertical gradient of the image processing window through an edge detection operator; and calculating the gradient and the angle of the image processing window according to the horizontal gradient and the vertical gradient.
The scaled filter curve module 200 may be specifically configured to: selecting a zooming filter curve corresponding to the image edge according to a preset judgment condition; the scaling filter curve has a curvature that is adapted to the angle of the image edge.
It should be noted that the apparatus embodiment and the method embodiment are based on the same inventive concept. The corresponding contents of the method embodiments therefore also apply to the apparatus embodiments and are not described in detail here.
It will be further appreciated by those of skill in the art that the exemplary image scaling methods described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both, and that the components and steps of the various examples have been described generally in terms of their functionality in the foregoing description for clarity of illustration of interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation.
Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. The computer software may be stored in a computer readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory or a random access memory.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes performed by the present specification and drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.