CN105096261A - Image processing device and image processing method - Google Patents

Abstract

The present invention provides an image processing device and an image processing method, wherein the image processing method includes: performing estimation processing on a received original image to obtain a blur kernel corresponding to the original image; performing iterative processing on the original image according to the blur kernel, In order to obtain a clear image; determine the target area in the clear image, and extract the target image corresponding to the target area; perform perspective correction processing on the target image to obtain the front-view image corresponding to the target image; enhance the front-view image to Get the resulting image. Through this technical solution, users are no longer affected by viewing angles when viewing images or videos, and can see clear, high-contrast, front-facing images without perspective distortion when viewing images or videos from any orientation.

Description

Image processing device and image processing method

technical field

The present invention relates to the field of image technology, in particular, to an image processing device and an image processing method.

Background technique

Due to the limitations of current technical means, when people watch images or videos, they often cannot see clear images or videos, and even can only see images or videos with severe perspective distortion. The reason is that when people observe images or video information, they are often used to observe from the front of the subject object. At this time, the image or video information observed by people is the original state of the subjective object, that is, people will see clear and undistorted images. image or video information; but people often have to view the image or video information from a relatively oblique perspective due to the limited environment, such as a fixed location. At this time, the image or video information people see is often not subjective. Original, but blurry or severely perspective distorted.

Therefore, how to make people not be affected by the viewing orientation, no matter from which orientation they can see clear images or videos without perspective distortion has become an urgent problem to be solved.

Contents of the invention

Based on the above problems, the present invention proposes an image processing device and an image processing method, which can solve the problems of image or video blurring and perspective distortion existing in the prior art, so that users can see Clear, undistorted images.

In view of this, an aspect of the present invention proposes an image processing device, including: an estimation processing unit that performs estimation processing on the received original image to obtain a blur kernel corresponding to the original image; an iterative processing unit connected to To the estimation processing unit, perform iterative processing on the original image according to the blur kernel, so as to obtain a clear image corresponding to the original image; a region determination unit, connected to the iterative processing unit, in the clear Determine the target area in the image, and extract the target image corresponding to the target area; the deviation correction processing unit is connected to the area determination unit, and performs perspective deviation correction processing on the target image to obtain the front view corresponding to the target image Image; an enhancement processing unit, connected to the deviation correction processing unit, to perform enhancement processing on the front view image to obtain a result image.

In this technical solution, after processing the original image or video, the blur kernel of the image or video is obtained, and the function of the blur kernel is to facilitate deblurring of the image or video; after deblurring, a clear image can be seen, However, when observing, there is still perspective distortion, and the original appearance of the image cannot be seen; therefore, it is necessary to lock the main area of the clear image or video, and then perform perspective correction on this area to make it a front-view image, and it will no longer be visible when observing again. If there is perspective distortion, the image or video itself will be observed; after the enhanced processing of the front-view image, the clarity and contrast of the front-view image will be further improved, and the user's experience effect will also be significantly enhanced; therefore, the technical solution , can deblur images or videos, and perform perspective correction and enhancement on them, so that users can see undistorted images with high clarity and high contrast no matter from which angle they start.

In the above technical solution, preferably, the estimation processing unit performs estimation processing on the original image according to the following formula:

$\underset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 1</span>}}$

Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, and α, β are the weight values of the blur kernel.

In this technical solution, the purpose of the estimation process is to obtain the blur kernel of the original image, and restore a clear original image when the blur kernel is known, and obtaining the very important information of the blur kernel will make the deconvolution work becomes easier, therefore, this technical solution can simplify the deblurring work, and is beneficial to quickly restore the original blurred image to a clear blurred image.

Wherein, the original image can be smoothed first, so as to obtain the preprocessed image, and the smoothing here can be realized by using the existing smoothing technology, which will not be repeated here.

In the above technical solution, preferably, the iterative processing unit performs iterative processing on the original image according to the following formula:

$\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&gamma;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; u</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; v</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; ,</span>$

Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, is the transverse gradient of the original image, is the longitudinal gradient of the original image, and α and γ are weight parameters.

In this technical scheme, a clear image can be restored by substituting the obtained blur kernel into the iterative formula. At the same time, α, γ and these two gradient parameters can be adjusted by adjusting the proportion of the gradient information in the final restored clear image. weight, so that the horizontal and vertical proportions of the restored image are coordinated.

In the above technical solution, preferably, determining the target area in the clear image specifically includes: extracting the edge of the clear image and performing Hough transform to create a Hough parameter space, and using the Hough parameter Find the peak value of the accumulator in the space to detect the straight line; divide the detected straight line into four groups of quadrilaterals according to the slope, and use the enumeration method to obtain the vertices of all the quadrilaterals to obtain the vertex set; use Harris corner detection pair Vertex detection is performed on the clear image, and the vertices in the vertex set are matched with vertices obtained by using the Harris corner point detection to find the largest and most dominant quadrilateral as the target area.

In this technical solution, the target area is a quadrilateral, and the four vertices of the quadrilateral are obtained by matching the multiple vertices in the vertex set with the multiple vertices obtained by Harris corner point detection. This method is compared to the enumeration method only Or in terms of determining the four vertices of the quadrilateral only through the Harris corner points, the four vertices of the quadrilateral can be determined more accurately. Therefore, the technical solution can ensure that the quadrilateral found in the clear image is the largest and most important, thereby ensuring that the quadrilateral contains most of the information in the original image.

Among them, the Hough transform is one of the basic methods for recognizing geometric shapes from images in image processing. The Hough transform is not affected by the rotation of the graphics, and it is easy to change the geometry quickly. It is widely used and there are many improved algorithms. The most basic Hough transform is to detect straight lines from black and white images. The Harris corner detection algorithm is a common template-based detection algorithm. This detection algorithm mainly considers the grayscale changes of points in the pixel field, that is, the changes in image brightness, and defines points that are sufficiently large in contrast to adjacent brightness as corner points. .

In the above technical solution, preferably, the perspective deflection processing specifically includes: determining the deflection correction parameters corresponding to the target image and the original image according to the four vertices of the target area and the original image, and The deviation correction parameters determine each pixel in the target image, wherein the deviation correction parameters corresponding to the target image and the original image are determined according to the following formula:

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccccc}\mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}\\ \mathrm{<span\; class="notranslate">\; d</span>}\\ \mathrm{<span\; class="notranslate">\; e</span>}\\ \mathrm{<span\; class="notranslate">\; f</span>}\\ \mathrm{<span\; class="notranslate">\; g</span>}\\ \mathrm{<span\; class="notranslate">\; h</span>}\end{array}\right]<span\; class="notranslate">\; .</span>$

Wherein, x, y are the coordinates of the pixels in the original image, x', y' are the coordinates of the pixels in the target area, a, b, c, d, e, f, g and h are the coordinates of the pixels in the target area Correction parameters.

In this technical solution, according to the corresponding positions of the four vertices of the quadrilateral in the front view image and the four points in the perspective image, the deviation correction parameter between the front view image and the perspective image can be calculated. The pixels can be mapped one by one in the front-view image, and finally the entire original image is corrected into a front-view image, and the front-view image no longer has the problem of perspective distortion. At the same time, the image processing device can also count the corrected front-view image. Histogram, and perform histogram equalization to further improve the contrast of the front-view image, and then sharpen the edge position of the front-view image to further improve the clarity of the text. Therefore, this technical solution can correct the perspective image and improve its contrast and clarity, so that no matter where the user stands, he can see a clear, high-contrast and undistorted image.

In the above technical solution, preferably, further comprising: a display unit, configured to display the original image, the target image and/or the resultant image according to the received display command.

In this technical solution, the original image, the target image obtained during the intermediate processing, and the final result image can be displayed to the user, which is convenient for the user to view. Among them, the target image is displayed to the user, and the user can also adjust the target area corresponding to the target image according to his needs. For example, the target area corresponding to the target image is not actually the area that the user wants to view. At this time, the user can set The area you want to view is set as the target area, so that the final result image can meet the user's viewing requirements.

In another aspect of the present invention, an image processing method is proposed, including: performing estimation processing on the received original image to obtain a blur kernel corresponding to the original image; according to the blur kernel, performing Iterative processing to obtain a clear image corresponding to the original image; determining a target area in the clear image, and extracting a target image corresponding to the target area; performing perspective correction processing on the target image to obtain A front-view image corresponding to the target image; performing enhancement processing on the front-view image to obtain a result image.

In this technical solution, after the original image or video is processed, the blur kernel of the image or video is obtained, and the function of the blur kernel is to facilitate deblurring of the image or video; after deblurring, people will see a clear image , but when observing, there is still perspective distortion, and the original appearance of the image cannot be seen; therefore, it is necessary to lock the main area of the clear image or video, and then perform perspective correction on this area to make it a front-view image, so that it will not be visible when observing again. If there is perspective distortion again, the image or video itself will be observed; after the enhanced processing of the front-view image, the clarity and contrast of the front-view image will be further improved, and the user's experience effect will also be significantly enhanced; therefore, this technology The solution can deblur images or videos, and perform perspective correction and enhancement on them, so that users can see undistorted images with high clarity and high contrast no matter from which angle they start.

In the above technical solution, preferably, the original image is estimated according to the following formula:

$\underset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 1</span>}}$

Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, and α, β are the weight values of the blur kernel.

In this technical solution, the purpose of the estimation process is to obtain the blur kernel of the original image, and restore a clear original image when the blur kernel is known, and obtaining the very important information of the blur kernel will make the deconvolution The work becomes easier, therefore, the technical solution can simplify the deblurring work, and is beneficial to quickly restore the original blurred image to a clear blurred image.

Wherein, the original image can be smoothed first, so as to obtain the preprocessed image, and the smoothing here can be realized by using the existing smoothing technology, which will not be repeated here.

In the above technical solution, preferably, the original image is iteratively processed according to the following formula:

$\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&gamma;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; u</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; v</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; ,</span>$

Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, is the transverse gradient of the original image, is the longitudinal gradient of the original image, α, γ are weight parameters.

In this technical solution, a clear image can be restored by substituting the obtained blur kernel into the iterative formula. At the same time, the two gradient parameters, α and γ, can adjust the proportion of the gradient information in the final restored clear image. Weight, so that the horizontal and vertical proportions of the restored image are coordinated.

In the above technical solution, preferably, determining the target area in the clear image specifically includes: extracting the edge of the clear image and performing Hough transform to create a Hough parameter space, and using the Hough parameter Find the peak value of the accumulator in the space to detect the straight line; divide the detected straight line into four groups of quadrilaterals according to the slope, and use the enumeration method to obtain the vertices of all the quadrilaterals to obtain the vertex set; use Harris corner detection pair Vertex detection is performed on the clear image, and the vertices in the vertex set are matched with vertices obtained by using the Harris corner point detection to find the largest and most dominant quadrilateral as the target area.

In this technical scheme, the target area is a quadrilateral, and the four vertices of the quadrilateral are obtained by matching many vertices in the vertex set with the multiple vertices obtained by Harris corner point detection. Compared with only enumeration methods or In terms of determining the four vertices of the quadrilateral only through the Harris corner points, the four vertices of the quadrilateral can be determined more accurately. Therefore, the technical solution can ensure that the quadrilateral found in the clear image is the largest and most important, thereby ensuring that the quadrilateral contains most of the information in the original image.

Among them, the Hough transform is one of the basic methods for recognizing geometric shapes from images in image processing. The Hough transform is not affected by the rotation of the graphics, and it is easy to change the geometry quickly. It is widely used and there are many improved algorithms. The most basic Hough transform is to detect straight lines from black and white images. The Harris corner detection algorithm is a common template-based detection algorithm. This detection algorithm mainly considers the grayscale changes of points in the pixel field, that is, the changes in image brightness, and defines points that are sufficiently large in contrast to adjacent brightness as corner points. .

In the above technical solution, preferably, the perspective deflection processing specifically includes: determining the deflection correction parameters corresponding to the target image and the original image according to the four vertices of the target area and the original image, and The correction parameters determine each pixel in the target image,

Wherein, the deviation correction parameters corresponding to the target image and the original image are determined according to the following formula:

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccccc}\mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}\\ \mathrm{<span\; class="notranslate">\; d</span>}\\ \mathrm{<span\; class="notranslate">\; e</span>}\\ \mathrm{<span\; class="notranslate">\; f</span>}\\ \mathrm{<span\; class="notranslate">\; g</span>}\\ \mathrm{<span\; class="notranslate">\; h</span>}\end{array}\right]<span\; class="notranslate">\; .</span>$

Wherein, x, y are the coordinates of the pixels in the original image, x', y' are the coordinates of the pixels in the target area, a, b, c, d, e, f, g and h are the Correction parameters.

In this technical solution, according to the corresponding positions of the four vertices of the quadrilateral in the front view image and the four points in the perspective image, the deviation correction parameter between the front view image and the perspective image can be calculated. The pixels can be mapped one by one in the front-view image, and finally the entire original image is corrected into a front-view image, and the front-view image no longer has the problem of perspective distortion. At the same time, the image processing device can also count the corrected front-view image. Histogram, and perform histogram equalization to further improve the contrast of the front-view image, and then sharpen the edge position of the front-view image to further improve the clarity of the text. Therefore, this technical solution can correct the perspective image and improve its contrast and clarity, so that no matter where the user stands, he can see a clear, high-contrast and undistorted image.

In the above technical solution, preferably, further comprising: displaying the original image, the target image and/or the resultant image according to the received display command.

In this technical solution, the original image, the target image obtained during the intermediate processing, and the final result image can be displayed to the user, which is convenient for the user to view. Among them, the target image is displayed to the user, and the user can also adjust the target area corresponding to the target image according to his needs. For example, the target area corresponding to the target image is not actually the area that the user wants to view. At this time, the user can set The area you want to view is set as the target area, so that the final result image can meet the user's viewing requirements.

Through the above technical solution, the user is no longer limited by the viewing angle when watching the image, and can see a clear and undistorted front-view image from any angle.

Description of drawings

FIG. 1 shows a schematic structural diagram of an image processing device according to an embodiment of the present invention;

Fig. 2 shows the flowchart of the image processing method according to an embodiment of the present invention;

FIG. 3 shows a specific flowchart of an image processing method according to another embodiment of the present invention;

4A to 4D illustrate screenshots of image processing results according to an embodiment of the present invention.

Detailed ways

In order to understand the above-mentioned purpose, features and advantages of the present invention more clearly, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described here. Therefore, the protection scope of the present invention is not limited by the specific details disclosed below. EXAMPLE LIMITATIONS.

Fig. 1 shows a schematic structural diagram of an image processing device according to an embodiment of the present invention.

As shown in FIG. 1, an image processing device 100 according to an embodiment of the present invention includes: an estimation processing unit 102, which performs estimation processing on a received original image to obtain a blur kernel corresponding to the original image; an iterative processing unit 104, connected to the estimation processing unit 102, performing iterative processing on the original image according to the blur kernel, so as to obtain a clear image corresponding to the original image; an area determination unit 106, connected to the iterative processing unit 104. Determine the target area in the clear image, and extract the target image corresponding to the target area; the deviation correction processing unit 108 is connected to the area determination unit 106, and performs perspective deviation correction processing on the target image to Obtain the front view image corresponding to the target image; the enhancement processing unit 110 is connected to the deviation correction processing unit 108, and performs enhancement processing on the front view image to obtain a result image.

In this technical solution, after the original image or video is processed, the blur kernel of the image or video is obtained, and the function of the blur kernel is to facilitate deblurring of the image or video; after deblurring, people will see a clear image , but when observing, there is still perspective distortion, and the original appearance of the image cannot be seen; therefore, it is necessary to lock the main area of the clear image or video, and then perform perspective correction on this area to make it a front-view image, so that it will not be visible when observing again. If there is perspective distortion again, the image or video itself will be observed; after the enhanced processing of the front-view image, the clarity and contrast of the front-view image will be further improved, and the user's experience effect will also be significantly enhanced; therefore, this technology The solution can deblur images or videos, and perform perspective correction and enhancement on them, so that users can see undistorted images with high clarity and high contrast no matter from which angle they start.

In the above technical solution, preferably, the estimation processing unit 102 performs estimation processing on the original image according to the following formula:

$\underset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 1</span>}}$

Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, and α, β are the weight values of the blur kernel.

In this technical solution, the purpose of the estimation process is to obtain the blur kernel of the original image, because we use non-blind deconvolution to deblur, that is, to restore a clear original image when the blur kernel is known. Obtaining the very important information of the blur kernel will make the work of deconvolution easier. Therefore, this technical solution can simplify the work of deblurring, and is conducive to quickly restoring the original blurred image to a clear blurred image.

Wherein, the original image can be smoothed first, so as to obtain the preprocessed image, and the smoothing here can be realized by using the existing smoothing technology, which will not be repeated here.

In the above technical solution, preferably, the iterative processing unit 104 iteratively processes the original image according to the following formula:

$\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&gamma;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; u</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; v</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; ,</span>$

Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, is the transverse gradient of the original image, is the longitudinal gradient of the original image, and α and γ are weight parameters.

In this technical solution, a clear image can be restored by substituting the obtained blur kernel into an iterative formula, and at the same time, α and γ are weight parameters. These two gradient parameters can adjust the weight of the gradient information in the final restored clear image, so that the horizontal and vertical proportions of the restored image are coordinated.

In the above technical solution, preferably, determining the target area in the clear image specifically includes: extracting the edge of the clear image and performing Hough transform to create a Hough parameter space, and using the Hough parameter Find the peak value of the accumulator in the space to detect the straight line; divide the detected straight line into four groups of quadrilaterals according to the slope, and use the enumeration method to obtain the vertices of all the quadrilaterals to obtain the vertex set; use Harris corner detection pair Vertex detection is performed on the clear image, and the vertices in the vertex set are matched with vertices obtained by using the Harris corner point detection to find the largest and most dominant quadrilateral as the target area.

In this technical solution, the target area is a quadrilateral, and the four vertices of the quadrilateral are obtained by matching a plurality of vertices in the vertex set with a plurality of vertices obtained by Harris corner point detection. For example, the four vertices of the quadrilateral can be determined more accurately by using the method or only by Harris corner points. Therefore, the technical solution can ensure that the quadrilateral found in the clear image is the largest and most important, thereby ensuring that the quadrilateral contains most of the information in the original image.

Among them, the Hough transform is one of the basic methods for recognizing geometric shapes from images in image processing. The Hough transform is not affected by the rotation of the graphics, and it is easy to change the geometry quickly. It is widely used and there are many improved algorithms. The most basic Hough transform is to detect straight lines from black and white images. The Harris corner detection algorithm is a common template-based detection algorithm. This detection algorithm mainly considers the grayscale changes of points in the pixel field, that is, the changes in image brightness, and defines points that are sufficiently large in contrast to adjacent brightness as corner points. .

In the above technical solution, preferably, the perspective deflection processing specifically includes: determining the deflection correction parameters corresponding to the target image and the original image according to the four vertices of the target area and the original image, and The deviation correction parameters determine each pixel in the target image, wherein the deviation correction parameters corresponding to the target image and the original image are determined according to the following formula:

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccccc}\mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}\\ \mathrm{<span\; class="notranslate">\; d</span>}\\ \mathrm{<span\; class="notranslate">\; e</span>}\\ \mathrm{<span\; class="notranslate">\; f</span>}\\ \mathrm{<span\; class="notranslate">\; g</span>}\\ \mathrm{<span\; class="notranslate">\; h</span>}\end{array}\right]<span\; class="notranslate">\; .</span>$

Wherein, x, y are the coordinates of the pixels in the original image, x', y' are the coordinates of the pixels in the target area, a, b, c, d, e, f, g and h are the Correction parameters.

In this technical solution, according to the corresponding positions of the four vertices of the quadrilateral in the front view image and the four points in the perspective image, the deviation correction parameter between the front view image and the perspective image can be calculated. The pixels can be mapped one by one in the front-view image, and finally the entire original image is corrected into the front-view image, and the front-view image no longer has the problem of perspective distortion. At the same time, the image processing device 100 can also count the corrected front-view image histogram, and perform histogram equalization to further improve the contrast of the front-view image, and then sharpen the edge position of the front-view image to further improve the clarity of the text. Therefore, this technical solution can correct the perspective image and improve its contrast and clarity, so that no matter where the user stands, he can see a clear, high-contrast and undistorted image.

In the above technical solution, preferably, further comprising: a display unit 112, configured to display the original image, the target image and/or the resultant image according to the received display command.

In this technical solution, the original image, the target image obtained during the intermediate processing, and the final result image can be displayed to the user, which is convenient for the user to view. Among them, the target image is displayed to the user, and the user can also adjust the target area corresponding to the target image according to his needs. For example, the target area corresponding to the target image is not actually the area that the user wants to view. At this time, the user can set The area you want to view is set as the target area, so that the final result image can meet the user's viewing requirements.

Fig. 2 shows a flowchart of an image processing method according to an embodiment of the present invention.

As shown in FIG. 2 , the image processing method according to the embodiment of the present invention includes the following steps: step 202, perform estimation processing on the received original image to obtain the blur kernel corresponding to the original image; step 204, according to the The blur kernel is used to iteratively process the original image to obtain a clear image corresponding to the original image; step 206, determine the target area in the clear image, and extract the target image corresponding to the target area ; Step 208, performing perspective correction processing on the target image to obtain a front view image corresponding to the target image; Step 210, performing enhancement processing on the front view image to obtain a result image.

In this technical solution, after the original image or video is processed, the blur kernel of the image or video is obtained, and the function of the blur kernel is to facilitate deblurring of the image or video; after deblurring, people will see a clear image , but when observing, there is still perspective distortion, and the original appearance of the image cannot be seen; therefore, it is necessary to lock the main area of the clear image or video, and then perform perspective correction on this area to make it a front-view image, so that it will not be visible when observing again. If there is perspective distortion again, the image or video itself will be observed; after the enhanced processing of the front-view image, the clarity and contrast of the front-view image will be further improved, and the user's experience effect will also be significantly enhanced; therefore, this technology The solution can deblur images or videos, and perform perspective correction and enhancement on them, so that users can see undistorted images with high clarity and high contrast no matter from which angle they start.

In the above technical solution, preferably, the original image is estimated according to the following formula:

$\underset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 1</span>}}$

Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, and α, β are the weight values of the blur kernel.

In this technical solution, the purpose of the estimation process is to obtain the blur kernel of the original image, and restore a clear original image when the blur kernel is known, and obtaining the very important information of the blur kernel will make the deconvolution The work becomes easier, therefore, the technical solution can simplify the deblurring work, and is beneficial to quickly restore the original blurred image to a clear blurred image.

Wherein, the original image can be smoothed first, so as to obtain the preprocessed image, and the smoothing here can be realized by using the existing smoothing technology, which will not be repeated here.

In the above technical solution, preferably, the original image is iteratively processed according to the following formula:

$\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&gamma;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; u</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; v</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; ,</span>$

Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, is the transverse gradient of the original image, is the longitudinal gradient of the original image, and α and γ are weight parameters.

In this technical solution, a clear image can be restored by substituting the obtained blur kernel into the iterative formula, and at the same time, and These two gradient parameters can adjust the weight of the gradient information in the final restored clear image, so that the horizontal and vertical proportions of the restored image are coordinated.

In the above technical solution, preferably, determining the target area in the clear image specifically includes: extracting the edge of the clear image and performing Hough transform to create a Hough parameter space, and using the Hough parameter Find the peak value of the accumulator in the space to detect the straight line; divide the detected straight line into four groups of quadrilaterals according to the slope, and use the enumeration method to obtain the vertices of all the quadrilaterals to obtain the vertex set; use Harris corner detection pair Vertex detection is performed on the clear image, and the vertices in the vertex set are matched with the vertices obtained by using the Harris corner point detection to find the largest and most dominant quadrilateral as the target area.

In this technical solution, the target area is a quadrilateral, and the four vertices of the quadrilateral are obtained by matching a plurality of vertices in the vertex set with many vertices obtained by Harris corner point detection. This method is compared to only enumeration method or In terms of determining the four vertices of the quadrilateral only through the Harris corner points, the four vertices of the quadrilateral can be determined more accurately. Therefore, the technical solution can ensure that the quadrilateral found in the clear image is the largest and most important, thereby ensuring that the quadrilateral contains most of the information in the original image.

Among them, the Hough transform is one of the basic methods for recognizing geometric shapes from images in image processing. The Hough transform is not affected by the rotation of the graphics, and it is easy to change the geometry quickly. It is widely used and there are many improved algorithms. The most basic Hough transform is to detect straight lines from black and white images. The Harris corner detection algorithm is a common template-based detection algorithm. This detection algorithm mainly considers the grayscale changes of points in the pixel field, that is, the changes in image brightness, and defines points that are sufficiently large in contrast to adjacent brightness as corner points. .

In the above technical solution, preferably, the perspective deflection processing specifically includes: determining the deflection correction parameters corresponding to the target image and the original image according to the four vertices of the target area and the original image, and The correction parameters determine each pixel in the target image,

Wherein, the deviation correction parameters corresponding to the target image and the original image are determined according to the following formula:

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccccc}\mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}\\ \mathrm{<span\; class="notranslate">\; d</span>}\\ \mathrm{<span\; class="notranslate">\; e</span>}\\ \mathrm{<span\; class="notranslate">\; f</span>}\\ \mathrm{<span\; class="notranslate">\; g</span>}\\ \mathrm{<span\; class="notranslate">\; h</span>}\end{array}\right]<span\; class="notranslate">\; .</span>$

Wherein, x, y are the coordinates of the pixels in the original image, x', y' are the coordinates of the pixels in the target area, a, b, c, d, e, f, g and h are the Correction parameters.

In this technical solution, according to the corresponding positions of the four vertices of the quadrilateral in the front view image and the four points in the perspective image, the deviation correction parameter between the front view image and the perspective image can be calculated. The pixels can be mapped one by one in the front-view image, and finally the entire original image is corrected into a front-view image, and the front-view image no longer has the problem of perspective distortion. At the same time, the image processing device can also count the corrected front-view image. Histogram, and perform histogram equalization to further improve the contrast of the front-view image, and then sharpen the edge position of the front-view image to further improve the clarity of the text. Therefore, this technical solution can correct the perspective image and improve its contrast and clarity, so that no matter where the user stands, he can see a clear, high-contrast and undistorted image.

In the above technical solution, preferably, further comprising: displaying the original image, the target image and/or the resultant image according to the received display command.

In this technical solution, the original image, the target image obtained during the intermediate processing, and the final result image can be displayed to the user, which is convenient for the user to view. Among them, the target image is displayed to the user, and the user can also adjust the target area corresponding to the target image according to his needs. For example, the target area corresponding to the target image is not actually the area that the user wants to view. At this time, the user can set The area you want to view is set as the target area, so that the final result image can meet the user's viewing requirements.

Fig. 3 shows a specific flowchart of an image processing method according to another embodiment of the present invention.

As shown in Figure 3, the image processing method according to another embodiment of the present invention includes the following steps:

Step 302: Estimate the blur kernel of the input image with the prior condition based on regularization sparsity, specifically to solve the following formula:

$\underset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>$

Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, and α, β are the weight values of the blur kernel.

Step 304: Substituting the obtained blur kernel into the following deconvolution formula to obtain a clear image x:

$\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&gamma;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; u</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; v</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; ,</span>$

Wherein, x, y are the same as the above definition, is the horizontal gradient of the original image, is the longitudinal gradient of the original image, and in practical applications, γ=3000, α=0.8 can be set.

Step 306: According to the clear image after deblurring, extract the edge of the image, perform Hough transform on it, detect the straight line by finding the peak value of the accumulator in the Hough parameter space, and divide the detected straight line segment into four quadrilateral parts according to the slope. group, and then use the enumeration method to determine the vertices of the quadrilateral. Finally, match the points of the vertex set with the vertices obtained using Harris corner detection to find the largest and most dominant quadrilateral.

Step 308: Display the finally determined quadrilateral to the user, wherein the user is allowed to correct the quadrilateral detection result, if the user does not modify the detected quadrilateral, perform step 310; otherwise, re-execute step 306, and loop in turn until the user If the detected quadrilateral is satisfied, step 310 is executed.

Step 310: The perspective transformation can transform the perspective image back to the front view image, and its coordinate transformation relationship can be expressed as the following matrix form:

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccccc}\mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}\\ \mathrm{<span\; class="notranslate">\; d</span>}\\ \mathrm{<span\; class="notranslate">\; e</span>}\\ \mathrm{<span\; class="notranslate">\; f</span>}\\ \mathrm{<span\; class="notranslate">\; g</span>}\\ \mathrm{<span\; class="notranslate">\; h</span>}\end{array}\right]<span\; class="notranslate">\; .</span>$

Among them, x, y are the coordinates of the pixels in the original image, x', y' are the coordinates of the pixels in the target (quadrilateral) area, and a-h are the deviation correction parameters. Before perspective correction, we need to determine the correction parameters corresponding to the target image and the original image according to the coordinates of the four vertices of the target area and the coordinates of the corresponding four points in the original image, and then according to the calculated correction parameters, Map the pixels in the original image to the target image one by one. However, the rectified front-view image generally contains fewer pixels than the perspective image, so some holes will appear on the front-view image (conversely, coverage will appear). In order to fill these holes, it is necessary to perform an inverse transformation on the corrected image. The specific process is: use the following formula to map the points in the hole area on the orthographic image back to the original image with perspective distortion, and find the points in the hole area Position (x, y) on the original perspective image, and fill the content of the pixel point corresponding to the position on the front view image, so as to fill the hole in the front view image.

$\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; hf</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; e</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; hc</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; ec</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; b\; f</span>}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; e.g.</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; d\; h</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; ah</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; bg</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; db</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; ae</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; fg</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; cg</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; af</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; dc</span>}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; e.g.</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; d\; h</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; ah</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; bg</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; db</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; ae</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>\end{array}$

Among them, the above formula is the formula for mapping the hollow point on the front-view image back to the original perspective image, and the meaning of the parameters in the formula is the same as above. In general, but the x and y obtained by using this formula are not integers. Therefore, the bicubic interpolation method can be used to calculate the corresponding pixel position of the hole point on the orthographic image on the original perspective image. Among them, the bicubic interpolation method is a more complex interpolation method, which can create smoother than bilinear interpolation. Image edges, which can better fill holes.

Step 312: Statize the histogram of the image after skew correction, and perform histogram equalization on it to improve the contrast of the image. Then use Gaussian deconvolution to sharpen the edge position of the corrected image to improve the clarity of the text.

Through the above steps, the blurred image or video can be made clear, and the perspective distorted image or video can be turned into a front view image, so that no matter where the user stands to watch the picture or video, they can see the original appearance of the image or video clearly. frontal image of .

Specifically, the present invention can not only perform front-view processing on images, but also on video, etc., and if the processing is video, the video is processed frame by frame as an image in sequence from step 302 to step 312.

4A to 4D illustrate screenshots of image processing results according to an embodiment of the present invention.

Among them, Fig. 4A shows the original image, which is blurred; Fig. 4B shows the deblurred image, which is obviously clearer than Fig. 4A, but it is still a perspective distortion map, and there is a certain angle of inclination. It fully conforms to the user's viewing habits; Figure 4C shows the image that has been corrected for perspective, and the front view image is completely in line with the user's viewing habits without any perspective distortion; Compared with 4C, this image has higher contrast and clarity.

The technical solution of the present invention has been described in detail above in conjunction with the accompanying drawings. Through the technical solution of the present invention, when users watch images or videos, they are not affected by viewing angles, and they can see clearly High-resolution, high-contrast front-view images without perspective distortion.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (12)

1. An image processing device, characterized in that, comprising: an estimation processing unit, performing estimation processing on the received original image to obtain a blur kernel corresponding to the original image; An iterative processing unit, connected to the estimation processing unit, performs iterative processing on the original image according to the blur kernel, so as to obtain a clear image corresponding to the original image; an area determining unit, connected to the iterative processing unit, determining a target area in the clear image, and extracting a target image corresponding to the target area; a deviation correction processing unit, connected to the area determination unit, and performing perspective deviation correction processing on the target image to obtain a front view image corresponding to the target image; The enhancement processing unit is connected to the deviation correction processing unit, and performs enhancement processing on the front view image to obtain a result image. 2. The image processing device according to claim 1, wherein the estimation processing unit performs estimation processing on the original image according to the following formula: $\underset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 1</span>}}$ Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, and α, β are the weight values of the blur kernel. 3. The image processing device according to claim 1, wherein the iterative processing unit performs iterative processing on the original image according to the following formula: $\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&gamma;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; u</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; v</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; ,</span>$ Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, is the transverse gradient of the original image, is the longitudinal gradient of the original image, and α and γ are weight parameters. 4. The image processing device according to claim 1, wherein determining the target area in the clear image comprises: Extracting the edge of the clear image and performing Hough transform to create a Hough parameter space, and finding the accumulator peak in the Hough parameter space to detect a straight line; Divide the detected straight lines into four groups of quadrilaterals according to their slopes, and use an enumeration method to obtain vertices of all quadrilaterals to obtain a vertex set; Use Harris corner detection to perform vertex detection on the clear image, and match the vertices in the vertex set with the vertices obtained by using the Harris corner detection to find the largest and most dominant quadrilateral as the target area. 5. The image processing device according to claim 1, wherein the perspective correction processing specifically comprises: determining deviation correction parameters corresponding to the target image and the original image according to the four vertices of the target area and the original image, and determining each pixel in the target image according to the deviation correction parameters, Wherein, the deviation correction parameters corresponding to the target image and the original image are determined according to the following formula: $\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccccc}\mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}\\ \mathrm{<span\; class="notranslate">\; d</span>}\\ \mathrm{<span\; class="notranslate">\; e</span>}\\ \mathrm{<span\; class="notranslate">\; f</span>}\\ \mathrm{<span\; class="notranslate">\; g</span>}\\ \mathrm{<span\; class="notranslate">\; h</span>}\end{array}\right]<span\; class="notranslate">\; .</span>$ Wherein, x, y are the coordinates of the pixels in the original image, x', y' are the coordinates of the pixels in the target area, a, b, c, d, e, f, g and h are the coordinates of the pixels in the target area Correction parameters. 6. The image processing device according to any one of claims 1 to 5, further comprising: A display unit, connected to the deviation correction processing unit, for displaying the original image, the target image and/or the resultant image according to the received display command. 7. An image processing method, characterized in that, comprising: Estimating the received original image to obtain a blur kernel corresponding to the original image; performing iterative processing on the original image according to the blur kernel to obtain a clear image corresponding to the original image; determining a target area in the clear image, and extracting a target image corresponding to the target area; Performing perspective deflection processing on the target image to obtain a front view image corresponding to the target image; Perform enhancement processing on the orthographic image to obtain a result image. 8. The image processing method according to claim 7, wherein the original image is estimated according to the following formula: $\underset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 1</span>}}$ Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, and α, β are the weight values of the blur kernel. 9. The image processing method according to claim 7, wherein the original image is iteratively processed according to the following formula: $\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\mathrm{<span\; class="notranslate">\; \&gamma;</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; u</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; v</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; ,</span>$ Wherein, y is the original image, x is the preprocessed image obtained by smoothing the original image, k is the blur kernel, * is the convolution operation, is the transverse gradient of the original image, is the longitudinal gradient of the original image, and α and γ are weight parameters. 10. The image processing method according to claim 7, wherein determining the target area in the clear image comprises: Extracting the edge of the clear image and performing Hough transform to create a Hough parameter space, and finding the accumulator peak in the Hough parameter space to detect a straight line; Divide the detected straight lines into four groups of quadrilaterals according to their slopes, and use an enumeration method to obtain vertices of all quadrilaterals to obtain a vertex set; Use Harris corner detection to perform vertex detection on the clear image, and match the vertices in the vertex set with the vertices obtained by using the Harris corner detection to find the largest and most dominant quadrilateral as the target area. 11. The image processing method according to claim 7, wherein the perspective correction processing specifically comprises: determining deviation correction parameters corresponding to the target image and the original image according to the four vertices of the target area and the original image, and determining each pixel in the target image according to the deviation correction parameters, Wherein, the deviation correction parameters corresponding to the target image and the original image are determined according to the following formula: $\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccccc}\mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; x</span>}& \mathrm{<span\; class="notranslate">\; the\; y</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; x</span>}& {<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}\\ \mathrm{<span\; class="notranslate">\; d</span>}\\ \mathrm{<span\; class="notranslate">\; e</span>}\\ \mathrm{<span\; class="notranslate">\; f</span>}\\ \mathrm{<span\; class="notranslate">\; g</span>}\\ \mathrm{<span\; class="notranslate">\; h</span>}\end{array}\right]<span\; class="notranslate">\; .</span>$ Wherein, x, y are the coordinates of the pixels in the original image, x', y' are the coordinates of the pixels in the target area, a, b, c, d, e, f, g and h are the coordinates of the pixels in the target area Correction parameters. 12. The image processing method according to any one of claims 7 to 11, further comprising: The original image, the target image and/or the resultant image are displayed according to the received display command.