CN115334245A - Image correction method and device, electronic equipment and storage medium - Google Patents

Abstract

The disclosure relates to an image rectification method, an image rectification device, an electronic device and a storage medium. The image rectification method comprises the following steps: when the camera has a stain, determining a shielded area in a first image shot by the camera; determining an offset angle according to the actual size information of the shielded area and a first constraint relation of the offset angle; controlling the camera to adjust the shooting angle according to the offset angle through the electronic equipment, and shooting a second image, wherein the second image and the first image are shot by the same camera; extracting a region image corresponding to the shielded region in the second image, and adjusting the extracted region image according to a second constraint relation between the shooting angle adjusted by the camera and the element value of the extracted region image to obtain a replacement image; and replacing the image of the shielded area in the first image with the replacement image for image correction. Through the scheme, the influence of the sheltered area in the image on the image quality is reduced, extra cleaning equipment is not needed, and the cost is reduced.

Description

Image correction method, device, electronic equipment and storage medium

divisional application

This application is a divisional application, the application number of the original application is: 201911241547.X, the application date is December 06, 2019, and the title of the invention is "an image correction method, electronic equipment and storage medium".

technical field

Embodiments of the present invention relate to the field of image processing, and in particular, to an image correction method, device, electronic equipment, and storage medium.

Background technique

When the camera is collecting, if there are stains (such as mud spots, etc.) on the camera lens, it will cause some data in the subsequent collected images or video data to be polluted (the data collected by the part blocked by the mud spots will be distorted) .

However, the inventors have found that at least the following problems exist in the prior art: in order to reduce the impact of stains on the captured images, it is necessary to clean the stains on the camera with an external cleaning device, which is costly.

Contents of the invention

The purpose of the embodiment of the present invention is to provide an image correction method, device, electronic equipment and storage medium, so that the influence of the occluded area in the image on the image quality is reduced, no additional cleaning equipment is needed, and the cost is reduced.

In order to solve the above-mentioned technical problems, an embodiment of the present invention provides an image correction method, which includes the following steps: when there is a stain on the camera, determining the occluded area in the first image captured by the camera; according to the occluded area The first constraint relationship between the actual size information of the offset angle and the offset angle determines the offset angle; the electronic device controls the camera to adjust the shooting angle according to the offset angle, and takes a second image, and the second image is consistent with the offset angle The first image is taken by the same camera; an area image corresponding to the covered area in the second image is extracted, and according to the second constraint relationship between the shooting angle adjusted by the camera and the element value of the extracted area image, Adjusting the extracted area image to obtain a replacement image; replacing the image of the covered area in the first image with the replacement image to perform image correction.

Embodiments of the present invention also provide an image correction device, including: an area determination module, configured to determine an occluded area in the first image captured by the camera when there is a stain on the camera; an angle determination module, configured to The actual size information of the covered area and the first constraint relationship of the offset angle determine the offset angle; the camera control module is used to control the camera through the electronic device to adjust the shooting angle according to the offset angle, and shoot The second image, the second image and the first image are taken by the same camera; the image adjustment module is used to extract the area image corresponding to the covered area in the second image, and adjust the image according to the camera The second constraint relationship between the shooting angle of the extracted area image and the element value of the extracted area image is adjusted to obtain a replacement image; an image correction module is used to replace all the elements in the first image with the replacement image Perform image correction on the image of the above-mentioned occluded area.

The embodiment of the present invention also provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication; wherein, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by at least one processor. Executed by a processor, so that at least one processor can execute the image correction method mentioned in the above-mentioned embodiments.

Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, and implementing the image correction method mentioned in the above embodiment when the computer program is executed by a processor.

Further, the determining the offset angle according to the actual size information of the occluded area and the first constraint relationship of the offset angle specifically includes: according to the image size information of the occluded area, determining the Actual size information: determining the offset angle according to the actual size information and the preset first constraint relationship between the actual size information and the offset angle.

Further, the actual size information of the occluded area is the actual height of the space corresponding to the occluded area, and the first constraint relationship is: offset angle=first scale parameter*actual height÷the focal length of the camera.

Further, in the extraction of the area image corresponding to the occluded area in the second image, according to the second constraint relationship between the shooting angle adjusted by the camera and the element value of the extracted area image, the extracted Adjust the area image of the area to obtain a replacement image, specifically including: using the image of the area corresponding to the occluded area as a candidate image; according to the second constraint relationship between the angle adjusted by the camera and the element value of the pixel of the candidate image , determining the element value of each pixel in the replacement image; adjusting the candidate image to obtain the replacement image.

Further, the determining the element value of each pixel in the replacement image according to the second constraint relationship between the angle adjusted by the camera and the element value of the pixel in the candidate image specifically includes: according to the angle adjusted by the camera, and The second constraint relationship of the element value of the pixel of the candidate image, the angle adjusted by the camera, and the element value of the pixel of the replacement image determines the element value of each pixel in the replacement image; wherein, the second constraint relationship is: the replacement image The element value of the pixel in=the element value of the pixel of the candidate image*the second scale parameter, the second scale parameter is determined according to the angle adjusted by the camera.

Further, the image correction method further includes: extracting the image features of the first image; comparing the image features of the first image with the image features of the third image stored in the image data set, and judging according to the comparison result Whether there is an occluded area in the first image.

Further, the judging whether there is an occluded area in the first image according to the comparison result specifically includes: if the comparison result between the first image and all the images in the image data set indicates that the first image If the number of comparison results of the occluded area in the first image is not less than N, it is determined that there is an occluded area in the first image; if the number of comparison results indicating that there is an occluded area in the first image is less than N, then It is determined that there is no occluded area in the first image.

Further, the specific method of determining the area corresponding to the occluded area in the second image includes: determining the left boundary coordinates and right boundary coordinates of the occluded area in the first image, and dividing the left boundary coordinates of the occluded area The boundary coordinates are used as the left boundary coordinates of the area corresponding to the occluded area in the second image, and the right boundary coordinates of the occluded area are used as the area corresponding to the occluded area in the second image. The right boundary coordinates of the area; according to the adjusted angle of the camera and the focal length of the camera, determine the adjustment amount of the upper boundary coordinates and the lower boundary coordinates; determine the upper boundary coordinates and the lower boundary coordinates of the covered area in the first image , the value obtained by adding the adjustment amount to the upper boundary coordinate of the occluded area is used as the upper boundary coordinate of the area corresponding to the occluded area in the second image, and the lower boundary coordinate of the occluded area is The value obtained by adding the adjustment amount is used as the lower boundary coordinate of the area corresponding to the occluded area in the second image.

Compared with the prior art, the embodiment of the present invention can automatically adjust the shooting angle of the camera to acquire the second image containing the image of the blocked area when the blocked area exists in the first image. Correcting the first image based on the second image can reduce the size of the occluded area in the first image, and reduce the loss of image data due to the existence of occlusions such as stains, so that reducing the occluded area in the image has an impact on image quality Impact. Since the image correction method does not require additional cleaning equipment to clean the stains, the cost is reduced.

Description of drawings

One or more embodiments are exemplified by the pictures in the corresponding drawings, and these exemplifications do not constitute a limitation to the embodiments. Elements with the same reference numerals in the drawings represent similar elements. Unless otherwise stated, the drawings in the drawings are not limited to scale.

Fig. 1 is a flowchart of an image correction method according to a first embodiment of the present invention;

2 is a schematic diagram of the positional relationship between the camera and the photographed object according to the first embodiment of the present invention;

Figure 3a is a schematic diagram of a first image according to a first embodiment of the present invention;

Figure 3b is a schematic diagram of a second image according to the first embodiment of the present invention;

Fig. 3c is a schematic diagram of the corrected first image according to the first embodiment of the present invention;

4 is a flowchart of an image correction method according to a second embodiment of the present invention;

5 is a schematic structural diagram of an image correction device according to a third embodiment of the present invention;

6 is a schematic structural diagram of an image correction device according to a fourth embodiment of the present invention;

7 is a schematic structural diagram of an image correction device according to a fifth embodiment of the present invention;

FIG. 8 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, various implementation modes of the present invention will be described in detail below in conjunction with the accompanying drawings. However, those of ordinary skill in the art can understand that, in each implementation manner of the present invention, many technical details are provided for readers to better understand the present application. However, even without these technical details and various changes and modifications based on the following implementation modes, the technical solution claimed in this application can also be realized.

Unless the context clearly requires, throughout the specification and claims, "comprises", "comprises" and similar words should be interpreted in an inclusive sense rather than an exclusive or exhaustive meaning; that is, "including but not limited to" meaning.

In the description of the present disclosure, it should be understood that the terms "first", "second" and so on are used for descriptive purposes only, and should not be understood as indicating or implying relative importance. In addition, in the description of the present disclosure, unless otherwise specified, "plurality" means two or more.

The first embodiment of the present invention relates to an image correction method, which is applied to an electronic device, such as a terminal or a server. As shown in Figure 1, the image correction method includes the following steps:

Step 101: When there is a stain on the camera, determine an occluded area in the first image captured by the camera.

Specifically, when there is a stain on the camera or the object to be photographed is blocked by other objects, the electronic device determines that there is a blocked area in the first image.

Step 102: Determine the offset angle according to the actual size information of the covered area and the first constraint relationship of the offset angle.

The manner in which the electronic device adjusts the shooting angle of the camera is described below with an example.

In mode a, the electronic device controls the rotation of the camera to the offset angle according to the preset offset angle.

Mode b, the electronic device determines the actual size information of the occluded area according to the image size information of the occluded area; determines the offset angle according to the actual size information, and the preset first constraint relationship between the actual size information and the offset angle; Control the camera rotation offset angle.

In one embodiment, the actual size information of the blocked area is the actual height of the space corresponding to the blocked area. The first constraint relationship is: offset angle=first proportional parameter*actual height÷the focal length of the camera. Wherein, the first ratio parameter can be obtained according to actual measurement results, and the value range is -90°˜90°. The schematic diagram of the positional relationship between the camera 11 and the shooting object 12 is shown in Figure 2. Assume that the actual height of the space corresponding to the image offset is d, and the focal length of the camera is f. As can be seen from Figure 2, the camera offset angle B and the offset The height is directly proportional, therefore, the functional relationship between the offset angle and the height of the blocked area is: B=k*d/f.

It should be noted that there may be other constraint relationships between the size of the shaded area and the offset angle. This embodiment is only for illustration, and does not limit the specific constraint relationship between the size of the shaded area and the offset angle.

Step 103: Control the camera through the electronic device to adjust the shooting angle according to the offset angle, and shoot a second image, where the second image is taken by the same camera as the first image.

When there is a stain on the camera, or the object to be photographed is blocked by other objects, and the electronic device considers that there is an occluded area in the first image, the electronic device adjusts the shooting angle of the camera and takes the picture according to the offset angle determined in step 102 A second image, so that some or all of the image that is obscured by a stain or other object appears in the second image.

Specifically, since the second image includes part or all of the image that is blocked by stains or other objects, the electronic device may improve the first image based on the second image, so as to reduce the size of the blocked area in the first image.

Step 104: extracting the area image corresponding to the occluded area in the second image, and according to the second constraint relationship between the shooting angle adjusted by the camera and the element value of the extracted area image, the extracted area The image is adjusted to obtain a replacement image.

Step 105: Replace the image of the occluded area in the first image with the replacement image to perform image correction.

Regarding steps 104 and 105, specifically, in one embodiment, the process of correcting the first image by the electronic device includes: determining the position information of the occluded area in the first image and the angle adjusted by the camera An area corresponding to the occluded area; extracting an image of the area corresponding to the occluded area in the second image to obtain a replacement image; replacing the image of the occluded area in the first image with the replacement image.

In one embodiment, the method for the electronic device to determine the area corresponding to the occluded area in the second image is: determine the left boundary coordinates and right boundary coordinates of the occluded area in the first image, and the left boundary coordinates of the occluded area As the left boundary coordinates of the area corresponding to the occluded area in the second image, the right boundary coordinates of the occluded area are used as the right boundary coordinates of the area corresponding to the occluded area in the second image. According to the adjusted angle of the camera and the focal length of the camera, the adjustment amounts of the upper boundary coordinates and the lower boundary coordinates are determined. Determine the upper boundary coordinates and lower boundary coordinates of the occluded area in the first image, add the value obtained by adding the adjustment amount to the upper boundary coordinates of the occluded area, as the upper boundary coordinates of the area corresponding to the occluded area in the second image, A value obtained by adding the adjustment amount to the lower boundary coordinate of the occluded area is used as the lower boundary coordinate of the area corresponding to the occluded area in the second image.

It should be noted that those skilled in the art can understand that in practical applications, other methods can also be used to determine the position of the region corresponding to the blocked region in the second image. This embodiment does not limit the electronic device to determine Method for the position of the region corresponding to the occluded region.

In one embodiment, the process of obtaining the replacement image includes: the electronic device takes an image of an area corresponding to the occluded area in the second image as a candidate image; and adjusts the candidate image according to the angle adjusted by the camera to obtain the replacement image.

In one embodiment, the process of the electronic device adjusting the candidate image includes: according to the angle adjusted by the camera, and the second constraint relationship between the element value of the pixel of the candidate image, the angle adjusted by the camera, and the element value of the pixel of the replacement image, Determine the element value of each pixel in the replacement image; wherein, the second constraint relationship is: the element value of the pixel in the replacement image=the element value of the pixel in the candidate image*the second scale parameter, and the second scale parameter is determined according to the angle adjusted by the camera . Wherein, the value range of the second ratio parameter is 0-1.

For example, the electronic device can transform the candidate image through nonlinear upscaling of the image to obtain a replacement image, so that the replacement image fits better with the first image. Specifically, the electronic device can convert the image information of the candidate image into matrix data for storage, construct matrix data src to represent the image information of the candidate image, and construct matrix data dst to represent the image information of the replacement image. Assuming that in src, the element value of position (i, j) is s, the element of position (i, j) of src is mapped to the position (i, j) of dst. The mapping relationship is related to the angle of camera adjustment, which can be Establish the following mapping relationship: t=s*k(B), wherein, t represents the element value on the position (i, j) of dst, s represents the element value of the position (i, j) of src, k(B) is a function about the adjustment angle B, that is, the second proportional parameter, which can be obtained according to actual measurement results.

It should be noted that those skilled in the art can understand that the constraint relationship between the second scale parameter and the angle adjusted by the camera can be set as required, which is not limited in this embodiment.

It should be noted that those skilled in the art may understand that in practical applications, the second constraint relationship may be set as required, and this embodiment is only for illustration.

For example, the first image captured by the camera is shown in Figure 3a, and the second image is shown in Figure 3b. The dashed box in Figure 3b indicates the area corresponding to the blocked area in the first image. In the image correction method, the image framed by a dotted line in FIG. 3b can be synthesized with the first image to correct the first image. The corrected first image is shown in FIG. 3c.

It should be noted that the above is only for illustration and does not limit the technical solution of the present invention.

Compared with the prior art, the image correction method provided in this embodiment can automatically adjust the shooting angle of the camera to obtain the second image containing the image of the occluded area when there is an occluded area in the first image . Correcting the first image based on the second image can reduce the size of the occluded area in the first image, and reduce the loss of image data due to the existence of occlusions such as stains, so that reducing the occluded area in the image has an impact on image quality Impact. Since the image correction method does not require additional cleaning equipment to clean the stains, the cost is reduced.

The second embodiment of the present invention relates to an image correction method. In this embodiment, an example is given to illustrate the method for the electronic device to determine that an occluded area exists in the first image in the first embodiment.

Specifically, as shown in FIG. 4, in this embodiment, the method for the electronic device to determine that there is an occluded area in the first image includes the following steps:

Step 201: Extract image features of the first image.

Specifically, the electronic device extracts image features of the currently collected first image.

It should be noted that the image feature may be any one or any combination of features such as grayscale value and brightness value of the image, which is not limited in this embodiment.

Step 202: Compare the image features of the first image with the image features of the third image stored in the image data set, and judge whether there is an occluded area in the first image according to the comparison result.

Specifically, the image data set may include a prestored image, and/or a third image captured by the camera before the first image is captured. If the third image in the image data set is an image without an occluded area, then when an occluded area exists in the first image, there is a difference between the image features of the first image and the image features of the third image. If the third image in the image data set is an image with an occluded area, then when an occluded area exists in the first image, the image features of the first image are similar to the image features of the third image whose area is also occluded.

The process of determining the comparison result by the electronic device will be illustrated below by taking the third image as an image without an occluded area as an example.

In one embodiment, the scene captured by the first image is a static scene. The electronic device compares image features of the first image with image features of a third image of the scene of the image data set. Optionally, the electronic device divides the first image into multiple regions, and if the image features of a certain region of the first image are not similar to the image features of the third image, the comparison result between the first image and the third image indicates that the third image An image has an occluded area. If the image features of all areas of the first image are similar to those of the third image, the comparison result between the first image and the third image indicates that there is no blocked area in the first image. Wherein, the criterion for judging whether the image features of the two regions are similar may be to judge whether the difference of the image features of the two regions is greater than a preset value, if it is greater than the preset value, it is considered that the two are not similar, and if it is not greater than the preset value. value, the two are considered to be similar.

In one embodiment, the scene captured by the first image is a dynamic scene, and the electronic device compares the image features of the first image with the image features of the third image of the scene in the image data set. Optionally, the electronic device divides the first image into multiple regions, and if the image features of a certain region of the first image are similar to those of the third image, the comparison result between the first image and the third image indicates that the first There are occluded areas in the image. If the image features of all regions in the first image are not similar to the image features of the third image, the comparison result between the first image and the third image indicates that there is no blocked area in the first image.

It should be noted that, the manner in which the electronic device determines the comparison result between the first image and the third image may be set as required, and this embodiment is only for illustration.

In an embodiment, the electronic device may determine whether the currently captured scene is a static scene or a dynamic scene according to the proportion of the area in the first image that is similar to the image feature of the third image in the first image. For example, if the ratio is greater than the preset ratio, the currently photographed scene is considered to be a static scene, and if not greater than the preset ratio, the currently photographed scene is considered to be a dynamic scene.

It should be noted that, in practical applications, the electronic device may also determine whether the currently photographed scene is a static scene or a dynamic scene according to external input or other methods, and this embodiment is only for illustration.

In one embodiment, if among the comparison results between the first image and all images in the image data set, the number of comparison results indicating that there is an occluded area in the first image is not less than N, then it is determined that there is an occluded area in the first image ; If the number of comparison results indicating that there is an occluded area in the first image is less than N, then determine that there is no occluded area in the first image. Wherein, N can be set as any positive integer as required.

It is worth mentioning that after the N comparison results indicate that there is an occluded area in the first image, it is determined that there is an occluded area in the first image, and the image correction method shown in steps 101 and 102 is executed, which reduces the An image with an occluded area is judged to be a waste of computing resources caused by an image with an occluded area.

In another embodiment, the first image is sequentially compared with the third image in the image data set, and the comparison result of the image features of any image in the image data set with the image features of the first image indicates that the first image is occluded area, it is considered that the first image has an occluded area.

It is worth mentioning that, if any comparison result indicates that the first image has an occluded area, it is determined that the first image has an occluded area, which improves the recall rate of the electronic device for checking whether the image has an occluded area.

It should be noted that those skilled in the art can understand that in practical applications, the criterion for judging that the first image has an occluded region can be set as required. The above-mentioned embodiment is only for illustration, and this embodiment does not limit the specific judging process.

It should be noted that the above is only for illustration and does not limit the technical solution of the present invention.

Compared with the prior art, in the image correction method provided in this embodiment, the electronic device compares the image features of the first image with the image features of the third image in the image data set to determine whether there is an occluded image in the first image. The region realizes the automatic detection of the blocked region of the first image, and improves the intelligence of the electronic device.

The step division of the above various methods is only for the sake of clarity of description. During implementation, it can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. ; Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but not changing the core design of the algorithm and process are all within the scope of protection of this patent.

The third embodiment of the present invention relates to an image correction device, as shown in FIG. 5 , including: a camera control module 501 and an image correction module 502 . The camera control module 501 is configured to control the camera to adjust the shooting angle and take a second image after determining that there is an occluded area in the first image captured by the camera. The image correction module 502 is used for correcting the image of the occluded area in the first image according to the second image.

More specifically, as shown in FIG. 6 , the image correction device includes: an area determination module 505 , an angle determination module 506 , a camera control module 501 , an image adjustment module 507 and an image correction module 502 .

An area determination module 505, configured to determine the covered area in the first image captured by the camera when there is a stain on the camera;

An angle determination module 506, configured to determine the offset angle according to the actual size information of the covered area and the first constraint relationship of the offset angle;

The camera control module 501 is configured to control the camera through an electronic device to adjust the shooting angle according to the offset angle, and take a second image, and the second image and the first image are taken by the same camera;

The image adjustment module 507 is configured to extract an area image corresponding to the occluded area in the second image, and perform an adjustment on the area image according to the second constraint relationship between the shooting angle adjusted by the camera and the element value of the extracted area image. adjusting the extracted region image to obtain a replacement image;

An image correction module 502, configured to replace the image of the occluded area in the first image with the replacement image to perform image correction.

Wherein, the specific execution functions of the area determination module 505 , the angle determination module 506 , the camera control module 501 , the image adjustment module 507 and the image correction module 502 can refer to the implementation of the relevant steps and methods in FIG. 1 .

It is not difficult to find that this embodiment is a system embodiment corresponding to the first embodiment, and this embodiment can be implemented in cooperation with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here in order to reduce repetition. Correspondingly, the relevant technical details mentioned in this implementation manner can also be applied in the first implementation manner.

It is worth mentioning that all the modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, or a part of a physical unit, or multiple physical units. Combination of units. In addition, in order to highlight the innovative part of the present invention, units that are not closely related to solving the technical problems proposed by the present invention are not introduced in this embodiment, but this does not mean that there are no other units in this embodiment.

The fourth embodiment of the present invention relates to an image correction device. This embodiment is a further improvement on the third embodiment. The specific improvement is: this embodiment also includes a feature extraction module 503 and an image analysis module 504, the two are coupled, It is used to judge whether there is an occluded area in the first image.

Specifically, as shown in FIG. 7 , the image correction device further includes: a feature extraction module 503 for extracting image features of the first image; an image analysis module 504 for centrally storing the image features and image data of the first image The image features of the third image are compared, and according to the comparison result, it is judged whether there is an occluded area in the first image.

Since the second embodiment corresponds to the present embodiment, the present embodiment can be implemented in cooperation with the second embodiment. The relevant technical details mentioned in the second embodiment are still valid in this embodiment, and the technical effects that can be achieved in the second embodiment can also be achieved in this embodiment, and in order to reduce repetition, details are not repeated here. Correspondingly, the relevant technical details mentioned in this embodiment mode can also be applied in the second embodiment mode.

The fifth embodiment of the present invention relates to an electronic device, as shown in FIG. 8 , including: at least one processor 601; and a memory 602 communicatively connected to at least one processor 601; Instructions executed by one processor 601, the instructions are executed by at least one processor 601, so that at least one processor 601 can execute the image correction method as mentioned in the first embodiment or the second embodiment.

The electronic device includes: one or more processors 601 and a memory 602, one processor 601 is taken as an example in FIG. 7 . The processor 601 and the memory 602 may be connected through a bus or in other ways, and connection through a bus is taken as an example in FIG. 7 . The memory 602, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs and modules. The processor 601 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions and modules stored in the memory 602, that is, implements the above-mentioned image correction method.

The memory 602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application program required by at least one function; the data storage area may store an option list and the like. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage devices. In some implementations, the memory 602 may optionally include a memory that is set remotely relative to the processor 601, and these remote memories may be connected to an external device through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

One or more modules are stored in the memory 602, and when executed by one or more processors 601, execute the image correction method in any method implementation above.

The above-mentioned products can execute the methods provided in the embodiments of this application, and have the corresponding functional modules and beneficial effects for executing the methods. For technical details not described in detail in this embodiment, please refer to the methods provided in the embodiments of this application.

A sixth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The above method embodiments are implemented when the computer program is executed by the processor.

That is, those skilled in the art can understand that all or part of the steps in the method of the above-mentioned embodiments can be completed by instructing related hardware through a program, the program is stored in a storage medium, and includes several instructions to make a device ( It may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and other media that can store program codes.

Those of ordinary skill in the art can understand that the above-mentioned embodiments are specific examples for realizing the present invention, and in practical applications, various changes can be made to it in form and details without departing from the spirit and spirit of the present invention. scope.

Claims (11)

1. An image rectification method, comprising:

when the camera has a stain, determining a shielded area in a first image shot by the camera;

determining an offset angle according to the actual size information of the shielded area and a first constraint relation of the offset angle;

controlling the camera to adjust the shooting angle according to the offset angle through electronic equipment, and shooting a second image, wherein the second image and the first image are shot by the same camera;

extracting a region image corresponding to the shielded region in the second image, and adjusting the extracted region image according to a second constraint relation between the shooting angle adjusted by the camera and the element value of the extracted region image to obtain a replacement image;

and replacing the image of the shielded area in the first image by the replacement image for image correction.

2. The image rectification method according to claim 1, wherein the determining the offset angle according to the actual size information of the occluded area and the first constraint relationship of the offset angle specifically includes:

determining actual size information of the shielded area according to the image size information of the shielded area;

and determining the offset angle according to the actual size information and a first constraint relation between the preset actual size information and the offset angle.

3. The image rectification method according to claim 2, wherein the actual size information of the occluded area is an actual height of a space corresponding to the occluded area, and the first constraint relationship is: offset angle = first scale parameter actual height ÷ focal length of the camera.

4. The image rectification method according to claim 1, wherein the extracting a region image corresponding to the occluded region in the second image, and adjusting the extracted region image according to a second constraint relationship between a shooting angle adjusted by the camera and an element value of the extracted region image to obtain a replacement image specifically includes:

taking an image of a region corresponding to the occluded region as a candidate image;

determining the element value of each pixel in the replacement image according to the second constraint relation between the angle adjusted by the camera and the element value of the pixel of the candidate image;

and adjusting the candidate image to obtain the replacement image.

5. The image rectification method according to claim 4, wherein the determining the element value of each pixel in the replacement image according to the second constraint relationship between the angle adjusted by the camera and the element value of the pixel of the candidate image specifically includes:

determining the element value of each pixel in the replacement image according to the angle adjusted by the camera and the second constraint relation among the element value of the pixel of the candidate image, the angle adjusted by the camera and the element value of the pixel of the replacement image;

wherein the second constraint relationship is: the element values of the pixels in the replacement image = element values of the pixels of the candidate image ×. A second scale parameter, the second scale parameter being determined according to the angle of adjustment of the camera.

6. The image rectification method according to claim 1, further comprising:

extracting image features of the first image;

and comparing the image characteristics of the first image with the image characteristics of a third image stored in an image data set, and judging whether the first image has an occluded area or not according to the comparison result.

7. The image rectification method according to claim 6, wherein the determining whether the first image has the occluded area according to the comparison result specifically includes:

if the number of comparison results indicating that the occluded area exists in the first image is not less than N in the comparison results of the first image and all images in the image data set, determining that the occluded area exists in the first image;

if the number of comparison results indicating that the occluded regions exist in the first image is smaller than N, determining that the occluded regions do not exist in the first image.

8. The image rectification method according to claim 1, wherein the determining a region corresponding to the occluded region in the second image includes:

determining left boundary coordinates and right boundary coordinates of an occluded area in the first image, taking the left boundary coordinates of the occluded area as left boundary coordinates of an area corresponding to the occluded area in the second image, and taking the right boundary coordinates of the occluded area as right boundary coordinates of an area corresponding to the occluded area in the second image;

determining the adjustment amount of the upper boundary coordinate and the lower boundary coordinate according to the adjustment angle of the camera and the focal length of the camera;

determining an upper boundary coordinate and a lower boundary coordinate of the occluded area in the first image, taking a value obtained by adding an adjustment amount to the upper boundary coordinate of the occluded area as an upper boundary coordinate of an area corresponding to the occluded area in the second image, and taking a value obtained by adding an adjustment amount to the lower boundary coordinate of the occluded area as a lower boundary coordinate of an area corresponding to the occluded area in the second image.

9. An image rectification apparatus characterized by comprising:

the area determining module is used for determining an occluded area in a first image shot by the camera when the camera has a stain;

the angle determining module is used for determining an offset angle according to the actual size information of the shielded area and a first constraint relation of the offset angle;

the camera control module is used for controlling the camera to adjust the shooting angle according to the offset angle through electronic equipment and shooting a second image, and the second image and the first image are shot by the same camera;

the image adjusting module is used for extracting a region image corresponding to the shielded region in the second image, and adjusting the extracted region image according to a second constraint relation between the shooting angle adjusted by the camera and the element value of the extracted region image to obtain a replacement image;

and the image correction module is used for replacing the image of the shielded area in the first image with the replacement image for image correction.

10. An electronic device, comprising: at least one processor; and (c) a second step of,

a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the image rectification method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the image rectification method according to any one of claims 1 to 8.

Images