CN116962855A - Image compensation method, device, electronic equipment and readable storage medium - Google Patents

Abstract

The application discloses an image compensation method, an image compensation device, electronic equipment and a readable storage medium, and belongs to the technical field of image processing. The method is executed by an electronic device, the electronic device comprises a camera module, the camera module comprises a lens, and the method comprises: acquiring brightness compensation data of a first image based on a first physical offset and calibration compensation data when the camera module acquires the first image, wherein the first physical offset is in a physical offset range of a lens corresponding to the calibration compensation data; the image data of the first image is compensated based on the brightness compensation data.

Description

Image compensation method, device, electronic equipment and readable storage medium

Technical field

This application belongs to the field of image processing technology, and specifically relates to an image compensation method, device, electronic equipment and readable storage medium.

Background technique

Currently, electronic devices can compensate for the brightness of captured images through image compensation data (for example, lens shading compensation data) to improve imaging quality. Usually, the image compensation data is calculated based on the brightness compensation data calibrated when the physical center of the lens in the electronic device is aligned with the physical center of the image sensor.

However, during the process of collecting images, the jitter of the electronic equipment will cause the physical center of the above-mentioned camera to shift, resulting in problems such as flickering of dark areas of water ripples and flickering of light and dark in the four corners after image compensation.

Contents of the invention

The purpose of the embodiments of the present application is to provide an image compensation method, device, electronic equipment and readable storage medium, which can solve the problem in related technologies that water ripples flash in dark areas or four corners flicker after image compensation.

In a first aspect, embodiments of the present application provide an image compensation method. The method is executed by an electronic device. The electronic device includes a camera module, and the camera module includes a lens. The method includes: based on the camera module when collecting the first image. The first physical offset and the calibration compensation data are used to obtain the brightness compensation data of the first image. The first physical offset is within the physical offset range of the lens corresponding to the calibration compensation data; based on the brightness compensation data, the first image is image data for compensation.

In a second aspect, embodiments of the present application provide an image compensation device. The device includes a camera module. The camera module includes a lens. The device includes an acquisition module and a compensation module; the acquisition module is used to collect the first image based on the camera module. The first physical offset and calibration compensation data of the image are obtained to obtain the brightness compensation data of the first image, and the first physical offset is within the physical offset range of the lens corresponding to the calibration compensation data; the compensation module is used to obtain based on The brightness compensation data obtained by the module compensates the image data of the first image.

In a third aspect, embodiments of the present application provide an electronic device. The electronic device includes a processor and a memory. The memory stores programs or instructions that can be run on the processor. The programs or instructions are processed by the processor. When the processor is executed, the steps of the method described in the first aspect are implemented.

In a fourth aspect, embodiments of the present application provide a readable storage medium. Programs or instructions are stored on the readable storage medium. When the programs or instructions are executed by a processor, the steps of the method described in the first aspect are implemented. .

In a fifth aspect, embodiments of the present application provide a chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the first aspect. the method described.

In a sixth aspect, embodiments of the present application provide a computer program product, the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the method as described in the first aspect.

In the embodiment of the present application, the electronic device can obtain the brightness compensation data of the first image based on the first physical offset and calibration compensation data when the camera module collects the first image. The first physical offset is in the calibration compensation data. Within the physical offset range of the corresponding lens; and based on the brightness compensation data, the image data of the first image is compensated. Through this solution, since the brightness compensation data used by the electronic device to compensate the image data of the first image collected is obtained based on the actual physical offset and calibration compensation data when the first image is collected, the brightness The compensation data can effectively avoid the impact of the shift of the physical center of the lens on the image quality, so using this brightness compensation data to compensate the first image can effectively avoid the flickering and dark areas of water ripples caused by the shift of the physical center of the lens. The problem of flickering light and dark in the four corners.

Description of the drawings

Figure 1 is one of the flow charts of the image compensation method provided by the embodiment of the present application;

Figure 2 is the second flow chart of the image compensation method provided by the embodiment of the present application;

Figure 3 is the third flow chart of the image compensation method provided by the embodiment of the present application;

Figure 4 is a schematic diagram of image compensation using image compensation data in the image compensation method provided by the embodiment of the present application;

Figure 5 is the fourth flowchart of the image compensation method provided by the embodiment of the present application;

Figure 6 is a schematic diagram of obtaining color compensation data in the image compensation method provided by the embodiment of the present application;

Figure 7 is the fifth flow chart of the image compensation method provided by the embodiment of the present application;

Figure 8 is a schematic diagram of determining brightness compensation data in the image compensation method provided by the embodiment of the present application;

Figure 9 is a schematic diagram of an image compensation device provided by an embodiment of the present application;

Figure 10 is a schematic diagram of an electronic device provided by an embodiment of the present application;

Figure 11 is a hardware schematic diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art fall within the scope of protection of this application.

The terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. It is to be understood that the figures so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in orders other than those illustrated or described herein, and that "first," "second," etc. are distinguished Objects are usually of one type, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and/or" in the description and claims indicates at least one of the connected objects, and the character "/" generally indicates that the related objects are in an "or" relationship.

The terms "at least one (item)", "at least one", etc. in the description and claims of this application refer to any one, any two or a combination of more than two of the included objects. For example, at least one (item) of a, b, c can represent: "a", "b", "c", "a and b", "a and c", "b and c" and "a , b and c", where a, b, c can be single or multiple. Similarly, "at least two (items)" refers to two or more than two, and its meaning is similar to "at least one (item)".

The image compensation method, device, electronic device and readable storage medium provided by the embodiments of the present application will be described in detail through specific embodiments and application scenarios with reference to the accompanying drawings.

The function of Optical Image Stabilizer (OIS) is to correct the optical center shift through the floating lens of the lens. The principle of OIS is to detect the slight shake of the lens through the gyroscope in the lens, and then transmit the detected signal to the microprocessor. After receiving the signal, the microprocessor calculates the amount of displacement that needs to be compensated, and compensates the lens through the compensation lens. The group compensates based on the direction and displacement of the lens, thereby effectively reducing image blur and screen shake caused by camera shake. The anti-shake effect of this anti-shake technology is relatively obvious, but the requirements for lens design and manufacturing are relatively high, and the cost is relatively high. Normally, turning on the OIS anti-shake function of an electronic device can reduce the shutter speed by 2-3 steps, thereby reducing the probability of users collecting blurry images when shooting handheld. Especially for telephoto lenses, even a slight shake will affect the imaging quality when the zoom is large. The use of OIS anti-shake function can significantly improve the imaging quality, thus greatly improving the user's shooting experience.

In addition, after the electronic device collects an image, it can compensate the brightness and color of the image through lens shading (i.e. Lens Shading) compensation data to solve the problem of shadows around the lens caused by uneven optical refraction of the lens. Further improve the imaging quality of electronic equipment. The lens shading compensation data can be divided into two parts: brightness compensation part (i.e. Luma Shading) and color compensation part (i.e. Color Shading); among them, the data of the brightness compensation part is usually the compensation calibrated under the fixed light source in the laboratory. The data in the brightness compensation part can indicate the brightness attenuation law of the lens; the data in the color compensation part is usually lens shading correction statistical data (Lens Shading) based on fixed specifications (for example, 32×24) during image processing. CorrectionStats, LSC Stats) are calculated in real time. The data in the color compensation part can indicate the color attenuation law of the lens. After the electronic device obtains the above two parts of data, it can finally form four compensation tables for the four channels of R/Gr/Gb/B through the two parts of data to compensate for the collected images.

Specifically, the process of the electronic device compensating the image through the lens shading compensation data can roughly include two modules:

One is the calculation module, which can use the LSC Stats of each frame of image and the data of the calibrated brightness compensation part as input. The final output result is four compensation tables for the four channels of R/Gr/Gb/B (i.e. meshtable); corresponding to the currently most commonly used image ratio of 4:3, each compensation table usually includes 17×13 compensation data, and then the entire image to be compensated is evenly divided into 17×13 according to the rules of evenly dividing rectangular images into blocks. area, corresponding to 17×13 compensation data according to the area location. For example, assuming that the image size of an image is 4096×3072, in order to make each area of the image evenly divided include an integer number of pixels, the center alignment rule can be used, leaving 8 pixels on each side in the width direction. , leaving 2 pixels on each side in the height direction, and aligning the center of the image with the pixels participating in the compensation of 4080×3068; then the image can be divided into 17×13 areas, and each area is compensated with a gain value (usually A floating point number between 1x and 8x) corresponds to 240×236 pixels, and the remaining pixel compensation gain value is equal to the gain value of the nearest area. In addition, the status of the LSC Stats and the brightness compensation part data in the single-channel compensation table is a multiplication rule, and once the calculation module outputs the two, they cannot be separated.

The other is a compensation module, which can compensate the compensation data in the four compensation tables calculated by the calculation module to the collected images.

The process of the electronic device calibrating the data of the brightness compensation part will be described in detail below.

At present, the process of electronic equipment calibrating the data of the above-mentioned brightness compensation part is as follows:

1. Lock the camera position that needs to be calibrated;

2. Open the raw (original image file) acquisition command and collect a raw image. The raw image is an image obtained only after correction by the Black Level Correction (BLS) module and contains four channels R/ Gr/Gb/B, four channels are counted separately;

3. Based on the center alignment rules, the remaining pixels on both sides of the collected raw image are evenly divided into 17×13 areas, each area corresponding to 240×236 pixels;

4. Calculate the average brightness of each area;

5. Based on the brightness of the central area, the gain value magnification required to averagely compensate the brightness of each surrounding area to the brightness of the central area is the calibration data.

However, the above-mentioned brightness compensation data is calibrated when the physical center of the lens (i.e. optical center, also called the optical axis) of the electronic device is aligned with the physical center of the image sensor. The compensation method is: based on the image sensor's physical center. The physical center is the compensation center, which performs fixed position compensation. In the case of electronic equipment without OIS anti-shake function, since the camera and image sensor in the electronic equipment are relatively fixed, the brightness attenuation law of the fixed camera position calibrated in the laboratory is basically the same as the actual brightness attenuation law when collecting images. Consistent, so that the predetermined effect can be achieved by compensating the image with the data in the brightness compensation part. However, when an electronic device has an OIS anti-shake function, the camera in the electronic device floats relative to the image sensor. During the process of collecting images, the jitter of the electronic equipment will cause the physical center of the camera to shift, and this shift will cause the fixed brightness compensation data to not match the actual brightness attenuation law of the camera, causing the center of the picture to appear. The problem of flickering in the alternating light and dark area of the annular water ripple in the center and the flickering of light and dark in the four corners of the picture, especially during the video collection process, will be recorded together, seriously affecting the imaging effect and video quality.

In order to solve the above problems, embodiments of the present application provide an image compensation method, device, electronic device and readable storage medium. The image compensation method provided by the embodiment of the present application can be applied to images collected by electronic devices with OIS anti-shake function. In scenes where lens shading compensation is performed.

For example, an electronic device with OIS anti-shake function can obtain the brightness compensation data of image 1 based on the physical offset and calibration compensation data when the camera module collects image 1. The physical offset is in the calibration compensation data. Within the physical offset range of the corresponding lens; and the image data of image 1 can be compensated based on the brightness compensation data.

In this way, since the brightness compensation data used by the electronic device to compensate the image data of image 1 is obtained based on the actual physical offset and calibration compensation data when the image 1 is collected, the brightness compensation data can effectively avoid lens failure. The influence of the physical center shift on the imaging quality of the image 1, so that the brightness compensation data is used to compensate the image 1, which can effectively avoid the flickering of the dark areas of water ripples and the flickering of light and dark in the four corners caused by the shift of the physical center of the lens. Question, improve the imaging effect of image 1.

The image compensation method provided by the embodiment of the present application is executed by an electronic device. The electronic device includes a camera module, and the camera module includes a lens. Figure 1 shows a flow chart of an image compensation method provided by an embodiment of the present application. As shown in Figure 1, the image compensation method provided by the embodiment of the present application may include the following steps 101 and 102.

Step 101: The electronic device obtains brightness compensation data of the first image based on the first physical offset and calibration compensation data when the camera module collects the first image.

Wherein, the first physical offset amount is within the physical offset range of the lens corresponding to the calibration compensation data.

In some embodiments, the above-mentioned lens may be a traditional optical lens, an infrared lens or a Time of Flight (TOF) lens in an electronic device, etc.

For example, assuming that the above-mentioned lens is the above-mentioned traditional optical lens, the lens may be a telephoto lens, a short-focus lens, a zoom lens, etc.

In some embodiments, the first image may be one video frame among multiple continuously collected video frames.

In some embodiments, the first physical offset refers to the number of pixels displaced in the up, down, left and right directions or the number of angles deflected from the posture of the camera module when collecting the first image relative to the posture at the initial moment.

It should be noted that the up, down, left and right in the embodiments of the present application are all illustrated by taking the screen of the electronic device facing the user as an example.

In some embodiments, the initial moment posture may be: the posture of the lens when the physical center of the lens is aligned with the physical center of the image sensor in the electronic device.

In some embodiments, the above-mentioned posture is the specific position of the above-mentioned lens in the world coordinate system, including the amount of rotation and translation on the X-axis, the amount of rotation and translation on the Y-axis, and the amount of rotation and translation on the Z-axis. quantity.

In some embodiments, the above-mentioned first physical offset may be obtained by the OIS module in the electronic device.

In some embodiments, the physical offset range can be determined by the camera module. Lenses in different camera modules can have different physical offset ranges. The physical offset range indicates the anti-shake range of the camera module.

In some embodiments, the brightness compensation data of the first image may be used to perform brightness compensation on the first image.

In some embodiments, the brightness compensation data of the first image may be data in a matrix, and one data in the matrix is used to perform brightness compensation on a block in the first image.

In some embodiments, the above-mentioned calibration compensation data can be used to perform brightness compensation on the images collected by the above-mentioned camera module.

In some embodiments, the calibration compensation data may be data in a matrix, and the number of data in the matrix is greater than the number of data in the matrix corresponding to the brightness compensation data of the first image.

In some embodiments, the brightness compensation data of the first image may be the data in the calibration compensation data.

The specific method for the electronic device to obtain the brightness compensation data of the above-mentioned first image will be described in detail below.

In some embodiments, as shown in FIG. 2 with reference to FIG. 1 , the above step 101 can be implemented through the following steps 101a and 101b.

Step 101a: The electronic device determines the first data range based on the first physical offset.

In some embodiments, the electronic device may determine the offset direction of the pixels in the first image based on the first physical offset, and then determine the first data range based on the offset direction.

For example, assuming that the above calibration compensation data is the data in a 19×15 matrix (hereinafter referred to as matrix 1), then:

If the offset direction of the pixels in the first image determined by the electronic device based on the first physical offset is upper left, and the first row and first column to the left and above the lens do not participate in imaging, then The top row of data and the leftmost column of data in the above matrix 1 are not involved in the calculation, and the bottom row of data and the rightmost column of data in the matrix 1 will be more involved in the calculation; thus the electronic device can determine the above-mentioned third A data range is (2:19, 2:15), indicating the matrix data from the 2nd column to the 19th column, and the matrix data from the 2nd row to the 15th row; if the electronic device determines based on the above-mentioned first physical offset The offset direction of the pixels in the above-mentioned first image is to the lower right. At this time, the rightmost column and the bottommost row of the above-mentioned lens do not participate in imaging, then the bottommost row of data and the rightmost column of data in the above-mentioned matrix 1 does not participate in the calculation, and at the same time, the top row of data and the leftmost column of data in the matrix 1 will be included in the calculation; thus the electronic device can determine that the above-mentioned first data range is (1:18, 1:14), which means that the first data range is (1:18, 1:14). Matrix data from column 1 to column 18, and matrix data from rows 1 to 14;

If the offset direction of the pixels in the first image determined by the electronic device based on the first physical offset is to the left, and the column to the left and the row above the lens do not participate in imaging, then the topmost pixel in the matrix 1 One row of data and the leftmost column of data will not participate in the calculation, and at the same time, the rightmost column of data in the matrix 1 will be included in the calculation; thus the electronic device can determine that the above-mentioned first data range is (2:19, 2:14 ), represents the matrix data from column 2 to column 19, and the matrix data from row 2 to row 14;

If the offset direction of the pixels in the first image determined by the electronic device based on the first physical offset is to the right, and the column to the right of the lens and the first row above the lens do not participate in imaging, then in the matrix 1 The top row of data and the rightmost column of data are not involved in the calculation, and at the same time, the bottom row of data in the matrix 1 will be included in the calculation; thus the electronic device can determine that the above-mentioned first data range is (2:18, 2: 15), indicating the matrix data from column 2 to column 18, and the matrix data from row 2 to row 15;

In this way, the determined first data range can correspond to the effective imaging area of the lens on the image sensor, so that the data in the first data range can match the light attenuation law after the lens is shifted.

Step 101b: The electronic device determines the brightness compensation data corresponding to the first data range from the calibration compensation data.

In some embodiments, after determining the first data range, the electronic device may determine the data corresponding to the first data range from the calibration compensation data, and use the determined data as the brightness compensation data of the first image.

In some embodiments, since the electronic device determines the above-mentioned brightness compensation data from the above-mentioned calibration compensation data based on the data range determined by the above-mentioned first physical offset, the determined brightness compensation data can be compared with the above-mentioned The brightness attenuation law changed by the lens due to the offset is matched, so that the brightness compensation data can perform image compensation on the images collected by the lens under different offsets.

Step 102: The electronic device compensates the image data of the first image based on the brightness compensation data.

In some embodiments, after determining the brightness compensation data, the electronic device may send the brightness compensation data to a compensation module in the electronic device, and use the compensation module to compensate the image data of the first image to improve the brightness compensation data. The imaging quality of the first image.

In the image compensation method provided by the embodiment of the present application, the brightness compensation data used by the electronic device when compensating the image data of the first image collected is based on the actual physical offset and calibration when the first image is collected. The brightness compensation data is obtained by compensating the data. Therefore, the brightness compensation data can effectively avoid the impact of the physical center shift of the lens on the image quality. Therefore, the brightness compensation data is used to compensate the first image, which can effectively avoid the impact of the lens physical center shift. The problem of the flickering of dark areas of water ripples and the flickering of light and dark in the four corners.

In some embodiments, the electronic device can offset the above-mentioned lenses in different directions by maximum physical offsets, and then perform data calibration in the different directions, and then determine the above-mentioned calibration compensation data.

The specific method for the electronic device to obtain the above calibration compensation data will be described in detail below.

In some embodiments, the above-mentioned camera module may further include an image sensor. For example, in conjunction with Figure 1, as shown in Figure 3, before the above-mentioned step 101, the image compensation method provided by the embodiment of the present application may also include the following steps 103 to 106.

Step 103: The electronic device controls the physical center of the lens to be aligned with the physical center of the image sensor, and performs first calibration to obtain first calibration data.

In some embodiments, the first calibration process is as follows: after aligning the physical center of the lens with the physical center of the image sensor, the electronic device starts a raw image acquisition command and collects a raw image; then the electronic device Based on the center alignment rules, the free pixels on both sides of the collected raw image are evenly divided into 17×13 areas; then the electronic device calculates the average brightness of each area, and based on the brightness of the central area, divides each surrounding area The gain value magnification required to averagely compensate the brightness of each area to the brightness of the central area.

In some embodiments, the data specification of the first calibration data may be 17×13.

Step 104: The electronic device controls the lens to shift a second physical shift amount in the first direction, and performs a second calibration to obtain second calibration data.

Wherein, the second physical offset is the maximum physical offset within the physical offset range.

In some embodiments, the first direction may be one of two diagonal directions of the image sensor.

In some embodiments, the second calibration process is as follows: after the electronic device shifts the lens to the first direction by the second physical offset, the electronic device starts a raw image acquisition command and collects a raw image; then Based on the center alignment rule, the electronic device divides the free pixels on both sides of the collected raw image into 17×13 areas evenly; then the electronic device calculates the average brightness of each area, and takes the brightness of the central area as the standard. The gain value magnification required to averagely compensate the brightness of each surrounding area to the brightness of the central area.

In some embodiments, the data specification of the second calibration data may be 17×13.

Step 105: The electronic device controls the lens to shift the second physical shift amount in the second direction, and performs a third calibration to obtain third calibration data.

Wherein, the above-mentioned first direction and the above-mentioned second direction are respectively two opposite directions on the same diagonal line of the above-mentioned image sensor.

For example, the diagonal line of the above image sensor includes diagonal line 1 and diagonal line 2, and diagonal line 1 is the diagonal line between the upper left corner and the lower right corner, and diagonal line 2 is the diagonal line between the upper right corner and the lower left corner. diagonal line; then the above-mentioned first direction can be the direction toward the upper left corner on the diagonal line 1, and the above-mentioned second direction can be the direction toward the lower right corner on the diagonal line 1; or, the first direction can be the direction toward the pair of diagonal lines 1. The second direction may be the direction on the diagonal line 2 toward the upper right corner, and the second direction may be the direction on the diagonal line 2 toward the lower left corner.

In some embodiments, the third calibration process is as follows: after the electronic device shifts the lens to the second direction by the second physical offset, the electronic device starts a raw image acquisition command and collects a raw image; then Based on the center alignment rule, the electronic device divides the free pixels on both sides of the collected raw image into 17×13 areas evenly; then the electronic device calculates the average brightness of each area, and takes the brightness of the central area as the standard. The gain value magnification required to averagely compensate the brightness of each surrounding area to the brightness of the central area.

In some embodiments, the data specification of the third calibration data may be 17×13.

Step 106: The electronic device determines calibration compensation data based on the first calibration data, the second calibration data and the third calibration data.

In some embodiments, the data specification of the above-mentioned calibration compensation data may be 19×15.

In some embodiments, the above-mentioned calibration compensation data may be data in a 19×15 matrix.

In some embodiments, the electronic device may first determine the 19×15 target data basetable-new based on the above-mentioned first calibration data, the above-mentioned second calibration data, and the above-mentioned third calibration data, which is expressed as follows:

basetable-new(2:18,2:14,m)＝basetable-1(1:17,1:13,m)

basetable-new(19:19,2:14,m)＝basetable-2(17:17,1:13,m)

basetable-new(2:18,15:15,m)＝basetable-2(1:17,13:13,m)

basetable-new(2:18,1:1,m)＝basetable-3(1:17,1:1,m)

basetable-new(1:1,2:14,m)=basetable-3(1:1,1:13,m);

Among them, basetable-1 is the above-mentioned first calibration data, basetable-2 is the above-mentioned second calibration data, basetable-3 is the above-mentioned third calibration data, and m is the channel.

In some embodiments, after the electronic device determines the above-mentioned calibration compensation data basetable-new, it can use a quadratic function to perform fitting based on the calibration compensation data basetable-new to determine the four corner point data in the above-mentioned 19×15 matrix. .

In some embodiments, for each corner point data in the above four corner point data, there is an adjacent row of data and a column of data, and the adjacent row of data and column of data of each corner point data are processed twice. Function fitting, the quadratic function is: f(x)=ax^2+bx+c, get two predicted values corresponding to each corner point data, take the average of the two predicted values to get the corresponding corner point The value of the data.

For example, as shown in Figure 4, taking the b1 corner point data among the above four corner point data as an example, the electronic device can use two sets of data A1 and A4 and the least squares method to perform quadratic function fitting to obtain two The optimal quadratic function; then predict the value of the b1 corner point data to obtain two predicted values. The average of the two predicted values is the value of the b1 corner point data.

It should be noted that the data specifications of each of the above data are related to the number of areas divided into the image during the calibration process. In the embodiments of this application, the area divided into the image is 17×13 as an example. In actual implementation, The area divided into the image can be any area, which is not limited in the embodiment of this application.

In some embodiments, the electronic device can perform data processing when the physical center of the lens is aligned with the physical center of the image sensor, and when the lens is displaced by a maximum physical offset in the first direction and the second direction. Calibration, and determines the above-mentioned calibration compensation data based on the calibration data obtained under different circumstances, so that the determined calibration compensation data can match any offset of the lens and expand the compensation range of the calibration compensation data.

In some embodiments, as shown in Figure 5 in conjunction with Figure 1, before the above step 102, the image compensation method provided by the embodiment of the present application can also include the following steps 107 and 108, and the above step 102 can be specifically performed by The following step 102a is implemented.

It should be noted that Figure 5 only illustrates that the above-mentioned step 107 and step 108 are executed after the above-mentioned step 101. In actual implementation, this step 107 and step 108 can also be executed before this step 101. Embodiments of the present application Not limited.

Step 107: The electronic device converts the first physical offset into a pixel offset of the pixels in the first image.

In some embodiments, the above-mentioned pixel offset may include: a pixel offset in the X-axis direction (including the positive direction or the negative direction), and the pixel offset in the Y-axis direction (including the positive direction or the negative direction). Shift amount.

In some embodiments, the electronic device may convert the first physical offset into a pixel offset of the pixels in the first image through the following manner 1 or 2.

method one

In some embodiments, the electronic device may directly map the first physical offset to the pixel offset.

For example, if the above-mentioned first physical offset is a millimeter to the left, and a millimeter is the length of b pixels, then the above-mentioned pixel offset converted by the electronic device is: b pixels in the X-axis direction Length; where a and b are both positive integers.

Method 2

In some embodiments, the electronic device may map the first physical offset into the pixel offset at a first ratio.

In some embodiments, the above-mentioned first ratio may be preset by the system, or may be arbitrarily set by the user according to actual usage requirements.

For example, if the above-mentioned first physical offset is an upward offset of c mm, and c mm is the length of d pixels, and the above-mentioned first ratio is 1/2, then the above-mentioned pixel offset converted by the electronic device is: The length of d/2 pixels in the Y-axis direction; where c and d are both positive integers.

Step 108: When the pixel offset is greater than or equal to the first threshold, the electronic device obtains the color compensation data of the first image according to the pixel offset and the offset direction of the pixels in the first image.

In some embodiments, the above-mentioned color compensation data may be used to perform color compensation on the above-mentioned first image.

In some embodiments, the color compensation data may be data in a matrix, and one data in the matrix is used to perform color compensation on a block in the first image.

In some embodiments, the color compensation data may be LSC Stats of the first image.

In some embodiments, the electronic device can obtain the above-mentioned LSC Stats through the LSC Stats extraction module.

In some embodiments, a pixel offset with a pixel offset greater than or equal to the above-mentioned first threshold may be called an effective offset.

In some embodiments, the above-mentioned first threshold may be preset by the system, or may be arbitrarily set by the user according to actual usage requirements.

For example, taking the above-mentioned first threshold as arbitrarily set by the user according to actual usage requirements, the first threshold may be set to 80 pixels, 90 pixels, or 95 pixels, etc.

In some embodiments, if the above-mentioned first threshold is preset by the system, the electronic device can be set according to the anti-shake maximum offset of the image sensor in the electronic device; the larger the anti-shake maximum offset, the electronic device will set The first threshold is also larger.

In some embodiments, the electronic device can determine the above-mentioned first threshold N through the following formula (1):

N＝min(x-offset,y-offset)/5; (1)

Among them, x-offset is the maximum anti-shake offset of the above-mentioned image sensor in the X-axis direction, and y-offset is the maximum anti-shake offset of the image sensor in the Y-axis direction.

It should be noted that if the above-mentioned pixel offset is less than the above-mentioned first threshold, it can be considered that the above-mentioned physical offset is small. At this time, the light attenuation law of the above-mentioned lens will not change significantly, and there is no need to modify the above-mentioned first threshold. image to compensate.

In some embodiments, before acquiring the color compensation data based on the pixel offset and the offset direction of the pixel, the electronic device may first determine the offset direction based on the pixel offset.

For example, if the pixel offset of the above pixel in the X-axis direction is 0, and there is a pixel offset in the positive direction of the Y-axis, the offset direction of the pixel is upward; if the pixel is in the X-axis direction The pixel offset on is 0, and there is a pixel offset in the negative direction of the Y-axis, then the offset direction of the pixel is downward; if the pixel offset of the pixel in the Y-axis direction is 0, and If there is a pixel offset in the negative direction of the X-axis, the offset direction of the pixel is to the left; if the pixel offset of the pixel in the Y-axis direction is 0, and there is a pixel offset in the positive direction of the shift amount, the offset direction of the pixel is to the right.

For another example, if the above pixel has a pixel offset in the positive direction of the X-axis and the positive direction of the Y-axis, the offset direction of the pixel is the upper right; If there is a pixel offset in the negative direction, the offset direction of the pixel is the lower left; if the pixel has a pixel offset in the positive direction of the X-axis and the negative direction of the Y-axis, the offset direction of the pixel is the lower right ; If the pixel has a pixel offset in the negative direction of the X-axis and the positive direction of the Y-axis, the offset direction of the pixel is the upper left.

In some embodiments, after obtaining the offset direction, the electronic device may first determine the data specification of the color compensation data (for example, 33×25 or 34×26, etc.) based on the pixel offset amount and the offset direction. , and then obtain the above color compensation data according to the data specification.

For example, taking the above-mentioned lens shifting to the upper left as an example, as shown in Figure 6, the physical center of the lens shifts from position a to a1 on the upper left side of position a. Then the electronic device can first pass through the electronic device. The OIS module obtains the physical offset of the lens, and then directly maps the physical offset to the pixel offset of the pixels in the image captured by the lens, and determines the offset of the pixels in the image based on the pixel offset. The moving direction is the upper left; then the electronic device changes the data specification for extracting the above color compensation data from the initial specification of 32×24 to 33×25 based on the pixel offset and the offset direction, so that the image 33×25 can be obtained color compensation data. Among them, the solid line blocks in Figure 6 are the 32×24 size color compensation data blocks when the physical center of the lens does not shift; the dotted line blocks and the shaded part in Figure 6 are the physical center deviation of the lens. When moving to this position a1, the color compensation data of 32×25 size is divided into blocks.

It can be seen that after the above-mentioned lens is shifted to the upper left, the above-mentioned color compensation data obtained has one more row and one column, so that it can match the shift of the lens to adapt to the color attenuation law of the lens at this time, so that it can Improve the compensation effect of the color compensation data.

Step 102a: The electronic device compensates the image data of the first image according to the brightness compensation data and the color compensation data.

In some embodiments, after acquiring the color compensation data and the brightness compensation data, the electronic device may send the color compensation data and the brightness compensation data to a computing module in the electronic device to determine the image compensation of the first image. data, and compensate the image data of the first image through the image compensation data.

In some embodiments, because the electronic device can obtain color compensation data that matches the first physical offset of the lens based on the pixel offset and offset direction of the pixel in the first image, and based on the above brightness compensation data and the The color compensation data compensates the image data of the first image, so the compensation effect on the first image can be further improved.

In some embodiments, as shown in FIG. 7 in conjunction with FIG. 5 , the above step 102a may be implemented through the following steps 102a1 to 102a3.

Step 102a1: The electronic device calculates the pixel mean value corresponding to each image area in the first image based on the corresponding data of each channel of the first image in the color compensation data.

In some embodiments, the electronic device can first determine the brightness values of the four channels of R, Gr, Gb, and B from the above-mentioned color compensation data, and then calculate each of the four channels in the above-mentioned image areas. The average brightness of the channel is used to obtain the average pixel value corresponding to each of the above image areas.

Step 102a2: The electronic device determines the image compensation data corresponding to each image area based on the pixel mean value and brightness compensation data of each image area.

In some embodiments, the electronic device can perform interpolation calculations on the above-mentioned brightness compensation data to expand the amount of the brightness compensation data to the same number as the image areas in the above-mentioned first image, so as to facilitate the image processing of the first image. data for compensation.

For example, assuming that the number of image areas in the first image is 32×25 and the number of brightness compensation data is 19×15, then the electronic device can perform interpolation calculations on the 19×15 brightness compensation data, 32×25 new brightness compensation data are obtained, thereby achieving data expansion.

It should be noted that interpolation calculation can interpolate a continuous function on the basis of discrete data, so that this continuous curve passes through all given discrete data points; interpolation calculation is an important method of discrete function approximation, and interpolation calculation can be used to pass functions in The value situation at a limited number of points can be used to estimate the approximate value of the function at other points.

In some embodiments, the above interpolation calculation is performed by an interpolation method, which may be Lagrangian interpolation, Newton interpolation, Hermitian interpolation, piecewise interpolation or spline interpolation, etc.

It should be noted that the above-mentioned interpolation method can also be called interpolation method or straight-line interpolation method. Its principle is to calculate the function value of the unknown function f(x) at several points in a certain interval and f (x) are equal to a specific function to approximate the original function f(x), and then this specific function can be used to calculate the approximate value of the original function f(x) at other points in the interval. The interpolation method can be divided into linear interpolation, nonlinear interpolation, etc. according to the properties of the specific function; according to the number of arguments (independent variables), it can be divided into single interpolation, double interpolation, triple interpolation, etc.

For the specific description of the above interpolation calculation, you can refer to the relevant description in the related art. In order to avoid repetition, it will not be described again here.

In some embodiments, for each image area in the first image, the electronic device can calculate two ratios (i.e., R/Gr and B/Gb) corresponding to an image area based on the mean pixel value of the image area, To obtain the two ratios corresponding to each image area.

For example, the electronic device can input the above color compensation data and the above brightness compensation data into a dynamic color shading correction algorithm, and multiply the color compensation data and the brightness compensation data through the algorithm to calculate the above corresponding image area. The two ratios, namely R/Gr, B/Gb; then the two ratios corresponding to each image area can be fine-tuned according to a certain rule algorithm, and the adjustment scale coefficient is multiplied by the brightness compensation data to obtain each image. Image compensation data corresponding to the area.

Step 102a3: The electronic device compensates the image data of the first image based on the image compensation data corresponding to each image area.

In some embodiments, the above image compensation data can fine-tune the R value and B value in each of the above image areas to achieve accurate compensation of the image data of the above first image.

In some embodiments, after the electronic device determines the image compensation data, the image compensation data can be sent to a compensation module in the electronic device, and the image data (including brightness data and color) of the first image can be processed by the compensation module. data) to improve the imaging quality of the first image.

In some embodiments, since the electronic device can fine-tune the image data of the first image from the channel level based on the image compensation data determined by the pixel average value of each image area and the brightness compensation data, the image data of the first image can be further improved. Compensation effect of image data of an image.

In some embodiments, after determining the above image compensation data, the electronic device can perform linear interpolation calculation on the image compensation data, scale down the image compensation data to the same data specification as the above brightness compensation data, and store it as the first A compensation table is used to compensate the image data of the collected image when the camera module is offset again by the first physical offset amount.

It should be noted that linear interpolation refers to an interpolation method in which the interpolation function is a linear polynomial, and its interpolation error at the interpolation node is zero; linear interpolation is simpler and more convenient than other interpolation methods (such as parabolic interpolation); linear interpolation Interpolation can be used to approximately replace the original function, and can also be used to calculate values that are not in the table during table lookup. The basic idea of linear interpolation is to insert some new data points between the original data points, so that the distance between the data points becomes more uniform; specifically, the new data points can be determined by calculating the slope between adjacent data points. The position and value of the inserted data point. Linear interpolation can scale down data to any size while maintaining data trends and patterns.

For a specific description of the above linear interpolation calculation, please refer to the relevant description in the related art. To avoid repetition, it will not be described again here.

The image compensation method provided by the embodiment of the present application is exemplarily described below.

For example, taking the above-mentioned lens shifting to the upper left as an example, assume that the number of image areas in the above-mentioned first image is 17×13; as shown in Figure 8, the physical center of the lens shifts from position b to b1 in the upper left corner of position b, then the electronic device can first obtain the physical offset of the lens through the OIS module in the electronic device, and determine the brightness compensation data of the first image from the above-mentioned calibration compensation data based on the physical offset. ;Then the physical offset is directly mapped to the pixel offset of the pixel in the first image, and based on the pixel offset, it is determined that the offset direction of the pixel in the first image is the upper left, and based on the pixel The offset amount and the offset direction determine the color compensation data of the first image; and based on the brightness compensation data and the color compensation data, the image data of the first image is compensated.

Each of the above method embodiments, or various possible implementations in each method embodiment, can be executed individually, or, provided there is no contradiction, they can also be executed in combination with each other. The details can be determined according to actual usage requirements. This application implements There is no restriction on this.

For the image compensation method provided by the embodiments of the present application, the execution subject may be an image compensation device. In the embodiment of the present application, the image compensation method performed by the image compensation device is used as an example to illustrate the image compensation device provided by the embodiment of the present application.

As shown in FIG. 9 , this embodiment of the present application provides an image compensation device 90 . The image compensation device 90 includes a camera module. The camera module includes a lens. The image compensation device 90 may include an acquisition module 91 and a compensation module 92 .

Among them, the acquisition module 91 can be used to acquire the brightness compensation data of the first image based on the first physical offset and calibration compensation data when the camera module collects the first image. The first physical offset is in the The calibration compensation data corresponds to the physical offset range of the lens. The compensation module 92 may be configured to compensate the image data of the first image based on the brightness compensation data obtained by the acquisition module 91 .

In a possible implementation, the above camera module further includes an image sensor. The acquisition module 91 may also be used to control the physical movement of the lens before acquiring the brightness compensation data of the first image based on the first physical offset and the calibration compensation data when the camera module collects the first image. The center is aligned with the physical center of the image sensor, and the first calibration is performed to obtain the first calibration data; and the lens is controlled to offset the second physical offset in the first direction, and the second calibration is performed to obtain the second calibration data. ; And control the lens to offset the second physical offset in the second direction, and perform the third calibration to obtain the third calibration data; and based on the first calibration data, the second calibration data and the third calibration data , determine the calibration compensation data. Wherein, the second physical offset is the maximum physical offset within the above physical offset range, and the first direction and the second direction are respectively two opposite directions on the same diagonal line of the image sensor.

In a possible implementation, the acquisition module 91 may be configured to determine the first data range according to the first physical offset; and determine the brightness compensation corresponding to the first data range from the calibration compensation data. data.

In a possible implementation, the acquisition module 91 may also be configured to convert the first physical offset into the first physical offset before the compensation module 92 compensates the image data of the first image based on the brightness compensation data. The pixel offset of a pixel in an image; and when the pixel offset is greater than or equal to the first threshold, the first image is obtained based on the pixel offset and the offset direction of the pixel in the first image color compensation data. The compensation module 92 may be specifically configured to compensate the image data of the first image according to the brightness compensation data and the color compensation data obtained by the acquisition module 91 .

In one possible implementation, the compensation module 92 may be configured to calculate the pixel mean value corresponding to each image area in the first image based on the data corresponding to each channel of the first image in the color compensation data; and based on The pixel mean value of each image area and the brightness compensation data determine the image compensation data corresponding to each image area; and based on the image compensation data corresponding to each image area, the image data of the first image is compensated.

In the image compensation device provided by the embodiment of the present application, the brightness compensation data used by the image compensation device when compensating the image data of the first image collected is based on the actual physical offset when the first image is collected. And the calibration compensation data is obtained, so the brightness compensation data can effectively avoid the impact of the physical center shift of the lens on the image quality, so the brightness compensation data is used to compensate the first image, which can effectively avoid the physical center shift band of the lens. There are problems with the flickering of dark areas of water ripples and the flickering of light and dark in the four corners.

The image compensation device in the embodiment of the present application may be an electronic device or a component in the electronic device, such as an integrated circuit or chip. The electronic device may be a terminal or other devices other than the terminal. For example, the electronic device may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle-mounted electronic device, a mobile Internet device (MID), or augmented reality (AR)/virtual reality (VR) Devices, robots, wearable devices, ultra-mobile personal computers (UMPC), netbooks or personal digital assistants (PDA), etc., can also be servers, network attached storage (Network Attached Storage, NAS) , personal computer (PC), television (TV), teller machine or self-service machine, etc., the embodiments of this application are not specifically limited.

The image compensation device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiments of this application.

The image compensation device provided by the embodiments of the present application can implement each process implemented by the above method embodiments and achieve the same technical effect. To avoid repetition, details will not be described here.

As shown in Figure 10, the embodiment of the present application also provides an electronic device 100, including a processor 101 and a memory 102. The memory 102 stores programs or instructions that can be run on the processor 101. The program or instructions are When the processor 101 executes, the steps of the above image compensation method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, the details will not be described here.

It should be noted that the electronic devices in the embodiments of the present application include mobile electronic devices and non-mobile electronic devices.

Figure 11 is a schematic diagram of the hardware structure of an electronic device that implements an embodiment of the present application.

As shown in Figure 11, the electronic device 1000 includes but is not limited to: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, and processor 1010 and other components.

Those skilled in the art can understand that the electronic device 1000 may also include a power supply (such as a battery) that supplies power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby managing charging, discharging, and function through the power management system. Consumption management and other functions. The structure of the electronic device shown in Figure 11 does not constitute a limitation on the electronic device. The electronic device may include more or less components than shown in the figure, or combine certain components, or arrange different components, which will not be described again here. .

Among them, the electronic device 1000 includes a camera module, and the camera module includes a lens. The processor 1010 may be configured to obtain the brightness compensation data of the first image based on the first physical offset and calibration compensation data when the camera module collects the first image. The first physical offset is used in the calibration compensation. The data corresponds to the physical offset range of the lens. The processor 1010 may also be configured to compensate the image data of the first image based on the obtained brightness compensation data.

In a possible implementation, the above camera module further includes an image sensor. The processor 1010 may also be configured to control the physical movement of the lens before acquiring the brightness compensation data of the first image based on the first physical offset and the calibration compensation data when the camera module collects the first image. The center is aligned with the physical center of the image sensor, and the first calibration is performed to obtain the first calibration data; and the lens is controlled to offset the second physical offset in the first direction, and the second calibration is performed to obtain the second calibration data. ; And control the lens to offset the second physical offset in the second direction, and perform the third calibration to obtain the third calibration data; and based on the first calibration data, the second calibration data and the third calibration data , determine the calibration compensation data. Wherein, the second physical offset is the maximum physical offset within the above physical offset range, and the first direction and the second direction are respectively two opposite directions on the same diagonal line of the image sensor.

In a possible implementation, the processor 1010 may be configured to determine the first data range according to the first physical offset; and determine the brightness compensation corresponding to the first data range from the calibration compensation data. data.

In a possible implementation, the processor 1010 may also be configured to convert the first physical offset into the first image before compensating the image data of the first image based on the brightness compensation data. The pixel offset of the pixel; and when the pixel offset is greater than or equal to the first threshold, obtain the color compensation of the first image based on the pixel offset and the offset direction of the pixel in the first image data. The processor 1010 may be specifically configured to compensate the image data of the first image according to the brightness compensation data and the obtained color compensation data.

In a possible implementation, the processor 1010 may be configured to calculate the pixel mean value corresponding to each image area in the first image based on the data corresponding to each channel of the first image in the color compensation data; and based on The pixel mean value of each image area and the brightness compensation data determine the image compensation data corresponding to each image area; and based on the image compensation data corresponding to each image area, the image data of the first image is compensated.

In the electronic device provided by the embodiment of the present application, the brightness compensation data used by the electronic device when compensating the image data of the first image collected is based on the actual physical offset and calibration when the first image is collected. The brightness compensation data is obtained by compensating the data. Therefore, the brightness compensation data can effectively avoid the impact of the physical center shift of the lens on the image quality. Therefore, the brightness compensation data is used to compensate the first image, which can effectively avoid the impact of the lens physical center shift. The problem of the flickering of dark areas of water ripples and the flickering of light and dark in the four corners.

It should be understood that in the embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042. The graphics processor 10041 is used for recording data generated by an image capture device (such as Process the image data of still pictures or videos obtained by the camera). The display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072 . Touch panel 10071, also known as touch screen. The touch panel 10071 may include two parts: a touch detection device and a touch controller. Other input devices 10072 may include but are not limited to physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be described again here.

Memory 1009 may be used to store software programs as well as various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, Image playback function, etc.) etc. Additionally, memory 1009 may include volatile memory or nonvolatile memory, or memory 1009 may include both volatile and nonvolatile memory. Among them, the non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electrically removable memory. Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory. Volatile memory can be random access memory (Random Access Memory, RAM), static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synch link DRAM) , SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DRRAM). The memory 1009 in the embodiment of the present application includes, but is not limited to, these and any other suitable types of memory.

The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, where the application processor mainly handles operations related to the operating system, user interface, application programs, etc., Modem processors mainly process wireless communication signals, such as baseband processors. It can be understood that the above modem processor may not be integrated into the processor 1010.

Embodiments of the present application also provide a readable storage medium. Programs or instructions are stored on the readable storage medium. When the program or instructions are executed by a processor, each process of the above image compensation method embodiment is implemented, and the same can be achieved. The technical effects will not be repeated here to avoid repetition.

Wherein, the processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes computer readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disk or optical disk, etc.

An embodiment of the present application further provides a chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the above image compensation method embodiment. Each process can achieve the same technical effect. To avoid duplication, it will not be described again here.

It should be understood that the chips mentioned in the embodiments of this application may also be called system-on-chip, system-on-a-chip, system-on-a-chip or system-on-chip, etc.

Embodiments of the present application provide a computer program product. The program product is stored in a storage medium. The program product is executed by at least one processor to implement each process of the above image compensation method embodiment, and can achieve the same technical effect. , to avoid repetition, will not be repeated here.

It should be noted that, in this document, the terms "comprising", "comprises" or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or device that includes a series of elements not only includes those elements, It also includes other elements not expressly listed or inherent in the process, method, article or apparatus. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article or apparatus that includes that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions may be performed, for example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better. implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a computer software product that is essentially or contributes to the existing technology. The computer software product is stored in a storage medium (such as ROM/RAM, disk , optical disk), including several instructions to cause a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of this application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Inspired by this application, many forms can be made without departing from the purpose of this application and the scope protected by the claims, all of which fall within the protection of this application.

Claims (10)

1. An image compensation method, performed by an electronic device, the electronic device including a camera module including a lens, the method comprising:

acquiring brightness compensation data of a first image based on a first physical offset and calibration compensation data when the camera module acquires the first image, wherein the first physical offset is in a physical offset range of the lens corresponding to the calibration compensation data;

and compensating the image data of the first image based on the brightness compensation data.

2. The method of claim 1, wherein the camera module further comprises an image sensor;

the method further comprises the steps of:

controlling the physical center of the lens to be aligned with the physical center of the image sensor, and performing first calibration to obtain first calibration data;

controlling the lens to deviate a second physical deviation amount to the first direction, and performing second calibration to obtain second calibration data;

controlling the lens to shift the second physical offset to a second direction, and performing third calibration to obtain third calibration data;

Determining the calibration compensation data according to the first calibration data, the second calibration data and the third calibration data;

wherein the second physical offset is the maximum physical offset in the physical offset range, and the first direction and the second direction are respectively: opposite directions on the same diagonal of the image sensor.

3. The method of claim 1, wherein the obtaining the brightness compensation data of the first image based on the first physical offset and calibration compensation data when the camera module collects the first image comprises:

determining a first data range according to the first physical offset;

and determining the brightness compensation data corresponding to the first data range from the calibration compensation data.

4. A method according to any one of claims 1 to 3, wherein before compensating the image data of the first image based on the brightness compensation data, the method further comprises:

converting the first physical offset into a pixel offset for a pixel in the first image;

acquiring color compensation data of the first image according to the pixel offset and the offset direction of pixels in the first image under the condition that the pixel offset is greater than or equal to a first threshold;

The compensating the image data of the first image based on the brightness compensation data includes:

and compensating the image data of the first image according to the brightness compensation data and the color compensation data.

5. The method of claim 4, wherein compensating the image data of the first image based on the brightness compensation data and the color compensation data comprises:

calculating pixel mean values corresponding to all image areas in the first image based on data corresponding to all channels of the first image in the color compensation data;

determining image compensation data corresponding to each image area based on the pixel mean value and the brightness compensation data of each image area;

and compensating the image data of the first image based on the image compensation data corresponding to each image area.

6. An image compensation device, the device comprises a camera module, the camera module comprises a lens, and the device is characterized by comprising an acquisition module and a compensation module;

the acquisition module is used for acquiring brightness compensation data of a first image based on a first physical offset and calibration compensation data when the camera module acquires the first image, wherein the first physical offset is in a physical offset range of the lens corresponding to the calibration compensation data;

The compensation module is used for compensating the image data of the first image based on the brightness compensation data acquired by the acquisition module.

7. The apparatus of claim 6, wherein the camera module further comprises an image sensor;

the acquisition module is further configured to control a physical center of the lens to be aligned with a physical center of the image sensor and perform a first calibration to obtain first calibration data before acquiring brightness compensation data of the first image based on the first physical offset and the calibration compensation data when the camera module acquires the first image; the lens is controlled to deviate a second physical deviation amount towards the first direction, and second calibration is carried out to obtain second calibration data; the lens is controlled to deviate the second physical deviation amount towards the second direction, and third calibration is carried out to obtain third calibration data; determining the calibration compensation data according to the first calibration data, the second calibration data and the third calibration data;

wherein the second physical offset is the maximum physical offset in the physical offset range, and the first direction and the second direction are respectively: opposite directions on the same diagonal of the image sensor.

8. The apparatus according to claim 6 or 7, wherein,

the acquisition module is further configured to convert the first physical offset into a pixel offset of a pixel in the first image before the compensation module compensates the image data of the first image based on the brightness compensation data; under the condition that the pixel offset is larger than or equal to a first threshold value, acquiring color compensation data of the first image according to the pixel offset and the offset direction of pixels in the first image;

the compensation module is specifically configured to compensate the image data of the first image according to the brightness compensation data and the color compensation data acquired by the acquisition module.

9. An electronic device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the image compensation method of any one of claims 1-5.

10. A readable storage medium, characterized in that the readable storage medium has stored thereon a program or instructions which, when executed by a processor, implement the steps of the image compensation method according to any of claims 1-5.