CN114842072A - Image processing method, device and system - Google Patents

Abstract

The present application discloses an image processing method, device and system. Wherein, the method includes: acquiring the position information of the target image in the display screen when the target image is displayed on the display screen; determining the working position information of the image acquisition device when the image acquisition device performs image acquisition at at least one position, and acquiring the corresponding information of the working position information compensation coefficients to obtain a first set of compensation coefficients, wherein the image acquisition device is at least used to collect the display screen of the display screen; according to the position information of the target image in the display screen, the compensation weight of the target image in different shooting directions is determined, and based on the position information of the target image in the display screen The compensation weight and the first compensation coefficient set determine the target compensation coefficient; the target image is compensated according to the target compensation coefficient. The present application solves the technical problems in the related art that when modifying the configuration of the monitoring terminal, the process is complicated, the efficiency is low, and the flexibility of the entire monitoring system is affected.

Description

Image processing method, device and system

technical field

The present application relates to the field of virtual photography, and in particular, to an image processing method, device and system.

Background technique

The LED display screen is used as the display background in film and television related shooting. In the scene where the display screen is used to shoot multiple screens, the color inconsistency introduced by different shooting positions due to the deviation of the viewing angle can easily lead to shooting problems.

In the solution of the related art, the current position of the camera is obtained from the server, and the color cast image is obtained according to the LED at the current position, and the server renders the image to solve the color cast problem. Due to the problem of large image delay caused by the solution of the related art in the server-side processing, especially when the camera moves rapidly, the rendering speed increased by the server does not match the dynamic adjustment of the display screen, resulting in restricted shooting, or after rapid movement. The problem of chromatic aberration occurs, or the image rendering accuracy on the server side is low, resulting in the problem of improper image compensation.

For the above problems, no effective solution has been proposed yet.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image processing method, device, and system to at least solve the technical problems in the related art that modifying the configuration of a monitoring terminal is complicated and inefficient, and affects the flexibility of the entire monitoring system.

According to an aspect of the embodiments of the present application, an image processing method is provided, including: acquiring position information of the target image in the display screen when the target image is displayed on the display screen; determining that the image acquisition device performs image acquisition at at least one position The working position information at the time is obtained, and the compensation coefficient corresponding to the working position information is obtained to obtain the first set of compensation coefficients, wherein the image acquisition device is at least used to collect the display screen of the display screen; the target image is determined according to the position information of the target image in the display screen. Compensation weights of images in different shooting directions, and target compensation coefficients are determined based on the compensation weights and the first compensation coefficient set; the target image is compensated according to the target compensation coefficients.

Optionally, the position information of multiple calibration points in the image acquisition device is obtained, and the size information of the display screen is determined, wherein the positions of the calibration points corresponding to the multiple calibration points correspond to different shooting directions one-to-one, and the size information is: It is determined based on the number of pixels in the display screen; the target angle of view corresponding to the position information of the multiple calibration points is determined, wherein the target angle of view is the shooting direction of the image acquisition device and the plane where the display screen is located when the image acquisition device is in different working positions. The angle formed by the normal line; the difference information between the coordinate information and the size information corresponding to the position information of the target image in the display screen; the compensation weight is determined according to the difference information and the target viewing angle.

Optionally, a difference operation is performed between the size information and the coordinate information corresponding to the position information of the target image in the display screen, and the compensation weight is determined according to the operation result and the target viewing angle.

Optionally, the first set of compensation coefficients is a coefficient matrix, wherein each element in the coefficient matrix is a compensation coefficient corresponding to each calibration point position information in the plurality of calibration point position information; based on the Determining the target compensation coefficient based on the compensation weight and the first compensation coefficient set includes: determining a second coefficient set according to all the coefficients in the coefficient matrix and the compensation weight; and based on the average of all the coefficients in the second coefficient set The value determines the target compensation factor.

Optionally, the multiple calibration points corresponding to the multiple calibration point position information include at least calibration points distributed on two straight lines, and the two straight lines are perpendicular to each other.

Optionally, the compensation coefficient corresponding to the working position information is received from the external device; or, the compensation coefficient corresponding to the working position information is determined from a compensation coefficient set pre-stored by the receiving device.

Optionally, acquiring a sample image collected at a sampling point in the candidate sampling point set when the image acquisition device moves within the effective moving space; acquiring a quality evaluation index of the sample image; selecting from the candidate sampling point set according to the quality evaluation index The target sampling point, and the location of the target sampling point is used as the working position information.

According to another aspect of the embodiments of the present application, an image processing apparatus is further provided, including: an acquisition module for acquiring position information of the target image in the display screen when the target image is displayed on the display screen; a first determination module , which is used to determine the working position information when the image acquisition device performs image acquisition at different working positions, and obtains the compensation coefficient corresponding to the working position information, and obtains the first set of compensation coefficients; the second determination module is used for displaying on the display screen according to the target image. The position information in the device determines compensation weights in different shooting directions, and determines a target compensation coefficient compensation module based on the compensation weights and the first compensation coefficient set for compensating the target image according to the target compensation coefficients.

According to another aspect of the embodiments of the present application, an image processing system is further provided, including: an image source device, a sending device, and a receiving device, wherein: the image source device is configured to provide an original image and send the original image to a sending device. equipment; sending equipment, used for receiving the original image, and processing the target image to obtain the target image; receiving equipment, used for executing any one of the above image processing methods.

In the embodiment of the present application, the position information of the target image in the display screen when the target image is displayed on the display screen is firstly obtained; the working position information of the image acquisition device when the image is collected at at least one position is determined, and the working position information is obtained Corresponding compensation coefficients, a first set of compensation coefficients is obtained, wherein the image acquisition device is at least used to collect images corresponding to the display screen of the display screen; according to the position information of the target image in the display screen, it is determined that the target image is in different shooting directions and the target compensation coefficient is determined based on the compensation weight and the first compensation coefficient set; the target image is compensated according to the target compensation coefficient. Therefore, the technical problems of complicated process and low efficiency when modifying the configuration of the monitoring terminal in the related art and affecting the flexibility of the entire monitoring system are solved.

In the embodiment of the present application, an image processing method is provided, which includes: acquiring the position information of the target image in the display screen when the target image is displayed on the display screen; working position information, and obtain the compensation coefficient corresponding to the working position information, and obtain a first set of compensation coefficients, wherein the image acquisition device is at least used to collect the image corresponding to the display screen of the display screen; according to the position information of the target image in the display screen Compensation weights of the target image in different shooting directions are determined, and target compensation coefficients are determined based on the compensation weights and the first compensation coefficient set; the target image is compensated according to the target compensation coefficients. In the process of fast movement during the shooting process, the image rendering speed is matched with the dynamic adjustment of the display screen, and the image is rendered with high precision, so that the image can be properly compensated under the premise of reducing the shooting cost, and then It solves the technical problem of complicated process and low efficiency when modifying the configuration of the monitoring terminal in the related art, which affects the flexibility of the entire monitoring system.

Description of drawings

The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The schematic embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the attached image:

FIG. 1(a) is a schematic structural diagram of an image processing method and apparatus provided according to an embodiment of the present application;

Fig. 1(b) is a schematic structural diagram of another image processing method and apparatus provided according to an embodiment of the present application

Fig. 1(c) is a schematic structural diagram of another image processing method and apparatus provided according to an embodiment of the present application

2 is a schematic flowchart of an image processing method provided according to an embodiment of the present application;

3 is a schematic diagram of a feature position of a camera provided according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a scheme compensation information description provided according to an embodiment of the present application;

FIG. 5 is a schematic diagram illustrating compensation information of another scheme provided according to an embodiment of the present application;

6 is a schematic work flow diagram of a position information conversion device in an image processing method device provided according to an embodiment of the present application;

FIG. 7 is a schematic diagram of the gray value relationship between input and output images after Gamma transformation provided according to an embodiment of the present application;

FIG. 8 is a schematic structural diagram of an image processing apparatus provided according to an embodiment of the present application.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only The embodiments are part of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the scope of protection of the present application.

It should be noted that the terms "first", "second", etc. in the description and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

In order to better understand the embodiments of the present application, the terms involved in the embodiments of the present application are explained as follows:

Server: According to user needs, perform image rendering processing, generate target images, and output them to LED sending devices.

Sending device: Undertake the screen generated by the server, and send the image data to the LED display screen according to the software configuration through software configuration.

LED display: The LED display is made of LED boxes spliced into a large screen, and there is an LED control system drive circuit inside, which undertakes the image data sent by the sending device for LED drive display.

Location information generator: a device mainly used to obtain the current location of the camera.

Camera: A combination of equipment built by one or more cameras to shoot LED displays.

Example 1

In order to solve the technical problems in the related art that the modification of the configuration of the monitoring terminal is complicated, the efficiency is low, and the flexibility of the entire monitoring system is affected, an embodiment of the present application proposes a new image processing system, and the method can be applied to In the image processing system shown in Fig. 1(a), but not limited to this.

Specifically, the image processing system provided by the embodiment of the present application, as shown in FIG. 1( a ), at least includes: an image source device 11 , a sending device 12 , and a receiving device 13 , wherein:

An image source device 11, used to provide the original image and send the original image to the sending device;

In the image processing process, the received original image is processed. In order to obtain a higher quality target image, the parameters can be adjusted manually. For example, when the brightness of the received original image is too dark or too bright, the Change the brightness by dragging the brightness slider to ensure that the image of the target image received by the receiving device has a better display effect. Similarly, we can continue to modify the original image locally by using similar adjustment methods above until the desired visual effect is achieved. .

The sending device 12 is used for receiving the original image, and processing the original image to obtain the target image;

In the process of image processing, the received original processing is processed. In order to obtain a higher quality target image, the parameters can be adjusted manually. For example, when the brightness of the received original image is too dark or too bright, the Change the brightness by dragging the brightness slider to ensure that the image received by the receiving device has a better display effect. We can use the same adjustment method to locally modify the original image until the desired visual effect is achieved.

The receiving device 13 is configured to execute the image processing method provided by the embodiment of the present application. For example, the image processing method shown in FIG. 2 is performed.

During the image processing process, it is also necessary to determine the position information of the image acquisition device and the camera shooting direction information (such as the lens orientation) information, and convert the position information into coordinate information. Therefore, in the image processing system, as shown in Figure 1(b) ), it also needs to include: an information generator 14 for acquiring the coordinate information corresponding to the working position information when the image capturing device 15 performs image capturing at different working positions, and acquiring the shooting direction of the image capturing device. The information generator 14 may be located in the sending device (eg, sending card), or may be a relatively independent device from the sending device.

In addition, in order to complete the distribution of data, as shown in FIG. 1(c), an information converter 16 may also be included. The information converter may be located in the sending device (such as a sending card), or may be relatively independent from the sending device’s camera information The device, the information converter may include the following processing units: acquisition unit: acquire the existing position information of one or more cameras from the outside; camera information conversion compensation coefficient: this module converts the camera's position information into compensation data for the LED display Information; compensation coefficient sending module: send the compensation coefficient to the receiving card device, the receiving card device is the control unit of the LED control system to drive the LED light board.

On the basis of the above image processing system, the embodiment of the present application further provides an image processing method, which is used to specifically describe the process performed by the position information generator in the image processing system. It should be noted that the steps shown in the flowcharts of the accompanying drawings may be executed in a computer system, such as a set of computer-executable instructions, and, although a logical sequence is shown in the flowcharts, in some cases, Steps shown or described may be performed in an order different from that herein.

FIG. 2 is a schematic flowchart of an optional image processing method according to an embodiment of the present application. As shown in FIG. 2 , the method includes at least steps S202-S208, wherein:

Step S202 , acquiring position information of the target image in the display screen when the target image is displayed on the display screen.

Specifically, after the image device source obtains the original image, the target image is determined from the original image, and the target image is sent to the display screen. While the display screen displays the target image, the receiving device obtains the position information of the target image in the display screen, Among them, the display screen can be a common LED display screen, and the minimum display unit of the LED display screen is an LED light point. Each minimum display unit can correspond to different position coordinate information on the display screen, so the target can be obtained on the LED display screen. The position coordinate information corresponding to the image, and the receiving card receives the position coordinate information corresponding to the target image on the LED display screen.

Obtaining the location information of the target image on the display screen when the target image is displayed on the display screen can be implemented in the following ways, but is not limited to this: after receiving the original image, the original image is partially adjusted to obtain the target image, and the target image is obtained according to the target image. The image data of the image obtains the information of the target pixel, and the information of the target pixel is the position information of the target pixel. The following will give an example to describe the determination of the position information of the target pixel in detail, which will not be repeated here.

For example: a common LED display includes an LED display panel and LED lights. The LED lights on the LED display panel are distributed in a matrix, and the pixels distributed on the LED display panel are distributed in the same way as the LED lights, and are also distributed in a matrix. , all have the characteristics of horizontal and vertical. In a 300*300 multi-pixel matrix, determine the position coordinate information corresponding to the LED light, the abscissa range can be any point in 1-300, the ordinate range can be any point in 1-300, the range of each LED light Different position coordinate information corresponds to the pixel on the LED display panel. In addition, the position information of the target pixel is the two-dimensional coordinate information of the target pixel, that is, the abscissa information of the target pixel and the ordinate information of the target pixel, so it can be The position information coordinates to the target pixel according to the position information coordinates of the pixel points on the LED display panel.

Step S204, determining the working position information when the image capturing device performs image capturing at at least one position, and acquiring the compensation coefficient corresponding to the working position information, to obtain a first set of compensation coefficients, wherein the image capturing device is at least used to capture the display of the display screen image corresponding to the screen.

Specifically, first determine the working position information of the image acquisition device when the image acquisition device performs image acquisition at at least one position, and obtain the working position information of the image acquisition device when the image acquisition device performs image acquisition in at least one position through the position information generator, wherein the image acquisition device can be At least one of cameras, video cameras, cameras, scanners, and other devices with camera functions (mobile phones, tablet computers, etc.).

Determining the working position information when the image acquisition device performs image acquisition at at least one position can be obtained by acquiring the sample images collected at the sampling points in the candidate sampling point set when the image acquisition device moves within the effective moving space; obtain the quality of the sample images Evaluation index; select the target sampling point from the candidate sampling point set according to the quality evaluation index, and use the location of the target sampling point as the working position information, wherein the effective moving space range is the image acquisition device in each direction (two-dimensional or three-dimensional). in all directions) the maximum distance that can be moved. For example, the sampling point corresponding to the sample image whose quality evaluation index is greater than the preset threshold is used as the above-mentioned target sampling point, and conversely, the sampling point corresponding to the sample image whose quality evaluation index is less than the preset threshold is used as the invalid sampling point. The position where the invalid sampling point is located is used as the above-mentioned working position information, that is, in the process of processing the above-mentioned target image, image sampling is not performed at the invalid sampling point.

For example, the above-mentioned quality evaluation indicators include but are not limited to at least one of the following: image integrity, image clarity, image distortion, etc., wherein, image integrity has two meanings: in the case of one display screen (ie Provide a background image through one display screen), only the integrity of the image displayed on the display screen is detected; in the case of multiple display screens, that is, the background images are provided by the images displayed by multiple display screens, it is necessary to detect all the display screens. The integrity of the image, that is, when the integrity of all display screens is greater than the integrity threshold, is considered to pass the integrity test. For example, in the collected images on each display screen, if the borders of all the display screens are detected in the images, it is considered that the integrity detection is passed. Among them, before determining the integrity of the image, the image segmentation algorithm can be used to segment the frame and the non-frame image, and then the segmented frame image is detected. If the frame image is not closed, the integrity detection fails. Otherwise, it is considered to pass the integrity test. When determining the completeness of the image, sampling can also be implemented in the following manner, but is not limited to this: the image can be divided into multiple parts and multiple or all of the multiple parts can be selected as sub-parts to be processed; Calculate the color feature value of the subsection according to the value of the subsection in the color space; and calculate the image integrity according to whether the color of each subsection and its adjacent subsections are similar, wherein, in the subsection and its adjacent subsections, the completeness of the image is calculated. In the case where the difference between the color feature values of the subsections is smaller than the first threshold, it is determined that the color of the subsection and its adjacent subsections are similar. For example, the more sub-sections with similar colors, the more single the color is, and the image can be considered incomplete at this time. Here, when the difference between the color feature values of the subsection and its adjacent subsections is smaller than the first threshold, it may be determined that the subsection and its adjacent subsections have similar colors.

Optionally, the compensation coefficient corresponding to the working position information is received from an external device, wherein the external device includes a sending device or a server; or, a compensation coefficient corresponding to the working position information is determined from a set of compensation coefficients pre-stored by the receiving device, The compensation coefficient is based on the position information of a plurality of calibration points of the image acquisition device, and according to the compensation coefficient algorithm, the compensation coefficient required by each calibration point of the image acquisition device is calculated by the sending device of the position information converter, and the compensation coefficient is a coefficient matrix with a large amount of data, which can compensate multiple calibration points at the same time. Alternatively, a mapping relationship between the calibration point position and the compensation coefficient may be maintained on the side of the external device, and after the calibration point position is determined, the compensation coefficient corresponding to the calibration point position is determined by using the mapping relationship.

In order to more accurately determine the specific compensation coefficient, according to the preset compensation coefficient algorithm in the receiving device, the compensation coefficient in the mapping relationship corresponding to the calibration point is obtained by using the mapping relationship between the positions of multiple calibration points and the compensation coefficient, wherein the compensation coefficient is The coefficient algorithm can be the De-Mura compensation algorithm, which analyzes the pixel distribution of the display screen observed at the calibration point position, and then calculates the compensation coefficient through the De-Mura compensation algorithm according to the pixel data, but is not limited to this.

Wherein, the receiving device may include: a device for storing compensation coefficient sets or a reservation device, a device for storing compensation coefficient sets, for storing a set of compensation coefficients corresponding to the working position of the calibration point of the image acquisition device; and a reservation device for reserving image acquisition Sets of multiple calibration point positions and corresponding compensation coefficients are installed.

For example, taking the image acquisition device as a camera as an example, the camera has 9 characteristic positions in space, and the position information generator calculates the real-time position of the camera according to the relationship between the camera and the characteristic positions, wherein the position information generator can be a sending device Or a module in the receiving device, which can be a relatively independent device from the sending device or the receiving device. The calculation process of the real-time position of the above-mentioned camera is described below in conjunction with FIG. 3 :

When the user has completed the selection of the feature positions, the positions of 9 points are selected (this solution is not limited to 9 points, it can be any point); and the matrix of the LED display corresponding to the feature points is obtained by algorithm or manual adjustment compensation coefficient;

After the camera position is moved, according to the different coordinate positions of the camera, the compensation coefficient corresponding to the coordinate position is determined and sent to the LED display screen;

When the camera is at the (100,100) coordinate position, after the (100,100) coordinates are sent to the sending device, the sending device will send (100,100) the compensation coefficient corresponding to the entire LED display to the LED display in real time;

When the camera moves to (2000,100) and the (2000,100) coordinates are sent to the sending device, the sending device will send (2000,100) the compensation coefficient corresponding to the entire LED display to the LED display in real time.

Specifically, when shooting people and scenes indoors, the display screen is used as the display background, which can reduce a lot of post-production costs. The camera is used to shoot the display screen to capture the display screen on the display screen, because the camera shoots at different angles and directions. , there will be a color cast problem. The commonly used display screen is an LED display screen. The smallest display unit of the LED display screen is an LED light point. A single LED light point contains three independent light-emitting units of RGB. These light-emitting units have a certain spatial arrangement relationship, including: a vertical one. Character arrangement, horizontal in-line arrangement, triangular arrangement, etc. For example, when the light-emitting units are arranged horizontally from left to right in RGB, and observed on the left side of the LED display screen, the luminous intensity relationship of the display screen is R>G>B, Introduce the phenomenon of reddishness; observed on the right side, the luminous intensity seen is: R<G<B, then the phenomenon of bluishness is seen, and the specific compensation coefficient needs to be determined according to the color cast of the image observed by the specific shooting. That is to say, when the image acquisition device collects images on the display screen at different positions, the target image collected at different positions will have a color cast problem, and the compensation for the target image is determined according to the working position information of the image acquisition device at different positions. Coefficient, the compensation coefficient refers to the multiple of the color difference of the target image to be processed. The compensation coefficient is a non-zero positive number. If the compensation coefficient is greater than 1, it means that the backlight of the target image to be processed is too dark, and the brightness needs to be increased; if the compensation coefficient is less than 1, it means that the To deal with the target image being over-bright and over-exposed, it is necessary to reduce the brightness.

Step S206, determining the compensation weight of the target image in different shooting directions according to the position information of the target image in the display screen, and determining the target compensation coefficient based on the compensation weight and the first compensation coefficient set. The weight is determined by the distance between the calibration point position of the image acquisition device and the target image. The distance is inversely proportional to the weight, that is, the larger the distance, the smaller the weight. For example, there are 4 points, and the pixel that is closer to the calibration point corresponding to the shooting direction, Then the compensation weight corresponding to the calibration point is larger.

Optionally, the position information of a plurality of calibration points in the image acquisition device is acquired, and the size information of the display screen is determined, wherein the positions of the calibration points corresponding to the position information of the plurality of calibration points correspond to different shooting directions one-to-one, wherein, The size information can be determined based on the number of pixels in the display screen, or can be determined by other measurement methods; determine the target viewing angle corresponding to the position information of multiple calibration points, wherein the target viewing angle is the image acquisition device in different working positions. The angle formed by the shooting direction of the acquisition device and the normal of the plane where the display screen is located; the difference information between the coordinate information and the size information corresponding to the position information of the target image in the display screen; the compensation weight is determined according to the difference information and the target viewing angle .

Specifically, when the image capture device captures the picture displayed on the display screen, the image capture device can capture from different directions at different working positions. Corresponding the calibration points and the corresponding shooting directions one by one, the acquisition of more Only one calibration point can obtain corresponding different shooting directions. At the same time, the angle between the image acquisition device and the normal line of the display screen when the acquisition device corresponding to the multiple calibration points is determined to shoot in different working positions is marked as the target angle.

While obtaining the calibration point position information and the target viewing angle corresponding to the calibration point position information, the size information of the display screen is determined. The size of the common LED display screen is affected by many factors, including area, display ratio, model, and module size. , box size, etc. There are various models. The size of the box can be determined according to the actual situation. The size of the display screen is determined according to the size of the box. The specific application process is not limited to this implementation.

Wherein, in order to save computing resources, the plurality of calibration points corresponding to the above-mentioned position information of the plurality of calibration points include at least calibration points distributed on two straight lines, and the two straight lines are perpendicular to each other. The two straight lines may be lines connecting calibration points located in different diagonal directions, but are not limited thereto. For example, the straight line composed of (128, 0) and (128, 256) and the straight line composed of (0, 128) and (256, 128) in Fig. 4 are perpendicular to each other, and there are four compensation coefficients for the above four coordinate positions. The number of the above-mentioned multiple marking points may be an even number or an odd number, which is flexibly set according to the actual situation.

After the above-mentioned display screen size information is determined, the difference information between the coordinate information corresponding to the position information of the target image in the display screen and the display screen size information is also required; and the compensation weight is determined according to the difference information and the target viewing angle. That is to say, when determining the above compensation weight, it can be determined according to the above difference information. However, it should be noted that the specific application process is not limited to this implementation manner. For example, the above compensation weight may be preset, such as , maintain the mapping relationship between marker points and weights in the database, and determine the corresponding compensation weights according to the mapping relationship.

When the compensation weight is determined according to the difference information and the target viewing angle, it can be implemented in the following manner, but is not limited to this: performing a difference operation between the size information and the coordinate information corresponding to the position information of the target image in the display screen, and according to the operation The result and the target angle of view determine the compensation weight, wherein the compensation coefficients in the first compensation coefficient set are obtained according to the position information of multiple calibration points of the image acquisition device, and the target compensation coefficient is obtained according to the position information of the current target image, the size of the box. The size and the first set of compensation coefficients are obtained, and the target compensation coefficient is a compensation coefficient matched with the target image determined by the position information of the current target image. Detailed examples will be given below, and will not be repeated here.

In some optional embodiments of the present application, the first set of compensation coefficients is a coefficient matrix, wherein each element in the coefficient matrix is a compensation coefficient corresponding to each mark point in the plurality of mark points; based on the compensation weight and the first A set of compensation coefficients determines a target compensation coefficient, for example, a second set of coefficients is determined according to all coefficients in the coefficient matrix and compensation weights; then the target compensation coefficient is determined based on the average value of all coefficients in the second coefficient set, specifically, the coefficient matrix Each coefficient of is multiplied by its corresponding weight, and then the operation result obtained after the multiplication is averaged, and the average value is taken as the target compensation coefficient.

In the following, in conjunction with Figure 4 and Figure 5, the two compensation schemes are described by taking the LED box with a resolution of 256*256 rectangle as an example. As shown in Figure 5, the compensation scheme B is described. The four circles represent the 4 coordinate positions, among which, each coordinate position has 4 compensation coefficients, and the adjustment coefficient has the following characteristics:

Data that can characterize the gradient is required, not necessarily 4, but may also be 2 or 1 data to characterize the compensation gradient of the LED display from left to right or from top to bottom; the meaning of the gradient here is 4 coefficients For example, each 2 coefficients is a pair, and the 2 coefficients only represent the left and right ends, or the upper and lower ends, but the point-by-point pixels will be processed gradually according to the coefficients corresponding to the two calibration points, that is, the corresponding gradient deal with.

The adjustment parameter can be 3*3 matrix data or 1*3 brightness compensation;

For example: the adjustment parameter M11Coef of screen 1 obtained from the feature point (128,0)

The adjustment parameter M12Coef of the screen 1 of the feature point (0,128)

...

The adjustment parameter M22Coef of the screen 1 of the feature point (128,256)

That is, the compensation coefficients are M11Coef, M12Coef, M21Coef, and M22Coef.

As can be seen from Figure 6, during the display process of the 256*256 box image, each point needs to obtain the position information of the current image, obtain the current image data according to the position information, and then perform Gamma conversion. According to the position information of the current image, the coordinates (X, Y) of the current image are determined. Calculate the data weight values corresponding to the current image data under the 4 coordinates according to XY, and then obtain the matrix coefficients of the current image according to the weight values.

If the coordinates of the current image are X=128, Y=128; Width represents the width of the LED box pixel, and Height represents the height of the LED box pixel. Then the corresponding current adjustment coefficient calculation method is:

M=M11tmp1+M12tmp+M21tmp+M22tmp

According to the relationship between the current position information and the characteristic position information, the compensation weight value of the final A and B is obtained according to the angle value, and the final realization data is obtained.

Then the value of M corresponding to (128,128) is

The image information is compensated according to the above gradient operation, and the compensated first image data is obtained, and the output is displayed on the display screen, so as to ensure that the display screen of the display screen shot by the camera at various angles and directions has no color deviation.

Step S208, the target image is compensated according to the target compensation coefficient.

The compensation coefficients in the first compensation coefficient set are obtained according to the position information of multiple calibration points of the image acquisition device, the size information of the display screen, the first compensation coefficient set and the position information of the target image, and the target compensation coefficient is obtained according to the current target The relationship between the position information of the image and the position information of multiple calibration points is determined according to the current target angle, and the unique compensation coefficient corresponding to the target image is determined. In order to ensure the compensation effect of the image, before compensating the target image according to the target compensation coefficient, the following processing can also be performed on the target image: obtain the original grayscale image of the target image, convert the original grayscale image, and obtain the target grayscale image , where the target grayscale image is exponentially related to the original grayscale image.

Specifically, the sensitivity value of the human eye to the external light source is not linearly related to the input light intensity, but is exponentially related. Under low illumination, it is easier for the human eye to distinguish the change of brightness. With the increase of illumination, it is difficult for the human eye to distinguish the change of brightness. The camera sensitivity is linearly related to the input light intensity. The first image data is enhanced by using Gamma transformation to ensure that the accuracy of the first image data is improved. Among them, Gamma transformation is a nonlinear operation on the gray value of the input image, so that the gray value of the output image has an exponential relationship with the gray value of the input image. Among them, the value range of the gray value of the input image is 0 to 1, so it needs to be normalized first, and then the index is taken.

Gamma transformation is used for image enhancement, which improves the details of dark parts. In short, through nonlinear transformation, the image changes from a linear response of exposure intensity to a response that is closer to the response of the human eye, that is, bleaching (camera exposure) or too dark. (underexposed) picture, correct it.

The gray value relationship between the input and output images after Gamma transformation is shown in Figure 7: the abscissa is the input gray value, the ordinate is the output gray value, and the downward curve of the opening is the input-output relationship when the gamma value is less than 1. The upward curve of the opening is the input-output relationship when the gamma value is greater than 1. It can be observed that when the gamma value is less than 1, the overall brightness value of the image is improved, while the contrast at low gray levels is increased, which is more conducive to distinguishing image details at low gray levels.

That can be summarized as follows:

Gamma>1, the grayscale of the brighter area is stretched, the grayscale of the darker area is compressed to be darker, and the overall image becomes darker;

Gamma<1, the grayscale of the brighter area is compressed, the grayscale of the darker area is stretched to be brighter, and the overall image becomes brighter;

Among them, the gray value of 0 is black, and 255 is white.

In the embodiment of the present application, the position information of the target image in the display screen when the target image is displayed on the display screen is firstly obtained; the working position information of the image acquisition device when the image is collected at at least one position is determined, and the working position information is obtained Corresponding compensation coefficients, a first set of compensation coefficients is obtained, wherein the image acquisition device is at least used to collect images corresponding to the display screen of the display screen; according to the position information of the target image in the display screen, it is determined that the target image is in different shooting directions and the target compensation coefficient is determined based on the compensation weight and the first compensation coefficient set; the target image is compensated according to the target compensation coefficient. Further, in the virtual shooting scene, the related technology over-compensates or improperly compensates the image on the server side, or the technical problem of the large image delay caused by the server-side processing is solved.

Example 2

According to an embodiment of the present application, an image processing apparatus for implementing the above image processing method is also provided, as shown in FIG. 8 , including: an acquisition module 81 , a first determination module 82 , a second determination module 83 , and a compensation module 84 .

in:

an acquisition module 81, configured to acquire position information of the target image in the display screen when the target image is displayed on the display screen;

The first determination module 82 determines the working position information when the image capturing device performs image capturing at at least one position, and acquires the compensation coefficient corresponding to the working position information to obtain a first set of compensation coefficients, wherein the image capturing device is at least used for capturing and displaying The image corresponding to the display screen of the screen;

Specifically, first determine the working position information of the image acquisition device when the image acquisition device performs image acquisition at at least one position, and obtain the working position information of the image acquisition device when the image acquisition device performs image acquisition in at least one position through the position information generator, wherein the image acquisition device can be At least one of cameras, video cameras, cameras, scanners, and other devices with camera functions (mobile phones, tablet computers, etc.).

Optionally, the compensation coefficient corresponding to the working position information is received from an external device, wherein the external device includes a sending device or a server; or, a compensation coefficient corresponding to the working position information is determined from a set of compensation coefficients pre-stored by the receiving device, The compensation coefficient is based on the position information of a plurality of calibration points of the image acquisition device, and according to the compensation coefficient algorithm, the compensation coefficient required by each calibration point of the image acquisition device is calculated by the sending device of the position information converter, and the compensation coefficient is a coefficient matrix with a relatively large amount of data, which can compensate multiple calibration points at the same time.

For example, the camera has 9 characteristic positions in space, and the position information generator calculates the real-time position of the camera according to the relationship between the camera and the characteristic positions. It is a relatively independent device from the sending device or the receiving device. The calculation process of the real-time position of the above-mentioned camera is described below in conjunction with FIG. 3 :

When the user has completed the selection of the feature positions, the positions of 9 points are selected (this solution is not limited to 9 points, it can be any point); and the matrix of the LED display corresponding to the feature points is obtained by algorithm or manual adjustment compensation coefficient;

After the camera position is moved, according to the different coordinate positions of the camera, the compensation coefficient corresponding to the coordinate position is determined and sent to the LED display screen;

When the camera is at the (100,100) coordinate position, after the (100,100) coordinates are sent to the sending device, the sending device will send (100,100) the compensation coefficient corresponding to the entire LED display to the LED display in real time;

When the camera moves to (2000,100) and the (2000,100) coordinates are sent to the sending device, the sending device will send (2000,100) the compensation coefficient corresponding to the entire LED display to the LED display in real time.

Specifically, when shooting people and scenes indoors, the display screen is used as the display background, which can reduce a lot of post-production costs. The camera is used to shoot the display screen to capture the display screen on the display screen, because the camera shoots at different angles and directions. , there will be a color cast problem. The commonly used display screen is an LED display screen. The smallest display unit of the LED display screen is an LED light point. A single LED light point contains three independent light-emitting units of RGB. These light-emitting units have a certain spatial arrangement relationship, including: a vertical one. Character arrangement, horizontal in-line arrangement, triangular arrangement, etc. For example, when the light-emitting units are arranged horizontally from left to right in RGB, and observed on the left side of the LED display screen, the luminous intensity relationship of the display screen is R>G>B, Introduce the phenomenon of reddishness; observed on the right side, the luminous intensity seen is: R<G<B, then the phenomenon of bluishness is seen, and the specific compensation coefficient needs to be determined according to the color cast of the image observed by the specific shooting. That is to say, when the image acquisition device collects images on the display screen at different positions, the target image collected at different positions will have a color cast problem, and the compensation for the target image is determined according to the working position information of the image acquisition device at different positions. Coefficient, the compensation coefficient refers to the multiple of the color difference of the target image to be processed. The compensation coefficient is a non-zero positive number. If the compensation coefficient is greater than 1, it means that the backlight of the target image to be processed is too dark, and the brightness needs to be increased; if the compensation coefficient is less than 1, it means that the To deal with the target image being over-bright and over-exposed, it is necessary to reduce the brightness.

The second determination module 83 determines the compensation weights of the target image in different shooting directions according to the position information of the target image in the display screen, and determines the target compensation coefficient based on the compensation weight and the first compensation coefficient set, and the compensation weight is the image acquisition device The distance between the calibration point and the target image is determined, and the distance is inversely proportional to the compensation weight, that is, the larger the distance, the smaller the weight. For example, if there are 4 points, the closer the pixel is to the calibration point corresponding to the shooting direction, then the calibration point corresponds to The greater the compensation weight;

Optionally, the position information of a plurality of calibration points in the image acquisition device is acquired, and the size information of the display screen is determined, wherein the positions of the calibration points corresponding to the position information of the plurality of calibration points are in one-to-one correspondence with different shooting directions, The size information is determined based on the number of pixels in the display screen, and can also be determined by other measurement methods; determine the target viewing angle corresponding to the position information of multiple calibration points, wherein the target viewing angle is when the image acquisition device is in different working positions, The angle formed by the shooting direction of the image acquisition device and the normal of the plane where the display screen is located; the difference information between the coordinate information and the size information corresponding to the position information of the target image in the display screen; the compensation is determined according to the difference information and the target viewing angle Weights.

Specifically, when the image capture device captures the picture displayed on the display screen, the image capture device can capture from different directions at different working positions. Corresponding the calibration points and the corresponding shooting directions one by one, the acquisition of more Only one calibration point can be obtained corresponding to different shooting directions. At the same time, the angle between the image acquisition device and the normal line of the display screen when the acquisition device corresponding to the multiple calibration points is determined to shoot at different working positions is marked as the target angle, where, The target angle is determined according to the position information of multiple calibration points of the image acquisition device and the angle between the direction vector of the normal line where the display screen is located.

While obtaining the calibration point position information and the target viewing angle corresponding to the calibration point position information, the size information of the display screen is determined. The size of the common LED display screen is affected by many factors, including area, display ratio, model, and module size. , box size, etc. There are various models. The size of the box can be determined according to the actual situation. The size of the display screen is determined according to the size of the box. The specific application process is not limited to this implementation.

Wherein, in order to save computing resources, the plurality of calibration points corresponding to the above-mentioned position information of the plurality of calibration points include at least calibration points distributed on two straight lines, and the two straight lines are perpendicular to each other. For example, in Figure 4, the straight line composed of (128, 0) and (128, 256) and the straight line composed of (0, 128) and (256, 128) are perpendicular to each other, and there are four compensation coefficients for the above four coordinate positions, The number of the above-mentioned multiple marking points may be an even number or an odd number, which is flexibly set according to the actual situation.

After the above-mentioned display screen size information is determined, the difference information between the coordinate information corresponding to the position information of the target image in the display screen and the display screen size information is also required; and the compensation weight is determined according to the difference information and the target viewing angle. That is to say, when determining the above compensation weight, it can be determined according to the above difference information. However, it should be noted that the specific application process is not limited to this implementation manner. For example, the above compensation weight may be preset, such as , maintain the mapping relationship between marker points and weights in the database, and determine the corresponding compensation weights according to the mapping relationship.

When the compensation weight is determined according to the difference information and the target viewing angle, it can be implemented in the following manner, but is not limited to this: performing a difference operation between the size information and the coordinate information corresponding to the position information of the target image in the display screen, and according to the operation The result and target perspective determine the compensation weights.

The compensation module 84 is used for compensating the target image according to the target compensation coefficient.

Specifically, the target compensation coefficient is obtained according to the position information of the current target image, the size of the box and the first compensation coefficient set, and the target compensation coefficient is a compensation coefficient determined by the position information of the current target image and matched with the target image.

In order to ensure the compensation effect of the image, before compensating the target image according to the target compensation coefficient, the following processing can also be performed on the target image: obtain the original grayscale image of the target image, convert the original grayscale image, and obtain the target grayscale image , where the target grayscale image is exponentially related to the original grayscale image.

Specifically, the sensitivity value of the human eye to the external light source is not linearly related to the input light intensity, but is exponentially related. Under low illumination, it is easier for the human eye to distinguish the change of brightness. With the increase of illumination, it is difficult for the human eye to distinguish the change of brightness. However, the light sensitivity of the camera has a linear relationship with the input light intensity, and Gamma conversion is used to enhance the first image data to ensure that the accuracy of the first image data is improved. According to an embodiment of the present application, an image processing system for implementing the above image processing method is also provided. As shown in FIG. 1( a ), the system at least includes: an image source device 11 , a sending device 12 , and a receiving device 13 , wherein the image source device 11 is used to provide the original image and send the original image to the sending device;

In the process of the image processing method, the received original image is processed. In order to obtain a higher quality target image, the parameters can be adjusted manually. For example, when the brightness of the received original image is too dark or too bright, You can change the brightness by dragging the brightness slider to ensure that the image of the target image received by the receiving device has a better display effect. Similarly, we can continue to modify the original image locally by using similar adjustment methods above until the desired visual effect is achieved. until.

The sending device 12 is used for receiving the original image, and processing the original image to obtain the target image;

In the process of the image processing method, the received original processing is processed. In order to obtain a higher quality target image, the parameters can be adjusted manually. For example, when the brightness of the received original image is too dark or too bright, You can change the brightness by dragging the brightness slider to ensure that the image received by the receiving device has a better display effect. We can use the same adjustment method to locally modify the original image until the desired visual effect is achieved.

The receiving device 13 is configured to execute the image processing method provided by the embodiment of the present application. For example, execute the image processing method shown in Figure 2;

In the process of executing the image processing method, since it is also necessary to determine the position information of the image acquisition device and the camera shooting direction information (such as the lens orientation) information, and convert the position information into coordinate information, therefore, in the image processing system, as shown in Figure 1 ( b), it also needs to include: an information generator 14 for acquiring the coordinate information corresponding to the working position information when the image capturing device 15 performs image capturing at different working positions, and acquiring the shooting direction of the image capturing device. The information generator 14 may be located in the sending device (eg, sending card), or may be a device relatively independent from the sending device's camera information.

In addition, in order to complete the distribution of data, as shown in FIG. 1(c), an information converter 16 may also be included. The information converter may be located in the sending device (such as a sending card), or may be relatively independent from the sending device’s camera information The device, the information converter may include the following processing units: acquisition unit: acquire the existing position information of one or more cameras from the outside; camera information conversion compensation coefficient: this module converts the camera's position information into compensation data for the LED display Information; compensation coefficient sending module: send the compensation coefficient to the receiving card device, the receiving card device is the control unit of the LED control system to drive the LED light board.

Example 3

According to an embodiment of the present application, a non-volatile storage medium is also provided, and the non-volatile storage medium includes a stored program, wherein when the program runs, the device where the non-volatile storage medium is located is controlled to execute the first embodiment image processing method.

Optionally, when the program is running, the following steps are performed: obtaining the position information of the target image in the display screen when the target image is displayed on the display screen; Obtain the compensation coefficient corresponding to the working position information, and obtain the first compensation coefficient set, wherein the image acquisition device is at least used to collect the image corresponding to the display screen of the display screen; The compensation weight in the direction is determined, and the target compensation coefficient is determined based on the compensation weight and the first compensation coefficient set; the target image is compensated according to the target compensation coefficient.

The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

In the above-mentioned embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are only illustrative, for example, the division of the units may be a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components may be combined or Integration into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of units or modules, and may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes .

The above are only the preferred embodiments of the present application. It should be pointed out that for those skilled in the art, without departing from the principles of the present application, several improvements and modifications can also be made. It should be regarded as the protection scope of this application.

Claims (10)

1. an image processing method, is characterized in that, comprises: acquiring the position information of the target image in the display screen when the target image is displayed on the display screen; Determining the working position information when the image capturing device performs image capturing at at least one position, and acquiring the compensation coefficient corresponding to the working position information to obtain a first set of compensation coefficients, wherein the image capturing device is at least used for capturing the display screen display; determining the compensation weight of the target image in different shooting directions according to the position information of the target image in the display screen, and determining the target compensation coefficient based on the compensation weight and the first compensation coefficient set; The target image is compensated according to the target compensation coefficient. 2. The method according to claim 1, wherein determining the compensation weights of the target image in different shooting directions according to the position information of the target image in the display screen, comprising: Acquire multiple calibration point position information of the image acquisition device, and determine the size information of the display screen, wherein the calibration point positions corresponding to the multiple calibration point position information are in one-to-one correspondence with the different shooting directions ; Determine the target angle of view corresponding to the position information of the multiple calibration points, wherein the target angle of view is the method of the shooting direction of the image capture device and the plane where the display screen is located when the image capture device is in different working positions. the angle formed by the line; Difference information between the coordinate information corresponding to the position information of the target image in the display screen and the size information; The compensation weight is determined according to the difference information and the target viewing angle. 3. The method according to claim 2, wherein determining the compensation weight according to the difference information and the target viewing angle comprises: The difference operation is performed between the size information and the coordinate information corresponding to the position information of the target image in the display screen, and the compensation weight is determined according to the operation result and the target viewing angle. 4 . The method according to claim 2 , wherein the first compensation coefficient set is a coefficient matrix, wherein each element in the coefficient matrix is each calibration point in the plurality of calibration point position information. 5 . Compensation coefficient corresponding to the fixed-point position information; determining the target compensation coefficient based on the compensation weight and the first compensation coefficient set, including: A second set of coefficients is determined according to all the coefficients in the coefficient matrix and the compensation weight; a target compensation coefficient is determined based on the average value of all the coefficients in the second set of coefficients. 5 . The method according to claim 2 , wherein the plurality of calibration points corresponding to the position information of the plurality of calibration points include at least calibration points distributed on two straight lines, and the two straight lines are perpendicular to each other. 6 . 6. The method according to claim 1, wherein obtaining the compensation coefficient corresponding to the working position information comprises: Receive the compensation coefficient corresponding to the working position information from the external device; or, The compensation coefficient corresponding to the working position information is determined from a compensation coefficient set pre-stored by the receiving device. 7. The method according to any one of claims 1 to 6, characterized in that, determining the working position information when the image acquisition device performs image acquisition at at least one position, comprising: acquiring sample images collected at sampling points in the candidate sampling point set when the image acquisition device moves within the effective moving space; obtaining the quality evaluation index of the sample image; A target sampling point is selected from the candidate sampling point set according to the quality evaluation index, and the location of the target sampling point is used as the working position information. 8. An image processing device, comprising: an acquisition module, configured to acquire the position information of the target image in the display screen when the target image is displayed on the display screen; a first determination module, configured to determine the working position information when the image acquisition device performs image acquisition at different working positions, and obtain the compensation coefficient corresponding to the working position information, so as to obtain a first set of compensation coefficients; A second determining module, configured to determine compensation weights in different shooting directions according to the position information of the target image in the display screen, and determine the target based on the compensation weights and the first set of compensation coefficients compensation coefficient; The compensation module is used for compensating the target image according to the target compensation coefficient. 9. An image processing system, comprising: an image source device, a sending device, and a receiving device, wherein: the image source device, for providing an original image and sending the original image to a sending device; The sending device is configured to receive the original image and process the target image to obtain the target image; The receiving device is configured to execute the image processing method described in any one of claims 1 to 7. 10. A non-volatile storage medium, characterized in that the non-volatile storage medium comprises a stored program, wherein when the program is run, a device on which the storage medium is located is controlled to execute one of claims 1 to 7 The image processing method described in any one of them.

Images