CN116009274A - Light field display method and device - Google Patents

Abstract

The disclosure provides a light field display method and device. The display method includes: acquiring light field data; wherein, the light field data includes a plurality of viewpoint images; the viewpoint images include pixel weight parameters; using an image processing model to process the viewpoint images, and determining target pixels and Non-target pixels; and update the value of the pixel weight parameter of the target pixel; determine each liquid crystal layer of the spatial light modulation array according to each of the viewpoint images and the value of the pixel weight parameter corresponding to each pixel after being processed by the image processing model The order of calculation of the transmittance of each pixel and the value of the transmittance. Such an approach helps to improve computational efficiency and the quality of light field display.

Description

Light field display method and device

technical field

The present disclosure relates to the field of display technology, in particular to a light field display method and device.

Background technique

A 3D display can provide a deeper sense of immersion than a 2D display. At present, there are mainly three-dimensional light field display methods, integrated imaging, multi-projection array, multi-layer liquid crystal and other display methods, among which the multi-layer liquid crystal display technology takes into account both maturity and cost.

Contents of the invention

In view of this, the purpose of the present disclosure is to provide a light field display method and device.

Based on the above purpose, in a first aspect, the present disclosure provides a light field display method, the display method comprising:

Acquiring light field data; wherein, the light field data includes a plurality of viewpoint images; the viewpoint images include pixel weight parameters;

Using an image processing model to process the viewpoint image, determining target pixels and non-target pixels according to the processing results; and updating the value of the pixel weight parameter of the target pixel;

According to each of the viewpoint images and the value of the pixel weight parameter corresponding to each pixel after being processed by the image processing model, the calculation sequence and the value of the light transmittance of each pixel of each liquid crystal layer of the spatial light modulation array are determined.

In a second aspect, the present disclosure provides a light field display device, the light field display device includes a spatial light modulation array and a processor; the spatial light modulation array includes at least two liquid crystal layers; the liquid crystal layer includes an array arrangement of pixels;

The processor is configured to perform the following steps:

Acquiring light field data; wherein, the light field data includes a plurality of viewpoint images; the viewpoint images include pixel weight parameters;

Using an image processing model to process the viewpoint image, determining target pixels and non-target pixels according to the processing results; and updating the value of the pixel weight parameter of the target pixel;

According to each of the viewpoint images and the value of the pixel weight parameter corresponding to each pixel after being processed by the image processing model, the calculation sequence and the value of the light transmittance of each pixel of each liquid crystal layer of the spatial light modulation array are determined.

It can be seen from the above that the light field display method and device provided by the present disclosure assign different pixel weights to the pixels of the viewpoint image through image processing, and determine the transparency of each pixel point of each liquid crystal layer of the spatial light modulation array based on the pixel weights. The calculation order of the light rate and the value of the light transmittance, this method helps to improve the calculation efficiency and the quality of the light field display.

Description of drawings

In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the following will briefly introduce the accompanying drawings that need to be used in the descriptions of the embodiments or related technologies. Obviously, the accompanying drawings in the following description are only for the present disclosure Embodiments, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1 is a schematic flowchart of a light field display method provided by an embodiment of the present disclosure;

FIG. 2 is a partial schematic diagram of light field data provided by an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a scene for determining an image of a target field of view provided by an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of the effect of the background removal model provided by the embodiment of the present disclosure;

FIG. 5 is a schematic diagram of the effect of the frame recognition model provided by the embodiment of the present disclosure;

FIG. 6 is a schematic diagram of deriving pixels according to rays provided by an embodiment of the present disclosure;

FIG. 7 is a schematic flowchart of another light field display method provided by an embodiment of the present disclosure;

FIG. 8 is a schematic structural diagram of a light field display device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure shall have ordinary meanings understood by those skilled in the art to which the present disclosure belongs. "First", "second" and similar words used in the embodiments of the present disclosure do not indicate any sequence, quantity or importance, but are only used to distinguish different components. "Comprising" or "comprising" and similar words mean that the elements or items appearing before the word include the elements or items listed after the word and their equivalents, without excluding other elements or items. Words such as "connected" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

In order to solve the problem of light field display, the existing multi-layer liquid crystal display technology is realized through the following scheme: through the light transmittance of each pixel of the multi-layer liquid crystal at different angles to modulate the light combined with the data decomposition algorithm to calculate each pixel from the light field data value to reconstruct the light field.

For the technical solution of multi-layer liquid crystal and algorithm reconstruction of light field to realize light field display, since the more layers of liquid crystal, the lower the overall light transmittance, and the less the number of layers, the higher the complexity of the algorithm, so it is difficult to take into account the image Brightness and quality issues. Specifically, when the number of liquid crystal layers is small, since each pixel shares more light, it is difficult for the algorithm to optimize and reconcile light from various angles when reconstructing the light field, resulting in poor display quality.

Based on this, the present disclosure provides a light field display method. Through image processing, different pixel weights are given to the pixels of the viewpoint image, and the calculation sequence and the value of the light transmittance of each pixel point of each liquid crystal layer of the spatial light modulation array are determined based on the pixel weights, and when any of the pixels After the light transmittance of the point is determined, the light transmittance of the corresponding pixel is locked. This method can ensure that the light transmittance of the pixel corresponding to the light passing through the pixel with high pixel weight in the viewpoint image is preferentially calculated and locked, ensuring that the reconstructed light meets the display effect in the viewpoint image, thus contributing to Improve display quality. In addition, when the pixel weight of a pixel is as low as zero, it indicates that the light corresponding to the pixel does not need to perform light field decomposition calculations, thereby reducing the total amount of light field data that needs to be subjected to light field decomposition, thereby helping to improve calculation efficiency.

Please refer to Figure 1 and Figure 7, the light field display method includes:

Step S101: Acquire light field data; wherein, the light field data includes a plurality of viewpoint images; and the viewpoint images include pixel weight parameters.

It should be understood that a light field (Light Field) is a complete representation of the collection of light in space, including psychological visual perception information and physiological visual perception information. Collecting and displaying a light field can visually reproduce the real world.

Here, using light field data, it is possible to visually reproduce the real world.

The viewpoint image refers to the image information of a single angle in the light field. Those skilled in the art can understand that the user can watch the light field display screen from multiple angles, so the light field data includes multiple viewpoint images V _x ×V _y , for example 7×7, and the resolution of each viewpoint image is R _x ×R _y (as shown in Figure 2).

Assuming 7×7×1080×720 light field data and 1080×720 liquid crystal layer pixels, the initial average load of each pixel is:

That is to say, when decomposing the light field data, each pixel needs to take into account 16.33 rays at the same time. Therefore, when decomposing the light field data, they will inevitably conflict with each other, which is the main reason for the low quality of the reconstructed light field. Wherein, N represents the number of liquid crystal layers.

Optionally, the initial weight value of the pixel weight parameter of each pixel is equal, for example, it may be 1. At this time, the pixel weights of each pixel in the viewpoint image are the same, and their importance in the calculation of light field decomposition is also consistent. It should be understood that the value of the pixel weight parameter of each pixel may also be different, which is not specifically limited here.

In some embodiments, as shown in FIG. 3 , an edge sensor 301 is used to acquire edge sensing data, and based on the edge sensing data, the light field data is filtered to determine a target viewpoint image 302 . It should be noted that the edge sensor 301 is used to determine the position of the user, that is, the specific position of the user relative to the spatial light modulation array can be determined through the edge sensor 301 .

For example, please refer to Figure 3, the number of original viewpoint images is 7×7, if there are two users, the number of filtered target viewpoint images is 2×5. Those skilled in the art can understand that the viewpoint images other than the target viewpoint image are all invalid viewpoint images, and do not participate in light field decomposition calculation and light field reconstruction.

Here, the edge sensor 301 may be a camera or a laser radar, which is not specifically limited here. Those skilled in the art can understand that according to the difference of the edge sensor 301 , the acquired edge sensing data is also correspondingly different. For example, the data corresponding to the camera is the camera image; the data corresponding to the lidar is the distance data, etc.

With such a technical solution, the target viewpoint image is determined through edge sensing data, and the light transmittance of each pixel in each liquid crystal layer in the spatial light modulation array is subsequently determined only by considering the light of the target viewpoint image object, which can greatly reduce the Computational amount of light field decomposition. That is to say, since the invalid viewpoint images are not observed by the user, their corresponding light rays are not considered when calculating the light transmittance of each pixel.

In addition, since the number of viewpoint images loaded by each pixel is reduced, it is convenient to optimize and reconcile light from various angles when reconstructing the light field, which also helps to improve display quality.

Further, the edge perception data includes a camera image; the camera image includes a plurality of regions and each region matches a viewpoint image; the display method includes:

Utilize the face recognition model to identify the face in the camera image and determine the area of the face in the camera image; here, the face recognition model can adopt a mature face recognition model in the prior art, which is no longer described here enumerate.

According to the region, the viewpoint images corresponding to the region and its surrounding regions are determined as the target viewpoint image 302 .

As an example, please refer to FIG. 3 , the specific range of the surrounding area may be the upper, lower, left and right viewpoint images of the area where the face is located. In consideration of the horizontal movement of the user, it is also possible to increase the viewpoint images in the left and right directions to five.

Step S103: Process the viewpoint image using an image processing model, determine target pixels and non-target pixels according to the processing result; and update the value of the pixel weight parameter of the target pixel.

It should be noted that if the edge sensor 301 is used to obtain edge perception data, based on the edge perception data, the light field data is filtered to determine the target viewpoint image, the viewpoint image in this step may be the target viewpoint image, such The method is beneficial to reduce the calculation amount of image processing, and is also beneficial to reduce the calculation amount of ray decomposition and improve the display quality.

It should be understood that after being processed by the image processing model, the pixels in the viewpoint image that meet the screening requirements of the image processing model are target pixels, and the pixels that do not meet the requirements are non-target pixels.

Exemplarily, the image processing model includes a background removal model. Here, the background removal model is pre-trained. For the specific training method of the background removal model, neural network, GrabCut algorithm, etc. can be used, which are not limited here.

As shown in FIG. 4 , based on the background removal model, it is determined that the background pixel in the viewpoint image is the first target pixel; the value of the pixel weight parameter corresponding to the first target pixel is reduced.

It should be understood that, for the background pixels in the viewpoint image, they are less important to realize the light field display, so the updated pixel weight parameter can be 0, then the light field data can be represented by L(2,5 ,1080,480) said. That is to say, light rays corresponding to background pixels may not participate in light field decomposition.

Exemplarily, as shown in FIG. 5 , the image processing model includes a border recognition model, and based on the border recognition model, it is determined that the border pixels in the viewpoint image are the second target pixels; the second target pixels correspond to Increase the value of the pixel weight parameter.

Here, the border recognition model may be Hough transform, or a model obtained through training using a neural network or a border regression algorithm, which is not limited here.

Generally, the frame of an object plays an important role in the visual effect of the object, and its importance for realizing light field display is higher than that of ordinary pixels, so the value of its pixel weight parameter can be 2, so as to preferentially participate in light field decomposition.

Since users usually pay more attention to text content, the image processing model may also include a text recognition model. Here, the text recognition model can be obtained through neural network and deep learning model training, which is not specifically limited here. Of course, those skilled in the art can understand that the character recognition model here can also be an open-source mature character recognition model.

Based on the character recognition model, it is determined that the character pixel in the viewpoint image is the third target pixel; the value of the pixel weight parameter corresponding to the third target pixel increases. It should be noted that the text is usually more important than the frame, so the pixel weight can be 3.

It can be seen from the above that for different target pixel types, due to their different importance to the effect of light field display, appropriate weights can be assigned to pixels according to the importance to distinguish them from ordinary pixels.

It should be noted that the above specific types of image processing models are only examples and not limitations.

Step S105: According to each of the viewpoint images and the value of the pixel weight parameter corresponding to each pixel after being processed by the image processing model, determine the calculation order and light transmittance of the light transmittance of each pixel of each liquid crystal layer of the spatial light modulation array and in response to the light transmittance of any pixel point being determined, lock the light transmittance of the corresponding pixel point. Here, locking means that during the process of determining the value of the light transmittance of each pixel of each liquid crystal layer of the spatial light modulation array, the light transmittance of the pixel remains unchanged.

In this way, the calculation order of the light transmittance of multiple pixel points is determined based on the value of the pixel weight parameter, which can ensure that the light transmittance of the pixel point through which the light corresponding to the pixel with a high pixel weight in the viewpoint image is calculated first And locked to ensure that the reconstructed light meets its display effect in the viewpoint image, which helps to improve the display quality. In addition, when the pixel weight of a pixel is as low as zero, it indicates that the light corresponding to the pixel does not need to perform light field decomposition calculations, thereby reducing the total amount of light field data that needs to be subjected to light field decomposition, thereby helping to reduce the amount of calculation.

In some embodiments, according to each of the viewpoint images and the value of the pixel weight parameter corresponding to each pixel after being processed by the image processing model, the calculation of the light transmittance of each pixel of each liquid crystal layer of the spatial light modulation array is determined The steps of order and numerical value of light transmittance specifically include:

According to the value of the pixel weight parameter, the light weight of the light corresponding to each pixel in each viewpoint image is determined.

Those skilled in the art can understand that each pixel in the viewpoint image corresponds to a light, and the viewpoint image is perceived through the light.

Exemplarily, if the value of the pixel weight parameter is 1, the light weight of the corresponding light is 1; if the value of the pixel weight parameter is 0, the light weight of the corresponding light is 0; The light weight is 2.

According to the preset light field decomposition algorithm, the light rays and their light weights, determine the calculation sequence and the value of the light transmittance of each pixel in each liquid crystal layer; wherein, the calculation sequence is configured according to the process The maximum value among the light weights of the light rays of the pixel point is determined.

Here, the preset light field decomposition algorithm may be a light field display method based on non-negative matrix decomposition, which is not limited here.

As shown in Figure 6, multiple light rays pass through a pixel. The number of rays for most pixels can be determined according to the number of viewpoint images. Each light passes through each liquid crystal layer, and its final display effect is comprehensively determined by each pixel it passes through. It should be noted that due to the limitation of the display range of the light field, the number of rays of some pixels may be less than the number of viewpoint images.

Based on this, in order to ensure the display effect of the light field, the calculation order of the light transmittance of a pixel is determined by the maximum value of the light weights of the light rays passing through the pixel point, so as to ensure that any pixel point passing through any important light can be processed first , which is beneficial to improve the display effect.

That is to say, if the light weights of the light rays passing through a pixel point are all low, it indicates that the pixel point is not important to the light field decomposition effect, and does not need to be prioritized, or even does not participate in the light field decomposition.

In some embodiments, the step of determining the calculation order of the light transmittance and the value of the light transmittance of each pixel in each liquid crystal layer according to the preset light field decomposition algorithm, the light and its light weight, specifically include:

The ray weights of the light rays passing through the pixel are compared, and the largest ray weight is determined as the sequence value for light field decomposition of the pixel.

The selected 10 viewpoints in FIG. 3 are displayed by the spatial light modulation array shown in FIG. 6. Exemplarily, there are ten rays passing through the pixel point a, and the light weights are 0, 1, 2, 1, and 2 respectively. , 0, 1, 2, 1, 2, the corresponding sequence value is 2.

Exemplarily, there are ten light rays passing through the pixel point b (not shown in the figure) in the B liquid crystal layer in FIG. , 1, the corresponding sequence value is 3.

Exemplarily, there are ten light rays passing through the pixel point c (not shown in the figure) in the C liquid crystal layer in FIG. , 0, the corresponding sequence value is 0.

Those skilled in the art can understand that if the number of light rays passing through a certain pixel point is zero, it means that light field decomposition is not required for the pixel point, so the light transmittance of the pixel point can be directly set to 100%.

The light field decomposition sequence values of each pixel are sorted from large to small, and the corresponding calculation sequence is matched according to the sorting results.

It should be noted that the calculation order of the pixel points whose sorting result is earlier is earlier; the calculation order of the pixel points whose ranking result is later is later.

It should be understood that, for the same light field decomposition sequence value, the order of sorting is the same. That is to say, the pixel points whose light field decomposition order value is 2 all have the same calculation order.

Exemplarily, there are 10 pixel points, and their corresponding light field decomposition order values are 1, 2, 2, 3, 2, 1, 0, 1, 2, 2, respectively. The sorting result is 3, (2, 2, 2, 2, 2), (1, 1, 1), 0. The matching calculation sequence is to first calculate the pixel corresponding to 3, then the pixel corresponding to 2, 2, 2, 2, 2, then the pixel corresponding to 1, 1, 1, and finally the pixel corresponding to 0.

Based on the preset light field decomposition algorithm, the calculation order and the light rays, the light transmittance of each pixel is sequentially determined; wherein, the calculation order of the light transmittance of each pixel in the same order is randomly determined.

Optionally, in response to the sequence value corresponding to the last pixel point being zero, the light field decomposition calculation is directly terminated. Here, according to the order value of zero, it can be determined that the light weight is zero, indicating that each light passing through the pixel is not important to the light field decomposition, and light field decomposition is not required. Optionally, the transmittance of the corresponding pixel can be directly set to 100%.

In some embodiments, the number of the liquid crystal layer is three layers, and the preset light field decomposition algorithm includes the calculation formula of formula (1):

是第j条重构光线且

a是更新前的像素透光率，a′更新后的像素透光率；Among them, L _j is the jth ray passing through the pixel point a of any liquid crystal layer; b _j and c _j are the intersection pixels of the light L _j and the other two liquid crystal layers except the liquid crystal layer where the pixel point a is located; V is The number of rays passing through pixel a;

is the jth reconstructed ray and

a is the transmittance of the pixel before updating, and a' is the transmittance of the pixel after updating;

Randomly initialize the light transmittance of each pixel of each liquid crystal layer, iteratively calculate a' according to formula (1) until it meets the predetermined end rule, then the corresponding a' is the light transmittance of pixel a and locked.

Optionally, the predetermined end rule is that the absolute value of the difference between the light transmittance of the pixel before and after the update is the smallest.

It should be understood that if one wants to calculate the light transmittance of a certain pixel point, an iterative calculation according to the formula (1) is sufficient.

Exemplarily, the number of viewpoint images screened by means of edge-aware data may be 10; the number of unfiltered viewpoint map images may be 49.

It should be noted that, generally, the value of V is equal to the value of the viewpoint image. Due to the limitation of the display range of the light field, the amount of light rays of some pixels may be less than the amount of viewpoint images, for example, the edge portion of the liquid crystal layer near the user side.

Here, in the first calculation, the light transmittance of each pixel is obtained by random initialization. When the light transmittance of a certain pixel is determined and locked, in the process of calculating the light transmittance of the remaining pixels, the locked light transmittance is directly used to participate in the calculation of formula (1). This method can ensure that the light transmittance of the remaining pixels is determined under the condition of first calculating the display effect of the light emitted by the pixels, so that the light transmittance value of the pixels is more accurate, which is conducive to obtaining better display quality.

Optionally, with reference to FIG. 6, the present disclosure also exemplifies the derivation process of formula (1) as follows:

The three liquid crystal layers are represented by A layer, B layer and C layer respectively. The layer A is taken as an example for illustration, and the pixel point a on it is taken as an example for illustration.

Update pixel values by differentiation:

代入，得到：Will

Substituting in, we get:

是第j条重构光线且

a是更新前的像素透光率，a′更新后的像素透光率。L _j is the jth ray passing through pixel point a; b _j and c _j are the intersection pixel points of light L _j and B liquid crystal layer and C liquid crystal layer; V is the number of light rays passing through pixel point a;

is the jth reconstructed ray and

a is the light transmittance of the pixel before updating, and a' is the light transmittance of the pixel after updating.

Step S107: Based on the transmittance of each pixel of each liquid crystal layer of the spatial light modulation array, control the spatial light modulation array to display a reconstructed light field.

Since a single LCD pixel can correspond to multiple light rays, for example, the reconstructable light rays of a 1080×720 pixel image are within the range of (1080×720) ² . number. According to this disclosure, the viewpoint image is optimized by using the edge sensor and the image processing model. The total number of reconstructed rays after optimization is assumed to be 2×5×720×480, and the average load of each pixel is:

Wherein, w represents the number of horizontal pixels in the liquid crystal layer, and h represents the number of vertical pixels in the liquid crystal layer.

It should be noted that the reduction of the number of viewpoint images is realized by an edge sensor, and the reduction of pixels in the viewpoint images is realized by an image processing model, such as a background removal model.

It can be seen that the optimization of the light field reconstruction in the light field display method of the present disclosure can greatly reduce the payload, thereby improving the output image quality and helping to enhance the immersion of the three-dimensional effect.

In some embodiments, the light field display method acquires light field data, for example, the number of viewpoints is 7x7, and the viewpoint image resolution is 1080x720; optionally, the light field display method can also obtain edge perception data, and the edge perception data is passed through the camera Acquisition, the specific effective area is 1000 in width, 700 in height, and the pixels in the two directions are 540 and 360 respectively; algorithm strategies for processing light field data can be obtained at the same time, such as viewpoint screening algorithm, background removal model algorithm, and frame recognition model algorithm. When the light field display method obtains the above-mentioned light field data, edge-aware data and processing algorithm strategy, the adaptation strategy configuration is listed as follows:

It should be noted that the methods in the embodiments of the present disclosure may be executed by a single device, such as a computer or a server. The method of this embodiment can also be applied in a distributed scenario, and is completed by cooperation of multiple devices. In the case of such a distributed scenario, one of the multiple devices may only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete all described method.

It should be noted that some embodiments of the present disclosure are described above. Other implementations are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in an order different from those in the above-described embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain embodiments.

The embodiment of the present disclosure also provides a display method. Specifically, the display method includes:

Acquiring light field data; wherein, the light field data includes a plurality of viewpoint images; the viewpoint images include pixel weight parameters;

Using an image processing model to process the viewpoint image, determining target pixels and non-target pixels according to the processing results; and updating the value of the pixel weight parameter of the target pixel;

According to each of the viewpoint images and the value of the pixel weight parameter corresponding to each pixel after being processed by the image processing model, the calculation sequence and the value of the light transmittance of each pixel of each liquid crystal layer of the spatial light modulation array are determined.

In some embodiments, the image processing model includes at least one of a background removal model, a frame recognition model, and a character recognition model; the method includes at least one of the following steps:

Based on the background removal model, it is determined that the background pixel in the viewpoint image is a first target pixel; the value of the pixel weight parameter corresponding to the first target pixel is reduced;

Based on the frame identification model, determining that frame pixels in the viewpoint image are second target pixels; the value of the pixel weight parameter corresponding to the second target pixel is increased; and

Based on the character recognition model, it is determined that the character pixel in the viewpoint image is the third target pixel; the value of the pixel weight parameter corresponding to the third target pixel increases.

In some embodiments, according to each of the viewpoint images and the value of the pixel weight parameter corresponding to each pixel after being processed by the image processing model, the calculation of the light transmittance of each pixel of each liquid crystal layer of the spatial light modulation array is determined The steps of order and numerical value of light transmittance specifically include:

determining the light weight of the light corresponding to each pixel in each of the viewpoint images according to the value of the pixel weight parameter;

According to the preset light field decomposition algorithm, the light rays and their light weights, determine the calculation order of the light transmittance and the value of the light transmittance of each pixel in each liquid crystal layer; and respond to the transmittance of any pixel point The light rate is determined, and the light transmittance of the corresponding pixel is locked;

Wherein, the calculation order is configured to be determined according to the maximum value among the light weights of the light rays passing through the pixel point.

In some embodiments, the step of determining the calculation order of the light transmittance and the value of the light transmittance of each pixel in each liquid crystal layer according to the preset light field decomposition algorithm, the light and its light weight, specifically include:

comparing the light weights of the light rays passing through the pixel, and determining the maximum light weight as the light field decomposition sequence value of the pixel;

Sorting the light field decomposition order values of each pixel point from large to small, and matching the corresponding calculation order according to the sorting results;

Based on the preset light field decomposition algorithm, the calculation order and the light rays, the light transmittance of each pixel is sequentially determined; wherein, the calculation order of the light transmittance of each pixel in the same order is randomly determined.

In some embodiments, the method also includes:

In response to the sequence value corresponding to the last sequence of pixels being zero, the light field decomposition calculation is directly terminated.

In some embodiments, the number of the liquid crystal layer is three layers, and the preset light field decomposition algorithm includes the calculation formula of formula (1):

是第j条重构光线且

a是更新前的像素透光率，a′更新后的像素透光率；Among them, L _j is the jth ray passing through the pixel point a of any liquid crystal layer; b _j and c _j are the intersection pixels of the light L _j and the other two liquid crystal layers except the liquid crystal layer where the pixel point a is located; V is The number of rays passing through pixel a;

is the jth reconstructed ray and

a is the transmittance of the pixel before updating, and a' is the transmittance of the pixel after updating;

Randomly initialize the light transmittance of each pixel of each liquid crystal layer, iteratively calculate a' according to formula (1) until it meets the predetermined end rule, then the corresponding a' is the light transmittance of pixel a and locked.

In some embodiments, before the step of using an image processing model to process the viewpoint image, the method further includes:

The edge sensor is used to acquire edge perception data, and based on the edge perception data, the light field data is filtered to determine a target viewpoint image.

In some embodiments, the edge-aware data includes a camera image; the camera image includes a plurality of regions and each region matches a viewpoint image;

The methods include:

Using a face recognition model to identify the face in the camera image and determine the area of the face in the camera image;

According to the region, the viewpoint images corresponding to the region and its surrounding regions are determined as target viewpoint images.

Based on the same inventive concept, the present disclosure further provides a light field display device corresponding to the methods in any of the foregoing embodiments.

Referring to FIG. 8, the light field display device includes: the light field display device includes a spatial light modulation array 81 and a processor 82; the spatial light modulation array includes at least two liquid crystal layers (811, 812, 813); The liquid crystal layer includes pixel points arranged in an array;

The processor 82 is configured to perform the following steps:

Acquiring light field data; wherein, the light field data includes a plurality of viewpoint images; the viewpoint images include pixel weight parameters;

Using an image processing model to process the viewpoint image, determining target pixels and non-target pixels according to the processing results; and updating the value of the pixel weight parameter of the target pixel;

According to each of the viewpoint images and the value of the pixel weight parameter corresponding to each pixel after being processed by the image processing model, the calculation sequence and the value of the light transmittance of each pixel of each liquid crystal layer of the spatial light modulation array are determined.

In some embodiments, the light field display device includes a backlight panel 84 for providing backlight.

In some embodiments, the image processing model includes at least one of a background removal model, a frame recognition model, and a character recognition model; the processor 82 is configured to perform at least one of the following steps:

Based on the background removal model, it is determined that the background pixel in the viewpoint image is a first target pixel; the value of the pixel weight parameter corresponding to the first target pixel is reduced;

Based on the frame identification model, determining that frame pixels in the viewpoint image are second target pixels; the value of the pixel weight parameter corresponding to the second target pixel is increased; and

Based on the character recognition model, it is determined that the character pixel in the viewpoint image is the third target pixel; the value of the pixel weight parameter corresponding to the third target pixel increases.

In some embodiments, the processor 82 is configured to perform the following steps:

determining the light weight of the light corresponding to each pixel in each of the viewpoint images according to the value of the pixel weight parameter;

According to the preset light field decomposition algorithm, the light rays and their light weights, determine the calculation order of the light transmittance and the value of the light transmittance of each pixel in each liquid crystal layer; and respond to the transmittance of any pixel point The light rate is determined, and the light transmittance of the corresponding pixel is locked;

Wherein, the calculation order is configured to be determined according to the maximum value among the light weights of the light rays passing through the pixel point.

In some embodiments, the processor 82 is configured to perform the following steps:

comparing the light weights of the light rays passing through the pixel, and determining the maximum light weight as the light field decomposition sequence value of the pixel;

Sorting the light field decomposition order values of each pixel point from large to small, and matching the corresponding calculation order according to the sorting results;

Based on the preset light field decomposition algorithm and the calculation sequence, the light transmittance of each pixel is sequentially determined; wherein, the calculation sequence of the light transmittance of each pixel in the same sequence is randomly determined.

In some embodiments, the processor 82 is configured to perform the following steps:

In response to the sequence value corresponding to the last sequence of pixels being zero, the light field decomposition calculation is directly terminated.

In some embodiments, the number of the liquid crystal layer is three layers, and the preset light field decomposition algorithm includes the calculation formula of formula (1):

是第j条重构光线且

a是更新前的像素透光率，a′更新后的像素透光率；Among them, L _j is the jth ray passing through the pixel point a of any liquid crystal layer; b _j and c _j are the intersection pixels of the light L _j and the other two liquid crystal layers except the liquid crystal layer where the pixel point a is located; V is The number of rays passing through pixel a;

is the jth reconstructed ray and

a is the transmittance of the pixel before updating, and a' is the transmittance of the pixel after updating;

The processor 82 is configured to perform the following steps:

Randomly initialize the light transmittance of each pixel of each liquid crystal layer, iteratively calculate a' according to formula (1) until it meets the predetermined end rule, then the corresponding a' is the light transmittance of pixel a and locked.

In some embodiments, the display device includes an edge sensor 83;

The processor 82 is configured to perform the following steps:

The edge sensor is used to acquire edge perception data, and based on the edge perception data, the light field data is filtered to determine a target viewpoint image.

In some embodiments, the edge-aware data includes a camera image; the camera image includes a plurality of regions and each region matches a viewpoint image;

The processor 82 is configured to perform the following steps:

Using a face recognition model to identify the face in the camera image and determine the area of the face in the camera image;

According to the region, the viewpoint images corresponding to the region and its surrounding regions are determined as target viewpoint images.

The device in the above embodiment is used to implement the corresponding display method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, so it will not be repeated here.

Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is exemplary only, and is not intended to imply that the scope of the present disclosure (including claims) is limited to these examples; under the idea of the present disclosure, the above embodiments or Combinations between technical features in different embodiments are also possible, steps may be implemented in any order, and there are many other variations of the different aspects of the disclosed embodiments as described above, which are not provided in detail for the sake of brevity.

In addition, for simplicity of illustration and discussion, and so as not to obscure the embodiments of the present disclosure, well-known power/supply circuits associated with integrated circuit (IC) chips and other components may or may not be shown in the provided figures. ground connection. Furthermore, devices may be shown in block diagram form in order to avoid obscuring the embodiments of the disclosure, and this also takes into account the fact that details regarding the implementation of these block diagram devices are highly dependent on the implementation of the embodiments of the disclosure to be implemented. platform (ie, the details should be well within the purview of those skilled in the art). Where specific details (eg, circuits) have been set forth to describe example embodiments of the present disclosure, it will be apparent to those skilled in the art that other applications may be made without or with variations from these specific details. Embodiments of the present disclosure are implemented below. Accordingly, these descriptions should be regarded as illustrative rather than restrictive.

Although the disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications and variations of those embodiments will be apparent to those of ordinary skill in the art from the foregoing description. For example, other memory architectures such as dynamic RAM (DRAM) may use the discussed embodiments.

The disclosed embodiments are intended to embrace all such alterations, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure shall fall within the protection scope of the present disclosure.

Claims (14)

1. A light field display method, the display method comprising:

acquiring light field data; wherein the light field data comprises a plurality of view images; the viewpoint image comprises pixel weight parameters;

processing the viewpoint image by using an image processing model, and determining target pixels and non-target pixels according to a processing result; updating the value of the pixel weight parameter of the target pixel;

and determining the calculation sequence of the light transmittance and the light transmittance value of each pixel point of each liquid crystal layer of the spatial light modulation array according to each viewpoint image and the value of the pixel weight parameter corresponding to each pixel after being processed by the image processing model.

2. The light field display method of claim 1, wherein the image processing model comprises at least one of a background removal model, a border recognition model, and a text recognition model; the method comprises at least one of the following steps:

determining a background pixel in the viewpoint image as a first target pixel based on the background removal model; the value of the pixel weight parameter corresponding to the first target pixel is reduced;

based on the frame recognition model, determining that frame pixels in the viewpoint image are second target pixels; the value of the pixel weight parameter corresponding to the second target pixel is increased; and

determining that text pixels in the viewpoint image are third target pixels based on the text recognition model; and the value of the pixel weight parameter corresponding to the third target pixel is increased.

3. The method of claim 1, wherein the step of determining the calculation sequence of the light transmittance and the value of the light transmittance of each pixel point of each liquid crystal layer of the spatial light modulation array according to each viewpoint image and the value of the pixel weight parameter corresponding to each pixel processed by the image processing model specifically comprises:

Determining the light weight of the light corresponding to each pixel in each viewpoint image according to the numerical value of the pixel weight parameter;

determining the calculation sequence of the light transmittance and the value of the light transmittance of each pixel point in each liquid crystal layer according to a preset light field decomposition algorithm, the light rays and the light ray weights thereof; and in response to the light transmittance of any one of the pixel points being determined, locking the light transmittance of the corresponding pixel point;

wherein the calculation order is configured to be determined from a maximum value of ray weights of the rays passing through the pixel point.

4. The light field display method according to claim 3, wherein the step of determining the calculation sequence of the light transmittance and the value of the light transmittance of each pixel point in each liquid crystal layer according to a preset light field decomposition algorithm, the light rays and the light ray weights thereof specifically comprises:

comparing the light weights of the light rays passing through the pixel points, and determining the maximum light weight as a light field decomposition sequence value of the pixel points;

sorting the light field decomposition sequence values of the pixel points from large to small, and matching the corresponding calculation sequence according to the sorting result;

sequentially determining the value of the light transmittance of each pixel point based on the preset light field decomposition algorithm, the calculation sequence and the light rays; the calculation sequence of the light transmittance of each pixel point in the same sequence is randomly determined.

5. The light field display method of claim 4, wherein the method further comprises:

and in response to the sequence value corresponding to the pixel point of the last sequence being zero, directly terminating the light field decomposition calculation.

6. The light field display method according to claim 4, wherein the number of the liquid crystal layers is three, and the preset light field decomposition algorithm includes a calculation formula of formula (1):

wherein L is _j The j-th ray passes through the pixel point a of any liquid crystal layer; b _j And c _j Is light L _j Intersection pixel points of the liquid crystal layer and the other two liquid crystal layers except the liquid crystal layer where the pixel point a is located; v is the number of light rays passing through the pixel point a;

is the j th reconstruction ray and

a is the pixel transmittance before updating and a' is the pixel transmittance after updating;

randomly initializing the light transmittance of each pixel point of each liquid crystal layer, and iteratively calculating a 'according to a formula (1) until the light transmittance meets a preset ending rule, wherein the corresponding a' is the light transmittance of the pixel point a and is locked.

7. The light field display method of claim 1, wherein prior to the step of processing the viewpoint image using an image processing model, the method further comprises:

and acquiring edge perception data by using an edge sensor, filtering the light field data based on the edge perception data, and determining a target viewpoint image.

8. The light field display method of claim 7, wherein the edge-aware data comprises a camera image; the camera image comprises a plurality of areas, and each area is matched with a viewpoint image;

the method comprises the following steps:

recognizing a face in the camera image by using a face recognition model and determining a region of the face in the camera image;

and according to the region, determining the viewpoint image corresponding to the region and the surrounding region as a target viewpoint image.

9. A light field display device, comprising a spatial light modulation array and a processor; the spatial light modulation array comprises at least two liquid crystal layers; the liquid crystal layer comprises pixel points arranged in an array;

the processor is configured to perform the steps of:

acquiring light field data; wherein the light field data comprises a plurality of view images; the viewpoint image comprises pixel weight parameters;

processing the viewpoint image by using an image processing model, and determining target pixels and non-target pixels according to a processing result; updating the value of the pixel weight parameter of the target pixel;

and determining the calculation sequence of the light transmittance and the light transmittance value of each pixel point of each liquid crystal layer of the spatial light modulation array according to each viewpoint image and the value of the pixel weight parameter corresponding to each pixel after being processed by the image processing model.

10. The light field display device of claim 9, wherein the image processing model comprises at least one of a background removal model, a border recognition model, and a text recognition model; the processor is configured to perform at least one of the following steps:

determining a background pixel in the viewpoint image as a first target pixel based on the background removal model; the value of the pixel weight parameter corresponding to the first target pixel is reduced;

based on the frame recognition model, determining that frame pixels in the viewpoint image are second target pixels; the value of the pixel weight parameter corresponding to the second target pixel is increased; and

determining that text pixels in the viewpoint image are third target pixels based on the text recognition model; and the value of the pixel weight parameter corresponding to the third target pixel is increased.

11. The light field display device of claim 9, wherein the processor is configured to perform the steps of:

determining the light weight of the light corresponding to each pixel in each viewpoint image according to the numerical value of the pixel weight parameter;

determining the calculation sequence of the light transmittance and the value of the light transmittance of each pixel point in each liquid crystal layer according to a preset light field decomposition algorithm, the light rays and the light ray weights thereof; and in response to the light transmittance of any one of the pixel points being determined, locking the light transmittance of the corresponding pixel point;

Wherein the calculation order is configured to be determined from a maximum value of ray weights of the rays passing through the pixel point.

12. The light field display device of claim 11, wherein the number of liquid crystal layers is three, and the preset light field decomposition algorithm comprises a calculation formula of formula (1):

wherein L is _j The j-th ray passes through the pixel point a of any liquid crystal layer; b _j And c _j Is light L _j Intersection pixel points of the liquid crystal layer and the other two liquid crystal layers except the liquid crystal layer where the pixel point a is located; v is the number of light rays passing through the pixel point a;

is the j th reconstruction ray and

a is before updatingThe pixel transmittance of a' after updating;

the processor is configured to perform the steps of:

randomly initializing the light transmittance of each pixel point of each liquid crystal layer, and iteratively calculating a 'according to a formula (1) until the light transmittance meets a preset ending rule, wherein the corresponding a' is the light transmittance of the pixel point a and is locked.

13. The light field display device of claim 9, wherein the display device comprises an edge sensor;

the processor is configured to perform the steps of:

and acquiring edge perception data by using the edge perceptron, filtering the light field data based on the edge perception data, and determining a target viewpoint image.

14. The light field display device of claim 9, wherein the edge-aware data comprises a camera image; the camera image comprises a plurality of areas, and each area is matched with a viewpoint image;

the processor is configured to perform the steps of:

recognizing a face in the camera image by using a face recognition model and determining a region of the face in the camera image;

and according to the region, determining the viewpoint image corresponding to the region and the surrounding region as a target viewpoint image.

Images