CN115002444B - Display module and display method thereof, display device, and virtual display equipment - Google Patents

Abstract

The embodiment of the disclosure provides a display method, which comprises the following steps: disassembling a frame of color picture into n sub-frame pictures; n sub-frame pictures are sequentially displayed, n is more than or equal to 3, and n is an integer; acquiring an eye image, processing the eye image, and determining the current pupil position corresponding to the first subframe picture; the first subframe is a subframe which is firstly displayed when n subframes are sequentially displayed; calculating the current pupil position corresponding to each subsequent subframe picture of the first subframe picture in the n subframe pictures according to the current pupil position corresponding to the first subframe picture and the pupil position corresponding to the first subframe picture obtained by the previous m times of measurement; m is more than or equal to 2, and m is an integer; calculating the actual offset of the original display position of each subsequent subframe picture relative to the current position of the first subframe picture; calculating the actual display position of each subsequent subframe picture according to the actual offset; and displaying the subsequent sub-frame pictures in turn according to the actual display positions of the subsequent sub-frame pictures.

Description

Display module and display method thereof, display device, and virtual display equipment

Technical Field

The present disclosure relates to the field of display technology, and in particular to a display module and a display method thereof, a display device, and a virtual display equipment.

Background technique

Virtual reality (VR) display devices use VR display devices to block people's vision and hearing from the outside world, and guide users to feel like they are in a virtual environment. The display principle is to use computer technology to simulate a three-dimensional, highly simulated 3D space. When the user wears a VR head display device, it will produce an illusion as if they are in reality. In this space, the operator can use a controller or keyboard to shuttle or interact in this virtual environment.

Summary of the invention

In one aspect, an embodiment of the present disclosure provides a display method, which includes:

Decomposing a color picture frame into n sub-frame pictures; displaying the n sub-frame pictures in sequence, n≥3, and n is an integer;

Acquire an eye image, process the eye image, and determine a current pupil position corresponding to a first sub-frame image; the first sub-frame image is a sub-frame image that is first displayed when the n sub-frame images are displayed in sequence;

Calculate the current pupil position corresponding to each subsequent sub-frame picture of the first sub-frame picture in the n sub-frame pictures according to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by the previous m measurements; m≥2, and m is an integer;

Calculating the actual offset of the original position to be displayed of each subsequent sub-frame picture relative to the current position of the first sub-frame picture according to the offset of the current pupil position corresponding to each subsequent sub-frame picture in the n sub-frame pictures relative to the current pupil position corresponding to the first sub-frame picture; and calculating the actual display position of each subsequent sub-frame picture according to the original position to be displayed of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture;

The subsequent sub-frame images are displayed in sequence according to their actual display positions.

In some embodiments, acquiring the eye image, processing the eye image, and determining the current pupil position corresponding to the first subframe image includes:

When the first subframe is displayed, capturing an eye image;

Convert the eye image to a grayscale image;

In the grayscale image, the left and right eye corners are detected according to the eye corner feature points; the line connecting the left and right eye corners is used as the X-axis, the axis perpendicular to the X-axis is used as the Y-axis, and the origin of the intersection of the X-axis and the Y-axis is the midpoint of the line connecting the left and right eye corners;

The grayscale image is processed in a coordinate plane formed by an X-axis and a Y-axis to determine a pupil area of the eye; and the center of the pupil area is determined as a current pupil position corresponding to the first subframe image.

In some embodiments, the processing of the grayscale image in a coordinate plane formed by an X-axis and a Y-axis to determine a pupil area of the eye; and determining the center of the pupil area as a current pupil position corresponding to the first subframe image include:

Binarize the grayscale image to obtain a binary image of the eye;

Using a connected region labeling method to detect candidate pupil connected regions on the binary image of the eye;

Based on the geometric constraint and distance constraint algorithm, the pupil region of the eye is screened out in the candidate pupil connected region;

The pupil area is covered with a circle with the smallest diameter, and the center of the circle is determined as the center of the pupil area.

In some embodiments, n=3, m=2;

Subsequent sub-frame pictures of the first sub-frame picture include a second sub-frame picture and a third sub-frame picture; the first sub-frame picture, the second sub-frame picture and the third sub-frame picture are displayed in sequence;

The calculating, according to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by previous m measurements, the current pupil position corresponding to each subsequent sub-frame picture of the first sub-frame picture in the n sub-frame pictures comprises:

The current pupil position corresponding to the second sub-frame picture is calculated according to formula (1), and the current pupil position corresponding to the third sub-frame picture is calculated according to formula (2);

pos_curr_g＝pos_curr+v_curr×delta+1/2×a_curr×delta ² ； (1)

pos_curr_b＝pos_curr+v_curr×delta×2+1/2×a_curr×(2×delta)2； (2)

in,

a_curr is the current acceleration of the eyeball rotation; v_curr is the current speed of the eyeball rotation; pos_1 and pos_2 are respectively the pupil positions corresponding to the first sub-frame images obtained by the previous two measurements; pos_curr is the current pupil position corresponding to the first sub-frame image; pos_curr_g is the current pupil position corresponding to the second sub-frame image; pos_curr_b is the current pupil position corresponding to the third sub-frame image; delta is the refresh duration of one sub-frame image, delta=1/refresh frequency of one sub-frame image.

In some embodiments, the method of calculating the actual offset of the original to-be-displayed position of each subsequent sub-frame picture relative to the current position of the first sub-frame picture according to the offset of the current pupil position corresponding to each subsequent sub-frame picture in the n sub-frame pictures relative to the current pupil position corresponding to the first sub-frame picture; and calculating the actual display position of each subsequent sub-frame picture according to the original to-be-displayed position of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture, includes:

Calculate a first offset matrix of an original to-be-displayed position of the second sub-frame picture relative to a current position of the first sub-frame picture according to an offset of a current pupil position corresponding to the second sub-frame picture relative to a current pupil position corresponding to the first sub-frame picture;

Calculating a second offset matrix of an original to-be-displayed position of the third sub-frame picture relative to a current position of the first sub-frame picture according to an offset of a current pupil position corresponding to the third sub-frame picture relative to a current pupil position corresponding to the first sub-frame picture;

Multiplying the original to-be-displayed position of the second sub-frame picture by the first offset matrix to obtain the actual display position of the second sub-frame picture;

The original to-be-displayed position of the third sub-frame picture is multiplied by the second offset matrix to obtain the actual display position of the third sub-frame picture.

In some embodiments,

The first offset matrix is:

The second offset matrix is:

Wherein, Rx1, Ry1, and Rz1 respectively represent the rotation amplitude of the original position to be displayed of the second subframe image along the X-axis, Y-axis, and Z-axis relative to the current position of the first subframe image; Tx1, Ty1, and Tz1 respectively represent the translation amplitude of the original position to be displayed of the second subframe image along the X-axis, Y-axis, and Z-axis relative to the current position of the first subframe image;

Rx2, Ry2, and Rz2 respectively represent the rotation amplitude of the original position to be displayed of the third subframe image along the X-axis, Y-axis, and Z-axis relative to the current position of the first subframe image; Tx2, Ty2, and Tz2 respectively represent the translation amplitude of the original position to be displayed of the third subframe image along the X-axis, Y-axis, and Z-axis relative to the current position of the first subframe image;

The Z axis is perpendicular to a two-dimensional coordinate plane formed by the intersection of the X axis and the Y axis, and the Z axis intersects the X axis and the Y axis at the origin.

In some embodiments, the method further includes: storing the current pupil position corresponding to the first sub-frame image and the current pupil position corresponding to each subsequent sub-frame image.

In a second aspect, the present disclosure also provides a display module, which includes:

The disassembly module is configured to disassemble a color picture frame into n sub-frame pictures; n≥3, and n is an integer;

A display module, configured to display the n sub-frame images in sequence;

a processing module configured to acquire an eye image, process the eye image, and determine a current pupil position corresponding to a first sub-frame image; the first sub-frame image is a sub-frame image that is displayed first among the n sub-frame images;

A first prediction module is configured to calculate the current pupil position corresponding to each subsequent sub-frame picture of the first sub-frame picture among the n sub-frame pictures according to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by previous m measurements; m≥2, and m is an integer;

a second prediction module configured to calculate an actual offset of an original position to be displayed of each subsequent sub-frame picture relative to a current position of the first sub-frame picture according to an offset of a current pupil position corresponding to each subsequent sub-frame picture among the n sub-frame pictures relative to a current pupil position corresponding to the first sub-frame picture; and calculate an actual display position of each subsequent sub-frame picture according to the original position to be displayed of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture;

The display module is further configured to display the subsequent sub-frame images in sequence according to the actual display positions of the subsequent sub-frame images.

In some embodiments, the display module includes a display panel and a lens, wherein the lens is located on a display side of the display panel;

The processing module includes an infrared transmitter and an infrared camera.

The infrared emitter is located on a side of the lens away from the display panel and is distributed around the edges of the lens, and is used to emit infrared light to the eyes;

The infrared camera is located on a side of the lens away from the display panel and on an edge of the lens, and is used for capturing eye images.

In some embodiments, the processing module further includes an image processing unit configured to convert the eye image into a grayscale image; detect left and right eye corner position points of the eye according to the eye corner feature points in the grayscale image; connect the left and right eye corner position points and use the line as the X-axis, and use the axis perpendicular to the X-axis as the Y-axis, and the origin of the intersection of the X-axis and the Y-axis as the midpoint of the line connecting the left and right eye corner position points;

The image processing unit is further configured to process the grayscale image of the eye to determine the pupil area of the eye; and determine the center of the pupil area as the current pupil position corresponding to the first subframe image.

In some embodiments, the second prediction module is configured to calculate a first offset matrix of an original to-be-displayed position of the second sub-frame picture relative to a current position of the first sub-frame picture according to an offset of a current pupil position corresponding to the second sub-frame picture relative to a current pupil position corresponding to the first sub-frame picture;

The second prediction module is further configured to calculate a second offset matrix of an original to-be-displayed position of the third sub-frame picture relative to a current position of the first sub-frame picture according to an offset of a current pupil position corresponding to the third sub-frame picture relative to a current pupil position corresponding to the first sub-frame picture;

The second prediction module is also configured to multiply the original display position of the second sub-frame picture by the first offset matrix to obtain the actual display position of the second sub-frame picture; and multiply the original display position of the third sub-frame picture by the second offset matrix to obtain the actual display position of the third sub-frame picture.

In some embodiments, a storage module is further included, which is configured to store the current pupil position corresponding to the first sub-frame image and the current pupil position corresponding to each subsequent sub-frame image.

In a third aspect, an embodiment of the present disclosure further provides a display device, which includes the above-mentioned display module.

In a fourth aspect, an embodiment of the present disclosure further provides a virtual display device, which includes the above-mentioned display apparatus.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

FIG. 1 is a schematic diagram showing the principle of color separation when VR displays a color image in the public technology.

FIG. 2 is a picture of a scene in which color separation occurs in the disclosed technology.

FIG. 3 is a functional block diagram of a display module in an embodiment of the present disclosure.

FIG. 4 is a top view schematically showing the arrangement of pixels and light sources in a display module according to an embodiment of the present disclosure.

FIG. 5 is a schematic diagram of decomposing a color picture frame into three sub-frame pictures in an embodiment of the present disclosure.

FIG. 6 is another schematic diagram of decomposing a color picture frame into three sub-frame pictures in an embodiment of the present disclosure.

FIG. 7 is a schematic diagram showing that each sub-frame image is refreshed in sequence and light sources of different colors in a backlight module are lit in sequence during the display process of the display module according to an embodiment of the present disclosure.

FIG. 8 is a schematic diagram of the structural disassembly of the display module in the embodiment of the present disclosure.

FIG. 9 a is a schematic diagram of a grayscale image of an eye pattern.

FIG9 b is a schematic diagram of a binary image of the eye obtained by processing by the image processing unit in an embodiment of the present disclosure.

FIG. 10 is a schematic diagram of the detection frequency of the pupil position corresponding to the first sub-frame image in an embodiment of the present disclosure.

FIG. 11 is a schematic diagram of a process of predicting actual display positions of the second sub-frame picture and the third sub-frame picture in an embodiment of the present disclosure.

FIG. 12 is a flow chart of a display module display method in an embodiment of the present disclosure.

Detailed ways

In order to enable those skilled in the art to better understand the technical solutions of the embodiments of the present disclosure, the display module and its display method, display device, and virtual display device provided by the embodiments of the present disclosure are further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

The embodiments of the present disclosure will be described more fully below with reference to the accompanying drawings, but the embodiments shown may be embodied in different forms and should not be construed as being limited to the embodiments set forth in the present disclosure. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete and to enable those skilled in the art to fully understand the scope of the present disclosure.

The embodiments of the present disclosure are not limited to the embodiments shown in the drawings, but include modifications of the configurations formed based on the manufacturing process. Therefore, the regions illustrated in the drawings have schematic properties, and the shapes of the regions shown in the drawings illustrate the specific shapes of the regions, but are not intended to be limiting.

In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

In the public technology, for VR display screens, users will feel immersive only when the picture refresh rate reaches 90Hz. When using a field sequential screen as a VR screen, in order to ensure the above effects, its refresh rate needs to reach 270Hz. The field sequential screen splits the previous frame of color image (RGB image) into three sub-frame images (i.e., images with backlight provided by red, green, and blue light sources, respectively) for display. Each sub-frame image is displayed sequentially, and the backlight used to display each sub-frame image is also provided sequentially.

Referring to FIG1, there is a schematic diagram of the principle of color separation when VR displays color images in the public technology; as the human eye rotates, the pixels at the same position on the screen are backlit by red, green, and blue light sources respectively, and the red, green, and blue images displayed fall on different positions of the human eye retina respectively, causing the human eye to feel the color separation phenomenon. Referring to FIG2, there is a scene picture of color separation in the public technology; from FIG2, it can be seen that there are red, green, and blue stripes on the window frame. Analysis of the scene where color separation occurs finds that the greater the difference between the three subframes (that is, the display grayscale of the three subframes is the same and the display grayscale is not 0), the more obvious the color separation. For example: when the three subframes (that is, the subframe backlit by the red light source, the subframe backlit by the green light source, and the subframe backlit by the blue light source) display 255 grayscales respectively, the backlight of each subframe is lit in sequence, and color separation is most likely to occur when the eye scans.

In order to solve the above-mentioned problems existing in the disclosed technology, in a first aspect, an embodiment of the present disclosure provides a display method, which includes: decomposing a frame of color picture into n sub-frame pictures; displaying the n sub-frame pictures in sequence, n ≥ 3, and n is an integer; acquiring an eye image, processing the eye image, and determining a current pupil position corresponding to a first sub-frame picture; the first sub-frame picture is a sub-frame picture that is first displayed when the n sub-frame pictures are displayed in sequence; according to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by the previous m measurements, calculating the first pupil position in the n sub-frame pictures The current pupil position corresponding to each subsequent sub-frame picture of the sub-frame picture; m≥2, and m is an integer; according to the offset of the current pupil position corresponding to each subsequent sub-frame picture in the n sub-frame pictures relative to the current pupil position corresponding to the first sub-frame picture, the actual offset of the original position to be displayed of each subsequent sub-frame picture relative to the current position of the first sub-frame picture is calculated; and the actual display position of each subsequent sub-frame picture is calculated according to the original position to be displayed of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture; and the subsequent sub-frame pictures are displayed in sequence according to the actual display position of each subsequent sub-frame picture.

The display method of the display module provided in the embodiment of the present disclosure determines the current pupil position corresponding to the first sub-frame picture that is first displayed among n sub-frame pictures through processing; calculates the current pupil positions corresponding to the subsequent sub-frame pictures of the first sub-frame picture; calculates the actual display positions of the subsequent sub-frame pictures of the first sub-frame picture according to the calculation results of the current pupil positions corresponding to the subsequent sub-frame pictures of the first sub-frame picture; can obtain the rotation trajectory information of the human eyeball so as to capture and track the rotation of the human eyeball; by adjusting the subsequent sub-frame pictures of the first sub-frame picture from the original to-be-displayed position to the actual display position, it is possible to make the actual display position of the subsequent sub-frame pictures of the first sub-frame picture catch up with the position of the eyeball rotation, so that the pictures displayed by the pixels at the same position on the subsequent sub-frame pictures of the first sub-frame picture are projected to the same position in the pupil area of the human eye (i.e., on the retina), and three pictures with backlight provided by light sources of different colors are merged here, thereby eliminating the color separation phenomenon and ensuring that the picture perceived by the human eye is a complete picture.

In a second aspect, an embodiment of the present disclosure provides a display module, refer to FIG3 , which is a principle block diagram of the display module in the embodiment of the present disclosure; wherein the display module includes: a disassembly module, configured to disassemble a frame of color picture into n sub-frame pictures; n ≥ 3, and n is an integer; a display module, configured to display the n sub-frame pictures in sequence; a processing module, configured to obtain an eye image, process the eye image, and determine the current pupil position corresponding to the first sub-frame picture; the first sub-frame picture is the sub-frame picture that is first displayed when the n sub-frame pictures are displayed in sequence; a first prediction module, configured to predict the pupil position corresponding to the first sub-frame picture according to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by the previous m measurements. , calculate the current pupil position corresponding to each subsequent sub-frame picture of the first sub-frame picture in the n sub-frame pictures; m≥2, and m is an integer; the second prediction module is configured to calculate the actual offset of the original to-be-displayed position of each subsequent sub-frame picture relative to the current position of the first sub-frame picture according to the offset of the current pupil position corresponding to each subsequent sub-frame picture in the n sub-frame pictures relative to the current pupil position corresponding to the first sub-frame picture; and calculate the actual display position of each subsequent sub-frame picture according to the original to-be-displayed position of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture; the display module is also configured to display each subsequent sub-frame picture in sequence according to the actual display position of each subsequent sub-frame picture.

In some embodiments, referring to FIG. 4, it is a top view schematic diagram of the arrangement of pixels and light sources in the display module of the embodiment of the present disclosure; wherein, the display module includes a display panel 1, and the display panel 1 includes a plurality of pixels 10, and the plurality of pixels 10 are arranged in an array. In this embodiment, referring to FIG. 5 and FIG. 6, FIG. 5 is a schematic diagram of disassembling a frame of color picture into three sub-frame pictures in the embodiment of the present disclosure; FIG. 6 is another schematic diagram of disassembling a frame of color picture into three sub-frame pictures in the embodiment of the present disclosure; a frame of color picture is disassembled into three sub-frame pictures, and the three sub-frame pictures are displayed in sequence by the array of pixels 10 respectively.

In some embodiments, the display module further includes a lens, which is located on the display side of the display panel 1. In some embodiments, referring to FIG. 4 , the display module further includes a backlight module 5, which is located on the side of the display panel 1 away from the lens, and is used to provide backlight for the display of the display panel 1; the backlight module 5 includes a light source of n colors; the display panel 1 includes an upper substrate and a lower substrate, and the upper substrate and the lower substrate are aligned to form an alignment gap, and the alignment gap is filled with liquid crystal.

In some embodiments, the backlight module 5 includes three colors of light sources, namely a red light source 51, a green light source 52 and a blue light source 53. The red light source 51, the green light source 52 and the blue light source 53 respectively provide backlight for three sub-frame images displayed sequentially.

In some embodiments, the backlight module 5 provides direct backlight for the display of the display panel 1 , that is, the light emitting surface of the light source faces the display surface of the display panel 1 .

In some embodiments, referring to FIG. 4 , the display area of the display panel 1 includes a plurality of sub-areas 100 , each of which includes a plurality of pixels 10 ; the plurality of pixels 10 are arranged in an array; the backlight module 5 includes a plurality of light source groups 50 , each of which includes one light source of n colors; the plurality of light source groups 50 are arranged in one-to-one correspondence with the plurality of sub-areas 100 .

In this embodiment, referring to FIG. 4 and FIG. 7, FIG. 7 is a schematic diagram of the sequential refreshing of each sub-frame image and the sequential lighting of light sources of different colors in the backlight module during the display process of the display module of the embodiment of the present disclosure; the backlight of each sub-area 100 is provided by a group of light sources 50 composed of red, green and blue light sources (such as LED lights). The light sources of different colors in the backlight module can be lit in sequence, such as red, green and blue light sources are lit in sequence, to provide backlight for the three sub-frame images displayed in sequence. The brightness of the light sources of different colors in the backlight module can be adjusted. Unlike the liquid crystal panel composed of one pixel composed of red, green and blue sub-pixels in the public technology, each pixel 10 of the field sequential display module in the embodiment of the present disclosure corresponds to an overall pixel area, and the pixel 10 can only display the grayscale image through the deflection of the liquid crystal in the display of each sub-frame image; the pixel 10 array in the display module cooperates with the sequential refreshing of the three sub-frame images and the sequential lighting of the three color light sources in the backlight module to realize the display of the color image. The design of the pixel 10 in this embodiment increases the display aperture ratio and reduces the display power consumption.

In some embodiments, referring to FIG7 , the display module generates a 90Hz original color picture. In this embodiment, the field sequential display of the display module simulates a refresh rate of 90Hz, and each frame of the original color picture is decomposed into three sub-frame pictures. In fact, the pixel value (i.e., grayscale value) of the corresponding color is extracted from the original color picture, that is, the pixel value of the first sub-frame picture provided by the red light source 51 (e.g., the pixel value is 167), the pixel value of the second sub-frame picture provided by the green light source 52 (e.g., the pixel value is 145), and the pixel value of the third sub-frame picture provided by the blue light source 53 (e.g., the pixel value is 189) is extracted from one frame of the original color picture; referring to FIG6 , the three sub-frame pictures corresponding to the original one frame of color picture after being decomposed, i.e., the R, G, and B three-channel pictures. The R, G, and B three-channel pictures are displayed in sequence, and the three color light sources in the backlight module are also lit in sequence to provide different colors of backlight for the three sub-frame pictures respectively.

In some embodiments, the refresh frequencies of the three sub-frame images are 270 Hz respectively, and the lighting frequencies of the three color light sources in the backlight module are 270 Hz respectively.

In some embodiments, refer to Figure 8, which is a schematic diagram of the structural disassembly of the display module in the embodiment of the present disclosure; wherein, the display module also includes a lens 2, and the lens 2 is located on the display side of the display panel 1; the processing module includes an infrared transmitter 3 and an infrared camera 4, and the infrared transmitter 3 is located on the side of the lens 2 away from the display panel 1, and is distributed around the edges of the lens 2, and is used to emit infrared light to the eye; the infrared camera 4 is located on the side of the lens 2 away from the display panel 1, and is located at the edge of the lens 2, and is used to capture eye images.

The function of the lens 2 is to introduce distortion to realize virtual reality display. There are multiple infrared emitters 3, and the multiple infrared emitters 3 are arranged around the edges of the lens 2. One infrared camera 4 is provided. The infrared emitter 3 emits infrared light to the eye, so that the infrared camera 4 can capture a clear infrared image of the eye.

In some embodiments, referring to Figure 3, the processing module also includes an image processing unit, which is configured to convert the eye image into a grayscale image; detect the left and right eye corner position points of the eye according to the eye corner feature points in the grayscale image; connect the left and right eye corner position points and use them as the X-axis, and the axis perpendicular to the X-axis as the Y-axis, and the origin of the intersection of the X-axis and the Y-axis is the midpoint of the line connecting the left and right eye corner position points; the image processing unit is also configured to process the grayscale image of the eye to determine the pupil area of the eye; and determine the center of the pupil area as the current pupil position corresponding to the first subframe image.

Wherein, referring to Fig. 9a, it is a schematic diagram of a grayscale image of an eye figure; the eye image taken by the infrared camera is a color picture; since the color component in the eye image is not needed in the process of determining the pupil area of the eye, the eye image is converted into a grayscale image, that is, only the brightness value (grayscale value) of the eye image is taken into account; such as: the calculation formula of the grayscale image can be: gray = 0.30R + 0.60G + 0.1B; wherein gray is the grayscale value of each pixel in the grayscale image; assuming that the pixel includes R (red) sub-pixel, G (green) sub-pixel and B (blue) sub-pixel, the grayscale value of each pixel in the grayscale image is calculated by taking into account 30% of the grayscale value of the R sub-pixel, 60% of the grayscale value of the G sub-pixel and 10% of the grayscale value of the B sub-pixel, thereby obtaining a grayscale image of the entire eye image. Referring to Figure 9a, the left and right eye corner position points eye_edeg_l and eye_edge_r are detected in the grayscale image according to the eye corner feature points, and the grayscale image is cropped with these two position points as the left and right edges; the line connecting the left and right eye corner position points eye_edeg_l and eye_edeg_r is used as the X-axis, the axis perpendicular to the X-axis is used as the Y-axis, and the origin where the X-axis and the Y-axis intersect is the midpoint of the line connecting the left and right eye corner position points eye_edeg_l and eye_edeg_r.

In some embodiments, refer to Figure 9b, which is a schematic diagram of the pupil area of the eye obtained by the image processing unit in the embodiment of the present disclosure; wherein the image processing unit is further configured to process the grayscale image of the eye to determine the pupil area of the eye; the center of the pupil area is determined as the current pupil position corresponding to the first subframe image, and the specific processing process of the image processing unit is: first, the grayscale image is binarized to obtain a binary image of the eye; then, the connected region marking method is used to detect candidate pupil connected regions on the binary image of the eye; then, based on the geometric constraint and distance constraint algorithm, the pupil area of the eye is screened out from the candidate pupil connected regions; finally, the pupil area is covered with a circle with the smallest diameter, and the center of the circle is determined as the center of the pupil area 0_t.

In some embodiments, binarization of a grayscale image includes: setting a grayscale threshold t, judging the grayscale value of each sub-pixel in the grayscale image, setting the grayscale value of the sub-pixel to 0 if it is greater than the grayscale threshold t, and setting the grayscale value of the sub-pixel to 1 if it is less than or equal to the grayscale threshold t, and after binarization, a binary image of the eye (such as a black and white image) as shown in Figure 9b is formed.

In some embodiments, the processing module may be an image processing chip in a display module, and an image processing unit is integrated in the image processing chip.

In some embodiments, n=3, m=2; the subsequent sub-frame pictures of the first sub-frame picture include the second sub-frame picture and the third sub-frame picture; the display module is configured to display the first sub-frame picture, the second sub-frame picture and the third sub-frame picture in sequence; the first prediction module is configured to calculate the current pupil position corresponding to the second sub-frame picture according to formula (1), and calculate the current pupil position corresponding to the third sub-frame picture according to formula (2).

pos_curr_g＝pos_curr+v_curr×delta+1/2×a_curr×delta ² ； (1)

pos_curr_b＝pos_curr+v_curr×delta×2+1/2×a_curr×(2×delta)2； (2)

in,

a_curr is the current acceleration of the eyeball rotation; v_curr is the current speed of the eyeball rotation; pos_1 and pos_2 are the pupil positions corresponding to the first sub-frame images obtained in the previous two measurements; pos_curr is the current pupil position corresponding to the first sub-frame image; pos_curr_g is the current pupil position corresponding to the second sub-frame image; pos_curr_b is the current pupil position corresponding to the third sub-frame image; delta is the refresh duration of a sub-frame image, delta = 1/refresh frequency of a sub-frame image.

In some embodiments, the first subframe picture is a subframe picture with backlight provided by a red light source; the second subframe picture is a subframe picture with backlight provided by a green light source; and the third subframe picture is a subframe picture with backlight provided by a blue light source.

In some embodiments, referring to FIG. 10 , there is shown a schematic diagram of the detection frequency of the pupil position corresponding to the first sub-frame image in the disclosed embodiment; wherein n=3, i.e., the pupil position of the eye is detected once every two sub-frame images and the actual display positions of the second sub-frame image and the third sub-frame image are predicted.

In some embodiments, the refresh frequency of a sub-frame image is 270 Hz, and the refresh duration of a sub-frame image is delta=1/270=0.0037 s. The lighting frequency of the light source in the backlight module providing backlight for each sub-frame image is 270 Hz.

In some embodiments, the pupil position corresponding to the first sub-frame image obtained by the previous m measurements is included in the calculation of the current pupil position corresponding to the subsequent sub-frame images of the first sub-frame image in the n sub-frame images, thereby obtaining the rotation trajectory information of the human eye, thereby facilitating the capture and tracking of the rotation of the human eye.

In some embodiments, the first prediction module is a hardware structure with computing functions in the display module, such as an algorithm.

In some embodiments, referring to Figure 11, there is a schematic diagram of the process of predicting the actual display positions of the second sub-frame picture and the third sub-frame picture in the embodiments of the present disclosure; wherein, the second prediction module is configured to calculate the first offset matrix of the original position to be displayed of the second sub-frame picture relative to the current position of the first sub-frame picture according to the offset of the current pupil position corresponding to the second sub-frame picture relative to the current pupil position corresponding to the first sub-frame picture; the second prediction module is also configured to calculate the second offset matrix of the original position to be displayed of the third sub-frame picture relative to the current position of the first sub-frame picture according to the offset of the current pupil position corresponding to the third sub-frame picture relative to the current pupil position corresponding to the first sub-frame picture; the second prediction module is also configured to multiply the original position to be displayed of the second sub-frame picture by the first offset matrix to obtain the actual display position of the second sub-frame picture; and multiply the original position to be displayed of the third sub-frame picture by the second offset matrix to obtain the actual display position of the third sub-frame picture.

Among them, the current position of the first sub-frame picture refers to the current display position of the first sub-frame picture. The current pupil position corresponding to the first sub-frame picture is obtained by detecting the currently established coordinate system. The current pupil position corresponding to the second sub-frame picture and the current pupil position corresponding to the third sub-frame picture are calculated based on the current pupil position corresponding to the first sub-frame picture, the speed and acceleration of the eyeball rotation, and the refresh frequency of a sub-frame picture. The position of the original position to be displayed of the second sub-frame picture in the human eye corresponds to the current pupil position corresponding to the second sub-frame picture, and the position of the original position to be displayed of the third sub-frame picture in the human eye corresponds to the current pupil position corresponding to the third sub-frame picture; the position of the actual display position of the second sub-frame picture in the human eye corresponds to the current pupil position corresponding to the first sub-frame picture; the position of the actual display position of the third sub-frame picture in the human eye corresponds to the current pupil position corresponding to the first sub-frame picture; that is, after the above calculation of the second prediction module, the positions of the first sub-frame picture, the second sub-frame picture and the third sub-frame picture in the human eye can all be located at the current pupil position corresponding to the first sub-frame picture, thereby eliminating the color separation phenomenon caused by the eyeball rotation.

In some embodiments, the first offset matrix is:

The second offset matrix is:

Among them, Rx1, Ry1, and Rz1 respectively represent the rotation amplitude of the original position to be displayed of the second sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx1, Ty1, and Tz1 respectively represent the translation amplitude of the original position to be displayed of the second sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Rx2, Ry2, and Rz2 respectively represent the rotation amplitude of the original position to be displayed of the third sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx2, Ty2, and Tz2 respectively represent the translation amplitude of the original position to be displayed of the third sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; the Z-axis is perpendicular to the two-dimensional coordinate plane formed by the intersection of the X-axis and the Y-axis, and the Z-axis intersects with the X-axis and the Y-axis at the origin.

In some embodiments, the second prediction module is a hardware structure with computing functions in the display module, such as an algorithm.

In some embodiments, referring to FIG5 , the display module is configured to sequentially display the second sub-frame image and the third sub-frame image according to their actual display positions. Specifically, the second sub-frame image and the third sub-frame image are sequentially displayed on the display module at their respective corresponding actual display positions, so that the images displayed by the pixels 10 at the same position on the first sub-frame image, the second sub-frame image, and the third sub-frame image can be projected to the same position in the pupil area of the human eye (i.e., on the retina), and the three images backlit by light sources of different colors are fused here, thereby eliminating the color separation phenomenon and ensuring that the image perceived by the human eye is a complete image.

In some embodiments, referring to FIG3 , the display module further includes a storage module configured to store the current pupil position corresponding to the first sub-frame image and the current pupil position corresponding to each subsequent sub-frame image, so that the current pupil position data corresponding to each sub-frame image can be used to predict the pupil position corresponding to each subsequent sub-frame image.

In some embodiments, the storage module is a hardware structure with storage function in the display module, such as a memory.

The display module provided in the embodiment of the present disclosure determines the current pupil position corresponding to the first sub-frame picture displayed first among n sub-frame pictures through processing by the processing module; the first prediction module calculates the current pupil position corresponding to the subsequent sub-frame pictures of the first sub-frame picture; the second prediction module calculates the actual display position of the subsequent sub-frame pictures of the first sub-frame picture according to the calculation result of the first prediction module; the rotation trajectory information of the human eye can be obtained so as to capture and track the rotation of the human eye; by adjusting the subsequent sub-frame pictures of the first sub-frame picture from the original to-be-displayed position to the actual display position, the actual display position of the subsequent sub-frame pictures of the first sub-frame picture can catch up with the position of the eyeball rotation, so that the pictures displayed by the pixels at the same position on the subsequent sub-frame pictures of the first sub-frame picture are projected to the same position in the pupil area of the human eye (i.e., on the retina), and the three pictures with backlight provided by different color light sources are merged here, thereby eliminating the color separation phenomenon and ensuring that the picture perceived by the human eye is a complete picture.

Based on the structure of the display module in the above embodiment, the embodiment of the present disclosure further provides a display method of the display module, referring to Figure 12, which is a flow chart of the display method of the display module in the embodiment of the present disclosure; wherein, the display method includes: Step S101: decomposing a frame of color picture into n sub-frame pictures; the n sub-frame pictures are displayed in sequence, n≥3, and n is an integer.

Step S102: Acquire an eye image, process the eye image, and determine a current pupil position corresponding to a first sub-frame image; the first sub-frame image is a sub-frame image that is displayed first when the n sub-frame images are displayed in sequence.

This step specifically includes: when the first sub-frame image is displayed, capturing an eye image; converting the eye image into a grayscale image; detecting left and right eye corner position points of the eye in the grayscale image according to eye corner feature points; connecting the left and right eye corner position points and using the line as the X-axis, using the axis perpendicular to the X-axis as the Y-axis, and the origin where the X-axis and the Y-axis intersect as the midpoint of the line connecting the left and right eye corner position points; processing the grayscale image in a coordinate plane formed by the X-axis and the Y-axis to determine the pupil area of the eye; and determining the center of the pupil area as the current pupil position corresponding to the first sub-frame image.

In some embodiments, the grayscale image is processed in a coordinate plane formed by an X-axis and a Y-axis to determine the pupil area of the eye; the center of the pupil area is determined as the current pupil position corresponding to the first subframe image, specifically including: binarizing the grayscale image to obtain a binary image of the eye; detecting candidate pupil connected areas on the binary image of the eye using a connected area marking method; screening out the pupil area of the eye from the candidate pupil connected areas based on geometric constraints and distance constraint algorithms; covering the pupil area with a circle with the smallest diameter, and determining the center of the circle as the center of the pupil area.

Step S103: Calculate the current pupil position corresponding to each subsequent sub-frame of the first sub-frame among the n sub-frames according to the current pupil position corresponding to the first sub-frame and the pupil position corresponding to the first sub-frame obtained by the previous m measurements; m≥2, and m is an integer.

In some embodiments, n=3, m=2; the subsequent sub-frame images of the first sub-frame image include a second sub-frame image and a third sub-frame image; the first sub-frame image, the second sub-frame image and the third sub-frame image are displayed in sequence; the step S103 includes: calculating the current pupil position corresponding to the second sub-frame image according to formula (1), and calculating the current pupil position corresponding to the third sub-frame image according to formula (2).

pos_curr_g＝pos_curr+v_curr×delta+1/2×a_curr×delta ² ； (1)

pos_curr_b＝pos_curr+v_curr×delta×2+1/2×a_curr×(2×delta) ² ； (2)

in,

a_curr is the current acceleration of the eyeball rotation; v_curr is the current speed of the eyeball rotation; pos_1 and pos_2 are the pupil positions corresponding to the first sub-frame images obtained in the previous two measurements; pos_curr is the current pupil position corresponding to the first sub-frame image; pos_curr_g is the current pupil position corresponding to the second sub-frame image; pos_curr_b is the current pupil position corresponding to the third sub-frame image; delta is the refresh duration of a sub-frame image, delta = 1/refresh frequency of a sub-frame image.

In some embodiments, the display method further includes: storing a current pupil position corresponding to the first sub-frame image and a current pupil position corresponding to each subsequent sub-frame image.

Step S104: Calculate the actual offset of the original position to be displayed of each subsequent sub-frame picture relative to the current position of the first sub-frame picture according to the offset of the current pupil position corresponding to each subsequent sub-frame picture among the n sub-frame pictures relative to the current pupil position corresponding to the first sub-frame picture; and calculate the actual display position of each subsequent sub-frame picture according to the original position to be displayed of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture.

This step specifically includes: calculating a first offset matrix of the original position to be displayed of the second sub-frame picture relative to the current position of the first sub-frame picture according to the offset of the current pupil position corresponding to the second sub-frame picture relative to the current pupil position corresponding to the first sub-frame picture. Calculating a second offset matrix of the original position to be displayed of the third sub-frame picture relative to the current position of the first sub-frame picture according to the offset of the current pupil position corresponding to the third sub-frame picture relative to the current pupil position corresponding to the first sub-frame picture. Multiplying the original position to be displayed of the second sub-frame picture with the first offset matrix to obtain the actual display position of the second sub-frame picture. Multiplying the original position to be displayed of the third sub-frame picture with the second offset matrix to obtain the actual display position of the third sub-frame picture.

In some embodiments, the first offset matrix is:

The second offset matrix is:

Among them, Rx1, Ry1, and Rz1 respectively represent the rotation amplitude of the original position to be displayed of the second sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx1, Ty1, and Tz1 respectively represent the translation amplitude of the original position to be displayed of the second sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Rx2, Ry2, and Rz2 respectively represent the rotation amplitude of the original position to be displayed of the third sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx2, Ty2, and Tz2 respectively represent the translation amplitude of the original position to be displayed of the third sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; the Z-axis is perpendicular to the two-dimensional coordinate plane formed by the intersection of the X-axis and the Y-axis, and the Z-axis intersects with the X-axis and the Y-axis at the origin.

Step S105: displaying the subsequent sub-frame images in sequence according to their actual display positions.

The display method of the display module provided in the embodiment of the present disclosure determines the current pupil position corresponding to the first sub-frame picture that is first displayed among n sub-frame pictures through processing; calculates the current pupil positions corresponding to the subsequent sub-frame pictures of the first sub-frame picture; calculates the actual display positions of the subsequent sub-frame pictures of the first sub-frame picture according to the calculation results of the current pupil positions corresponding to the subsequent sub-frame pictures of the first sub-frame picture; can obtain the rotation trajectory information of the human eyeball so as to capture and track the rotation of the human eyeball; by adjusting the subsequent sub-frame pictures of the first sub-frame picture from the original to-be-displayed position to the actual display position, it is possible to make the actual display position of the subsequent sub-frame pictures of the first sub-frame picture catch up with the position of the eyeball rotation, so that the pictures displayed by the pixels at the same position on the subsequent sub-frame pictures of the first sub-frame picture are projected to the same position in the pupil area of the human eye (i.e., on the retina), and three pictures with backlight provided by light sources of different colors are merged here, thereby eliminating the color separation phenomenon and ensuring that the picture perceived by the human eye is a complete picture.

In a third aspect, an embodiment of the present disclosure further provides a display device, which includes the display module in the above embodiment.

By adopting the display module in the above embodiment, the display device will not have color separation when performing virtual reality display, thereby improving the virtual reality display effect of the display device.

The display device can be: VR glasses, VR panels, VR TVs, mobile phones, tablet computers, laptop computers, monitors, digital photo frames, navigators, and any other products or components with VR display functions.

In a fourth aspect, an embodiment of the present disclosure further provides a virtual display device, comprising the display apparatus in the above embodiment.

By adopting the display device in the above embodiment, the virtual display device will not have color separation when performing virtual reality display, thereby improving the virtual reality display effect of the virtual display device.

The virtual display device can be: VR glasses, VR panels, VR TVs, mobile phones, tablet computers, laptop computers, monitors, digital photo frames, navigators, and any other products or components with VR display functions.

It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present disclosure, but the present disclosure is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and substance of the present disclosure, and these modifications and improvements are also considered to be within the scope of protection of the present disclosure.

Claims (11)

1. A display method, comprising: Decomposing a color picture frame into n sub-frame pictures; displaying the n sub-frame pictures in sequence, n≥3, and n is an integer; Acquire an eye image, process the eye image, and determine a current pupil position corresponding to a first sub-frame image; the first sub-frame image is a sub-frame image that is first displayed when the n sub-frame images are displayed in sequence; Calculate the current pupil position corresponding to each subsequent sub-frame picture of the first sub-frame picture in the n sub-frame pictures according to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by the previous m measurements; m≥2, and m is an integer; Calculating the actual offset of the original position to be displayed of each subsequent sub-frame picture relative to the current position of the first sub-frame picture according to the offset of the current pupil position corresponding to each subsequent sub-frame picture in the n sub-frame pictures relative to the current pupil position corresponding to the first sub-frame picture; and calculating the actual display position of each subsequent sub-frame picture according to the original position to be displayed of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture; Displaying the subsequent sub-frame images in sequence according to the actual display positions of the subsequent sub-frame images; The acquiring of the eye image, processing the eye image, and determining the current pupil position corresponding to the first subframe picture includes: When the first subframe is displayed, capturing an eye image; Convert the eye image to a grayscale image; In the grayscale image, the left and right eye corners are detected according to the eye corner feature points; the line connecting the left and right eye corners is used as the X-axis, the axis perpendicular to the X-axis is used as the Y-axis, and the origin of the intersection of the X-axis and the Y-axis is the midpoint of the line connecting the left and right eye corners; Processing the grayscale image in a coordinate plane formed by an X-axis and a Y-axis to determine a pupil area of the eye; determining the center of the pupil area as a current pupil position corresponding to the first subframe; The grayscale image is processed in a coordinate plane formed by an X-axis and a Y-axis to determine a pupil area of the eye; and the center of the pupil area is determined as a current pupil position corresponding to the first subframe image, including: Binarize the grayscale image to obtain a binary image of the eye; Using a connected region labeling method to detect candidate pupil connected regions on the binary image of the eye; Based on the geometric constraint and distance constraint algorithm, the pupil region of the eye is screened out in the candidate pupil connected region; The pupil area is covered with a circle with the smallest diameter, and the center of the circle is determined as the center of the pupil area. 2. The display method according to claim 1, wherein n=3, m=2; Subsequent sub-frame pictures of the first sub-frame picture include a second sub-frame picture and a third sub-frame picture; the first sub-frame picture, the second sub-frame picture and the third sub-frame picture are displayed in sequence; The calculating, according to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by previous m measurements, the current pupil position corresponding to each subsequent sub-frame picture of the first sub-frame picture in the n sub-frame pictures comprises: The current pupil position corresponding to the second sub-frame picture is calculated according to formula (1), and the current pupil position corresponding to the third sub-frame picture is calculated according to formula (2); pos_curr_g＝pos_curr+v_curr×delta+1/2×a_curr×delta ² ； (1) pos_curr_b＝pos_curr+v_curr×delta×2+1/2×a_curr×(2×delta) ² ； (2) in, a_curr is the current acceleration of the eyeball rotation; v_curr is the current speed of the eyeball rotation; pos_1 and pos_2 are respectively the pupil positions corresponding to the first sub-frame images obtained by the previous two measurements; pos_curr is the current pupil position corresponding to the first sub-frame image; pos_curr_g is the current pupil position corresponding to the second sub-frame image; pos_curr_b is the current pupil position corresponding to the third sub-frame image; delta is the refresh duration of one sub-frame image, delta=1/refresh frequency of one sub-frame image. 3. The display method according to claim 2, wherein the step of calculating the actual offset of the original position to be displayed of each subsequent sub-frame picture relative to the current position of the first sub-frame picture according to the offset of the current pupil position corresponding to each subsequent sub-frame picture in the n sub-frame pictures relative to the current pupil position corresponding to the first sub-frame picture; and calculating the actual display position of each subsequent sub-frame picture according to the original position to be displayed of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture, comprises: Calculating a first offset matrix of an original to-be-displayed position of the second sub-frame picture relative to a current position of the first sub-frame picture according to an offset of a current pupil position corresponding to the second sub-frame picture relative to a current pupil position corresponding to the first sub-frame picture; Calculating a second offset matrix of an original to-be-displayed position of the third sub-frame picture relative to a current position of the first sub-frame picture according to an offset of a current pupil position corresponding to the third sub-frame picture relative to a current pupil position corresponding to the first sub-frame picture; Multiplying the original to-be-displayed position of the second sub-frame picture by the first offset matrix to obtain the actual display position of the second sub-frame picture; The original to-be-displayed position of the third sub-frame picture is multiplied by the second offset matrix to obtain the actual display position of the third sub-frame picture. 4. The display method according to claim 3, wherein: The first offset matrix is: The second offset matrix is: Wherein, Rx1, Ry1, and Rz1 respectively represent the rotation amplitude of the original position to be displayed of the second subframe image along the X-axis, Y-axis, and Z-axis relative to the current position of the first subframe image; Tx1, Ty1, and Tz1 respectively represent the translation amplitude of the original position to be displayed of the second subframe image along the X-axis, Y-axis, and Z-axis relative to the current position of the first subframe image; Rx2, Ry2, and Rz2 respectively represent the rotation amplitude of the original position to be displayed of the third subframe image along the X-axis, Y-axis, and Z-axis relative to the current position of the first subframe image; Tx2, Ty2, and Tz2 respectively represent the translation amplitude of the original position to be displayed of the third subframe image along the X-axis, Y-axis, and Z-axis relative to the current position of the first subframe image; The Z axis is perpendicular to a two-dimensional coordinate plane formed by the intersection of the X axis and the Y axis, and the Z axis intersects the X axis and the Y axis at the origin. 5. The display method according to any one of claims 1 to 4, further comprising: storing a current pupil position corresponding to the first sub-frame image and a current pupil position corresponding to each subsequent sub-frame image. 6. A display module, comprising: The disassembly module is configured to disassemble a color picture frame into n sub-frame pictures; n≥3, and n is an integer; A display module, configured to display the n sub-frame images in sequence; a processing module configured to acquire an eye image, process the eye image, and determine a current pupil position corresponding to a first sub-frame image; the first sub-frame image is a sub-frame image that is displayed first among the n sub-frame images; A first prediction module is configured to calculate the current pupil position corresponding to each subsequent sub-frame picture of the first sub-frame picture among the n sub-frame pictures according to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by previous m measurements; m≥2, and m is an integer; a second prediction module configured to calculate an actual offset of an original position to be displayed of each subsequent sub-frame picture relative to a current position of the first sub-frame picture according to an offset of a current pupil position corresponding to each subsequent sub-frame picture among the n sub-frame pictures relative to a current pupil position corresponding to the first sub-frame picture; and calculate an actual display position of each subsequent sub-frame picture according to the original position to be displayed of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture; The display module is further configured to display the subsequent sub-frame images in sequence according to the actual display positions of the subsequent sub-frame images; The processing module further includes an image processing unit configured to convert the eye image into a grayscale image; detect left and right eye corner position points of the eye in the grayscale image according to the eye corner feature points; connect the left and right eye corner position points and use the line as the X-axis, and use the axis perpendicular to the X-axis as the Y-axis, and the origin of the intersection of the X-axis and the Y-axis as the midpoint of the line connecting the left and right eye corner position points; The image processing unit is further configured to process the grayscale image of the eye to determine the pupil area of the eye; and determine the center of the pupil area as the current pupil position corresponding to the first subframe image; The image processing unit is configured to perform binarization processing on the grayscale image to obtain a binarized image of the eye; Using a connected region labeling method to detect candidate pupil connected regions on the binary image of the eye; Based on the geometric constraint and distance constraint algorithm, the pupil region of the eye is screened out in the candidate pupil connected region; The pupil area is covered with a circle with the smallest diameter, and the center of the circle is determined as the center of the pupil area. 7. The display module according to claim 6, wherein the display module comprises a display panel and a lens, and the lens is located on a display side of the display panel; The processing module includes an infrared transmitter and an infrared camera. The infrared emitter is located on a side of the lens away from the display panel and is distributed around the edges of the lens, and is used to emit infrared light to the eyes; The infrared camera is located on a side of the lens away from the display panel and on an edge of the lens, and is used for capturing eye images. 8. The display module according to claim 7, wherein the subsequent sub-frame images of the first sub-frame image include a second sub-frame image and a third sub-frame image; the first sub-frame image, the second sub-frame image and the third sub-frame image are displayed in sequence; The second prediction module is configured to calculate a first offset matrix of an original to-be-displayed position of the second sub-frame picture relative to a current position of the first sub-frame picture according to an offset of a current pupil position corresponding to the second sub-frame picture relative to a current pupil position corresponding to the first sub-frame picture; The second prediction module is further configured to calculate a second offset matrix of an original to-be-displayed position of the third sub-frame picture relative to a current position of the first sub-frame picture according to an offset of a current pupil position corresponding to the third sub-frame picture relative to a current pupil position corresponding to the first sub-frame picture; The second prediction module is also configured to multiply the original display position of the second sub-frame picture by the first offset matrix to obtain the actual display position of the second sub-frame picture; and multiply the original display position of the third sub-frame picture by the second offset matrix to obtain the actual display position of the third sub-frame picture. 9. The display module according to any one of claims 6 to 8, further comprising a storage module configured to store the current pupil position corresponding to the first sub-frame image and the current pupil positions corresponding to subsequent sub-frame images. 10. A display device, comprising the display module according to any one of claims 6 to 9. 11. A virtual display device, comprising the display apparatus according to claim 10.