CN108663804B - Head mounted display and chromatic aberration compensation method using sub-pixel displacement - Google Patents

Abstract

The present invention provides a head-mounted display and a color difference compensation method using sub-pixel displacement. The head-mounted display includes: a left-eye display panel and a corresponding left-eye lens; a right-eye display panel and a corresponding right-eye lens; a display controller for obtaining image data from a main control terminal, wherein the image data includes a left-eye image and a right-eye image. The head-mounted display and the color difference compensation method using sub-pixel displacement of the present invention can solve the problem of color difference in the output image caused by the different refractive indices of light of different colors in the lens.

Description

Head mounted display and chromatic aberration compensation method using sub-pixel displacement

technical field

The present invention relates to the technical field of displays, in particular to a head-mounted display and a chromatic aberration compensation method utilizing sub-pixel displacement.

Background technique

Head Mounted Display (HMD) - head mounted display, can be mainly divided into immersive (immersive) and see-through (See-through) two types. Transmissive head-mounted displays are suitable for augmented reality (AR); immersive head-mounted displays are suitable for virtual reality (VR). The immersive head-mounted display mainly consists of a panel display and optical lenses and other components to form an image display system. For example, in an organic light emitting diode (OLED) panel display, there are many red, blue, green (R, G, B) light-emitting sub-pixels (Subpixels), which are composed of three primary color sub-pixels to form light-emitting pixels (pixels), and finally The image screen is composed of these pixels emitting light. In an immersive head-mounted display, after the panel display emits light, the light passes through the lens and is finally imaged to the human eye for viewing. However, due to the different refractive indices of light in different wavelengths, the phenomenon of dispersion will occur after the light is refracted through the lens.

FIG. 1 is a schematic diagram showing the dispersion phenomenon after light rays pass through a lens and are refracted. As shown in Figure 1. Ideally, the light 130 will focus on the point 110 after being refracted by the lens 120 . However, the refractive index of blue light (short wavelength) is higher than that of red light (long wavelength). After the light 130 passes through the lens 120, the refractive index of the blue light, green light, and red light are different, resulting in a dispersion phenomenon. For example, the red light of the light 130 , green light, and blue light will become rays 130R, 130G, and 130R respectively after being refracted by the lens 120 , and this dispersion phenomenon will cause the image viewed by the user to have chroma aberration.

In view of this, a head-mounted display and a chromatic aberration compensation method are required to solve the above problems.

SUMMARY OF THE INVENTION

The present invention provides a head-mounted display, comprising: a left-eye display panel and a corresponding left-eye lens; a right-eye display panel and a corresponding right-eye lens; and a display controller for controlling a main control terminal Obtain an image data, wherein the image data includes a left eye image and a right eye image, wherein the display controller also obtains a refraction characteristic curve corresponding to the left eye lens and the right eye lens, and calculates the image data. resolution, wherein the display controller also calculates the distance and relative direction of each sub-pixel of each color channel in the image data and an image center point of the image data, and adjusts the distance and relative direction in the image data according to the calculated distance and relative direction. A displacement amount of each sub-pixel of each color channel in the image data, wherein the display controller also performs a sub-pixel displacement compensation process according to the displacement amount of each sub-pixel of each color channel in the image data to adjust each color A corresponding sub-image is channeled to generate an output image, and the output image is played on the left-eye display panel and the right-eye display panel.

The present invention also provides a chromatic aberration compensation method using sub-pixel displacement for a head-mounted display, wherein the head-mounted display includes a left-eye display panel and a corresponding left-eye lens, a right-eye display panel and corresponding A right-eye lens and a display controller, the method includes: obtaining an image data from a main control terminal, wherein the image data includes a left-eye image and a right-eye image; obtaining the left-eye lens and the right-eye a refraction characteristic curve corresponding to the lens, and calculate the resolution of the image data; calculate the distance and relative direction of each sub-pixel of each color channel in the image data and an image center point of the image data; A displacement amount of each sub-pixel of each color channel in the image data is adjusted according to the distance and relative direction of the image data; according to the displacement amount of each sub-pixel of each color channel in the image data, a sub-pixel displacement compensation process is performed to adjust Sub-images corresponding to each color channel are used to generate an output image; the output image is played on the left-eye display panel and the right-eye display panel.

Description of drawings

FIG. 1 is a schematic diagram showing the dispersion phenomenon after light rays pass through a lens and are refracted.

FIG. 2 is a block diagram showing a head mounted display according to an embodiment of the present invention.

3A and 3B are schematic diagrams illustrating displacement of sub-pixels according to an embodiment of the present invention.

FIG. 4 is a flow chart illustrating a method for compensating for chromatic aberration using sub-pixel displacement according to an embodiment of the present invention.

Description of reference numbers:

110～point;

120～lens;

130～light;

130R～Red light;

130G～green light;

130B ~ Blu-ray;

200～head mounted display;

210 ~ left eye display panel;

211 ~ left eye lens;

220～right eye display panel;

221 ~ right eye lens;

230～shell;

240～data transmission interface;

250～display controller;

Steps S410-S470~.

Detailed ways

In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, a preferred embodiment is exemplified below, and is described in detail as follows in conjunction with the accompanying drawings.

FIG. 2 is a block diagram showing a head mounted display according to an embodiment of the present invention. In one embodiment, the head-mounted display 200 may include a housing 230 , a left-eye display panel 210 and a corresponding left-eye lens 211 , a right-eye display panel 220 and a corresponding right-eye lens 221 , and a transmission interface 240 . , and a display controller 250 . The left-eye display panel 210, the left-eye lens 211, the right-eye display panel 220, and the right-eye lens 221 are arranged in the casing 230 in a predetermined optical arrangement, and the casing 230 may include a strap or other auxiliary devices (not shown). ) for the user to wear on the head to view the picture through the head-mounted display 200 .

The transmission interface 240 is used for receiving image data (eg, including left-eye images and right-eye images, such as RGB images) from a host, and passing the image data through the display controller 250 to the left-eye display panel 210 and the right-eye display panel 210 The eye display panel 220 performs playback. For example, the transmission interface 240 may be a High Definition Multimedia Interface (HDMI) or a DisplayPort (DP) interface, but the invention is not limited thereto. In some embodiments, the left-eye display panel 210 and the right-eye display panel 220 can be implemented by the same display panel, and the left-eye display panel 210 and the right-eye display panel 220 are side-by-side and parallel with no angular difference therebetween.

The display controller 250 is used to control the display images on the left-eye display panel 210 and the right-eye display panel 220 . For example, the display controller 250 may be a general-purpose processor, a digital signal processor (DSP), an image signal processor (ISP), or a circuit with equivalent functions, which can be transmitted from the transmission interface 240 from the A main control terminal receives image data, for example, including a left eye image and a right eye image. Because the head-mounted display 200 is used to play the image, if no processing is performed on the played image, the different colors of light passing through the lens have different refractive indices, resulting in the left-eye displayed through the left-eye display panel 210 and the right-eye display panel 220. The image and right eye image have color cast. When the resolution of the image is larger, the pixels farther from the center of the image are more prone to color cast. Therefore, the display controller 250 compensates the offsets of the chromatic aberration for the sub-pixels of different color channels (eg, red, green, blue) in the left-eye image and the right-eye image respectively, so that the user's left and right eyes pass through the left and right eyes respectively. The images viewed by the eye lens 211 and the right eye lens 221 will not have color shift phenomenon.

In some embodiments, the display controller 250 includes one or more timing controllers (not shown in FIG. 2 ), and the left-eye display panel 210 and the right-eye display panel 220 in the head-mounted display 200 The display timing of the pixels in each row and column is controlled by one or more timing controllers in the display controller 250, and the one or more timing controllers are associated with the left-eye display panel 210 or the right-eye display panel 210. The eye display panel 220 can be integrated on a printed circuit board (PCB). The timing controller can be implemented by a processor or a microcontroller, or integrated into the display controller 250, such as an integrated circuit (IC) or a system-on-chip (SoC), but the invention is not limited thereto.

For example, testing the color difference in the HMD 200 may utilize a camera and a specific test image, such as a black grid image. If it is detected that the left-eye image and the right-eye image displayed by the head-mounted display 200 through the left-eye display panel 210 and the right-eye display panel 220 have a color cast, then estimate the difference of each color channel in the left-eye image and the right-eye image. The offset of the sub-pixels on the upper, lower, left and right boundaries means to measure the refraction characteristic curves of the left-eye lens and the right-eye lens, that is, the offset angles of red light, green light, and blue light passing through the left-eye lens 211 and the right-eye lens 221 respectively. Generally speaking, the refraction characteristic curves of the left-eye lens and the right-eye lens are matched, that is, they have the same refraction characteristic curve. The display controller 250 can perform corresponding sub-pixel displacement compensation according to the refraction characteristic curves of the left-eye lens and the right-eye lens.

For example, the offset of blue-light pixels is larger, and the offset of red-light pixels is smaller. In addition, the further away from the center point of the image, the greater the offset of the pixels. Note that the pixel offsets in the horizontal and vertical directions can be calculated independently.

Generally speaking, since the green light component is the majority, in some embodiments, the offset compensation may be performed only for the sub-pixels of the blue light and the red light.

3A and 3B are schematic diagrams illustrating displacement of sub-pixels according to an embodiment of the present invention. As shown in FIG. 3A , taking the blue light sub-pixels of the left-eye image as an example, if the horizontal and vertical resolutions (ie, the number of pixels) of the left-eye image are H and V, respectively. In one embodiment, if the blue light sub-pixel of the pixel at point A is taken as an example) the original coordinates of the image data at the main control end are (202, 100), after being refracted by the left-eye lens 211, what the user's left eye observes is The coordinates of point A may be offset to the right by two pixels from the actual coordinates of point A. Therefore, the position of the blue sub-pixel needs to be compensated. For example, the position of the blue light sub-pixel of the pixel at point A in the original image can be adjusted to be A' (200, 100) when played through the left-eye display panel, as shown in FIG. 3B . Therefore, after being refracted by the left-eye lens 211, the blue sub-pixel at the adjusted point A' will be accurately focused on the coordinate position of (202, 100), thereby allowing the user to see an image without color shift.

Furthermore, the point displacement of sub-pixels can be expressed by equation (1):

SubPixel(i, j)′=SubPixeli(i-Shift_Value _horizontal , j-Shift_Value _vertical )

(1)

The shifted sub-pixel coordinate SubPixel(i,j)' is equal to the coordinate position of the original sub-pixel coordinate SubPixl(i,j) after horizontal displacement Shift_Value _horizontal and vertical displacement Shift_Value _vertical . wherein i is an integer between 0 and H-1, and j is an integer between 0 and V-1.

In addition, the values of the two parameters of the horizontal displacement Shift_Value _horizontal and the vertical displacement Shift_Value _vertical are related to the distance D of the sub-pixel from the center of the image, where the distance D can be represented by the following formula (2):

Wherein, i and j are the coordinates of the sub-pixel, and (H-1)/2 and (V-1)/2 are the coordinates of the image center point.

Generally speaking, the greater the distance of the sub-pixel from the center point of the image, the greater the offset of the sub-pixel. When the distance between the sub-pixel and the center point of the image is smaller, the offset of the sub-pixel is also smaller. In one embodiment, the offset of the sub-pixels is linearly proportional to the distance D, but the invention is not limited thereto. When the resolution of the image data is larger, the distance between the pixels at the edge of the image and the center point of the image is also farther, and therefore the offset of the sub-pixels is also larger. If the calculated offset is not an integer, the display controller 250 uses an interpolation method to calculate the value of the sub-pixel to be compensated.

Boundary conditions about image edges are handled separately. For example, if the coordinates of the sub-pixels to be pre-shifted exceed the image boundary, the sub-pixels of the edges can be duplicated. For example, when the compensated sub-pixel coordinate exceeds the upper edge of the image, the sub-pixel at the upper edge of the image that has the same X-axis coordinate of the sub-pixel can be used for duplication. Alternatively, when the compensated sub-pixel coordinate exceeds the right edge of the image, the sub-pixel at the right edge of the image with the same Y-axis coordinate of the sub-pixel can be used for duplication.

It should be noted that the above embodiment takes the blue sub-pixel as an example, and the same process can also be used for the corresponding refraction curve characteristics of the red sub-pixel and the green sub-pixel to perform sub-pixel offset compensation.

For example, in a context of the present invention, the timing controller can directly display the left-eye image and the right-eye image from the display controller 250 on the left-eye display panel 210 and the right-eye display panel 220, respectively, and the left-eye image and the right eye image without any pixel offset processing. That is to say, in this situation, the present invention utilizes the display controller 250 to perform the chromatic aberration compensation processing of the pixel displacement.

In another aspect of the present invention, the timing controller performs the aforementioned pixel offset chromatic aberration compensation process on the left-eye image and the right-eye image from the display controller 250 respectively. That is, the display panel side (ie the timing controller and the display panel) can automatically determine the degree of chromatic aberration shift of the received image source and perform corresponding pixel shift processing for chromatic aberration compensation. In some embodiments, the display controller 250 can be omitted, and the timing controller can directly receive the image data from the main control terminal and perform corresponding color difference compensation processing.

FIG. 4 is a flow chart illustrating a method for compensating for chromatic aberration using sub-pixel displacement according to an embodiment of the present invention. In step S410, image data is obtained from the host. The image data (for example, including the left-eye image and the right-eye image) are transmitted from the host terminal to the display controller 250 through the transmission interface 240 .

In step S420, the refraction characteristic curves of the right eye lens and the left eye lens are obtained. The refraction characteristic curves of the right eye lens and the left eye lens can be obtained by using a camera and a test picture with a specific pattern in advance. Therefore, the display controller 250 can determine the offset of the sub-pixels of each color channel of the light under different resolutions according to the refraction characteristic curve (ie, perform sub-pixel compensation displacement).

In step S430, the resolution of the image data is calculated. For example, when the distance of the sub-pixel from the center point of the image is larger, the offset of the sub-pixel is larger. When the distance between the sub-pixel and the center point of the image is smaller, the offset of the sub-pixel is also smaller. In one embodiment, the offset of the sub-pixels is linearly proportional to the distance D, but the invention is not limited thereto. When the resolution of the image data is larger, the distance between the pixels at the edge of the image and the center point of the image is also farther, and therefore the offset of the sub-pixels is also larger. If the calculated offset is not an integer, the display controller 250 uses an interpolation method to calculate the value of the sub-pixel to be compensated.

In step S440, the distance and relative direction of each sub-pixel of each color channel in the image data and the image center point of the image data are calculated.

In step S450, the displacement amount of each sub-pixel of each color channel is adjusted according to the calculated distance and relative direction.

In step S460, a sub-pixel compensation process is performed according to the displacement of each sub-pixel of each color channel (eg, blue light, green light, red light) to adjust the sub-images of each color channel to generate a left-eye output image and a right-eye output image respectively output image.

In step S470, the left-eye output image and the right-eye output image are played on the left-eye display panel 210 and the right-eye display panel 220, respectively. It should be understood that, in conjunction with the aforementioned display controller and timing controller of the present invention, the above-mentioned chromatic aberration compensation method using sub-pixel displacement can be executed by the display controller or the timing controller according to different situations.

To sum up, the present invention provides a head-mounted display and a chromatic aberration compensation method using sub-pixel displacement, which utilizes pre-adjusted sub-pixel offsets of each color channel in the image data, so that the image displayed on the display panel has a higher quality. After the light passes through the lens, sub-pixels of different colors can be accurately focused on the same point, so that the user will not see the output image with chromatic aberration. Therefore, the head-mounted display and the chromatic aberration compensation method utilizing the sub-pixel displacement of the present invention can solve the problem of chromatic aberration of the output image caused by the difference in the refractive index of the lens with the light of different colors.

Although the present invention is disclosed above with preferred embodiments, it is not intended to limit the scope of the present invention. Any person skilled in the art can make some changes and modifications without departing from the concept and scope of the present invention. Therefore, The protection scope of the present invention shall be determined by the claims defined.

Claims (10)

1. A head-mounted display, comprising:

a left eye display panel and a corresponding left eye lens;

a right eye display panel and a corresponding right eyeglass head;

a display controller for obtaining an image data from a main control terminal, wherein the image data includes a left eye image and a right eye image, and the display controller further obtains a refractive characteristic curve corresponding to the left eye lens and the right eye lens, and calculates a resolution of the image data,

wherein the display controller further calculates a distance and a relative direction between each sub-pixel of each color channel in the image data and an image center point of the image data, and adjusts a displacement amount of each sub-pixel of each color channel in the image data according to the calculated distance and relative direction,

the display controller also performs a sub-pixel displacement compensation process according to the displacement of each sub-pixel of each color channel in the image data to adjust the sub-image corresponding to each color channel to generate an output image, and plays the output image on the left-eye display panel and the right-eye display panel.

2. The head mounted display of claim 1, wherein the refractive characteristic curve records an offset angle of blue light, green light, and red light passing through the left-eye lens and the right-eye lens, respectively.

3. The head-mounted display of claim 1, wherein the displacement of each sub-pixel of each color channel in the image data is linearly proportional to the calculated distance.

4. The head-mounted display of claim 3, wherein the display controller uses interpolation to calculate the value of each sub-pixel to be compensated for sub-pixel displacement when the calculated offset of each sub-pixel is not an integer.

5. The head-mounted display of claim 3, wherein the displacement comprises a horizontal displacement and a vertical displacement, and the display controller independently calculates the horizontal displacement and the vertical displacement, respectively.

6. A method for compensating chromatic aberration by sub-pixel displacement is used for a head-mounted display, wherein the head-mounted display comprises a left-eye display panel and a corresponding left-eye lens, a right-eye display panel and a corresponding right-eye lens, and a display controller, and the method comprises the following steps:

obtaining image data from a main control end, wherein the image data comprises a left eye image and a right eye image;

obtaining a refraction characteristic curve corresponding to the left-eye lens and the right-eye lens, and calculating the resolution of the image data;

calculating the distance and the relative direction between each sub-pixel of each color channel in the image data and an image center point of the image data;

adjusting a displacement amount of each sub-pixel of each color channel in the image data according to the calculated distance and the relative direction;

performing a sub-pixel displacement compensation process according to the displacement of each sub-pixel of each color channel in the image data to adjust the corresponding sub-image of each color channel to generate an output image;

and playing the output image on the left-eye display panel and the right-eye display panel.

7. The method of claim 6, wherein the refraction characteristic curve records an offset angle of blue light, green light, and red light passing through the left-eye lens and the right-eye lens.

8. The method according to claim 6, wherein the displacement of each sub-pixel of each color channel in the image data is linearly proportional to the calculated distance.

9. The color difference compensation method using sub-pixel displacement according to claim 8, further comprising:

when the calculated offset of each sub-pixel is not an integer, an interpolation method is used to calculate the value of each sub-pixel to be subjected to sub-pixel displacement compensation.

10. The method of claim 8, wherein the shift amount comprises a horizontal shift amount and a vertical shift amount, and the method further comprises: the horizontal displacement and the vertical displacement are calculated independently.

Images