WO2022247482A1

WO2022247482A1 - Virtual display device and virtual display method

Info

Publication number: WO2022247482A1
Application number: PCT/CN2022/085632
Authority: WO
Inventors: 朱帅帅; 毛春静; 熊宇辰; 王实现; 杨林林
Original assignee: 华为技术有限公司
Priority date: 2021-05-27
Filing date: 2022-04-07
Publication date: 2022-12-01
Also published as: CN115407504A

Abstract

Disclosed in embodiments of the present application are a virtual display device and a virtual display method, capable of avoiding problems such as vergence-accommodation conflict caused in a process of providing a virtual display function and thus avoiding visual fatigue of human eyes caused in the process of providing the virtual display function to a user. The specific solution is: a first optical display module is configured to implement, under the control of a processor, the following functions: displaying a first object, the vergence depth of the first object being a first vergence depth, and an imaging surface depth of the first optical display module being a first depth; and displaying a second object, the vergence depth of the second object being a second vergence depth, and the imaging surface depth of the first optical display module being a second depth. When the first vergence depth and the second vergence depth are different, the optical display module makes adjustment to make the first depth and the second depth different from each other.

Description

Virtual display device and virtual display method

This application claims the priority of a Chinese patent application filed with the State Intellectual Property Office on May 27, 2021, with application number 202110587591.7, and the title of the invention is "Virtual Display Device and Virtual Display Method", the entire contents of which are incorporated in this application by reference middle.

technical field

The embodiments of the present application relate to the field of artificial intelligence, and in particular to a virtual display device and a virtual display method.

Background technique

Display technology based on augmented reality (Augmented Reality, AR), virtual reality (Virtual Reality, VR) or mixed reality technology (Mixed Reality, MR) technology can provide users with a display solution close to the real scene experience, so it is being widely concerned .

However, current solutions for providing AR, VR or MR displays (that is, displays based on AR, VR or MR technologies) are not perfect, thereby detrimental to user experience.

Contents of the invention

The virtual display device and virtual display method provided by the embodiments of the present application can avoid problems such as vergence-accommodation conflict (VAC for short) generated in the process of providing virtual display functions, thereby avoiding problems such as providing virtual display to users. Visual fatigue of the human eye caused by the display function. Through this solution, it is also possible to adjust the optical power used for the image displayed to the user according to the ability of the user's human eyes to recognize graphics, so that even if the user's human eyes have problems such as myopia or astigmatism, the virtual The virtual display function provided by the display device.

In order to achieve the above purpose, the embodiment of the present application adopts the following technical solutions:

In a first aspect, a virtual display device is provided, and the virtual display device is used to provide a user with a display function of a virtual three-dimensional environment. The virtual display device includes:

A processor, and a first optical display module; the first optical display module is used to display images to the first human eyes under the control of the processor, and the first human eyes are both eyes of the user Any one of them; the first optical display module is used to realize the following functions under the control of the processor: display the first object, the depth of convergence of the first object is the first depth of convergence, and the first The depth of the imaging surface of the optical display module is the first depth; the second object is displayed, the depth of convergence of the second object is the second depth of convergence, and the depth of the imaging surface of the first optical display module is the second depth; wherein , the first object and the second object, the first object and the second object are included in the virtual three-dimensional environment, when the first depth of convergence and the second depth of convergence are different, the The optical display module adjusts the first depth to be different from the second depth.

Based on this solution, an example of a virtual display device is provided. In this example, when the virtual display device displays an image to the user's eyes, it can adjust the virtual image plane of the display direction to a depth close to or the same as the depth of convergence when the user observes the virtual object. In this way, in the process of providing the virtual display function to the user, the matching between the focusing depth and the depth of convergence can be realized (that is, the depth of the first virtual image plane is similar to or the same as the first depth of convergence). Thereby avoiding the problem of VAC caused by the difference between the focus depth and the depth of convergence. It should be noted that the virtual three-dimensional environment in this example may refer to all virtual environments constructed by the virtual display device based on display data. In some other embodiments, the virtual three-dimensional environment may also include objects in the real environment. For example, the virtual display device can collect relevant information of objects in the real environment in real time, and display the real environment corresponding to the information to the user through the display screen. That is to say, in this scene, the virtual three-dimensional environment may include a part of objects in the real environment. In some views, the scene may also include some imaginary objects in the virtual three-dimensional environment. It should be noted that, in this example, the virtual display device can determine the difference between the depth of the first virtual image plane and the first depth of convergence under the condition that the difference between the depth of the first virtual image plane and the first depth of convergence does not exceed 5%. similar. Of course, the above threshold setting of 5% can be flexibly adjusted, for example, it can be set to 6%, 8% and so on.

In a possible design, when the first depth of convergence is greater than the second depth of convergence, the first depth of convergence is greater than the second depth; when the first depth of convergence is smaller than the second depth of convergence , the first depth is smaller than the second depth. Based on this solution, a change relationship between the depth of convergence and the depth of focus (such as the first depth and the second depth) in this example is provided. In this example, the smaller the depth of convergence of the first object to be displayed, the smaller the corresponding focusing depth. Similarly, the greater the depth of convergence of the object to be displayed, the greater the corresponding focusing depth. In this way, the approaching or matching of the focusing depth and the convergence depth can be realized, thereby alleviating or solving the VAC problem.

In a possible design, the virtual display device further includes: a first eye-tracking module, the first eye-tracking module is used to perform eye movement on the first human eye under the control of the processor track. The processor is used to control the first eye-tracking module to perform eye-tracking on the gaze point of the first human eye, and when the depth of convergence of the gaze point of the first human eye is different, the first The optical display module is configured to adjust the depth difference of the imaging surface. Based on this solution, a specific solution for determining the depth of convergence of the user is provided. In this example, the virtual display device can track the user's eyes through the eye-tracking module, so as to obtain some data of the user's eyes during observation. For example, the gaze point of the human eye, and data such as the line of sight of the human eye. According to these data, the virtual display device can determine the angle between the line of sight when the user observes the object in the current virtual three-dimensional environment, so as to obtain the current depth of convergence.

In a possible design, when the diopters of the users are different, the first optical display module is configured to adjust the depth of the imaging surface to be different. Based on this scheme, another implementation of this example is provided. In this implementation, if users have different diopters, the first optical display module may provide the user with an imaging depth corresponding to the diopters. Therefore, even if the user's human eyes have the problem of ametropia, the virtual display function provided by the virtual display device can be used with naked eyes. Wherein, in some embodiments, the diopter of the user may include at least one of the following: myopia, hyperopia, and astigmatism.

In a possible design, the virtual display device further includes: a second optical display module, the second optical display module is used to display images to the second human eyes under the control of the processor, so The second human eye is one of the user's eyes that is different from the first human eye; the first optical display module is used to realize the following function under the control of the processor: display the first An object, showing that the depth of the imaging plane of the first object is a third depth, the depth of the first object in the virtual three-dimensional environment is a second depth, and the first depth is close to the third depth or the same. Based on this solution, a display solution for the other eye of the user's eyes is provided. In this example, the virtual display device can provide the second human eye of the user with a display in which the focus depth matches the convergence depth through the second optical display module. It should be noted that since the user's depth of convergence is generated when observing objects with both eyes, the solution provided in the first aspect shows that when the virtual display device displays images to the user's eyes, the corresponding depth of convergence can be Are the same.

In a possible design, the first optical module includes: a first zoom module. The optical power of the first zoom module is adjustable. The first zoom module is configured to adjust the optical power of the first zoom module to a first optical power when displaying the first object under the control of the processor, so that the first optical power has the A first optical module of one optical power is capable of imaging at the first depth; or, the first zoom module is configured to, under the control of the processor, adjust the The optical power of the first zoom module is the second optical power, so that the first optical module with the second optical power can image at the second depth. Based on this solution, a specific example of a solution for adjusting the depth of the virtual image plane is provided. In this example, in the virtual display device, the optical module (such as the first optical module corresponding to the first human eye) responsible for displaying images to the user's human eyes may include a zoom module, which has the ability to adjust the optical power Ability. By adjusting the optical power of the zoom module, the optical power of the virtual display device can be adjusted. It is understood that different optical powers may correspond to different depths of the virtual image plane. Therefore, by adjusting the focal power, the adjustment of the focus depth can be realized. For example, the VAC problem can be solved by adjusting the focal power to a certain value so that the depth of the virtual image plane matches the depth of convergence.

In a possible design, the first optical display module is further configured to display a third object at the first virtual position and the second virtual position in the virtual three-dimensional environment under the control of the processor. The first virtual position and the second virtual position have different distances to the user's eyes in the three-dimensional environment. Based on this solution, another usage mechanism of the virtual display device provided by the embodiment of the present application is provided. In this example, the virtual display device can present objects with different depths to the user. In this way, when the user's eyes observe the objects of different depths, they can naturally control the ciliary muscle to stretch, thereby achieving the effect of exercising the ciliary muscle. In this way, visual fatigue can be avoided, thereby preventing myopia. In some embodiments, the third object displayed at the first virtual location may be the same as the third object displayed at the second virtual location. In some other embodiments, the object displayed at the first virtual position may be different from the object displayed at the second virtual position. In the process of displaying images of different depths to the user's eyes, the virtual display device can also adjust the matching between the focus depth and the convergence depth in the corresponding scene according to the solution in the foregoing example, so that while exercising the ciliary muscles, Able to avoid VAC problems.

In a possible design, the virtual display device displays the third object at the first virtual position and the second virtual position respectively when the currently displayed scene is a preset scene. Wherein, the preset scene includes at least one of the following scenes: an advertisement playing scene, and a display resource loading scene. Based on this scheme, a possible scenario example of ciliary muscle exercise is provided. In this example, the virtual display device can display objects of different depths to the user in some specific scenarios. These specific scenes may be invalidly displayed scenes when the user uses the virtual display function. For example, before showing the displayed content to the user, when playing an advertisement, the virtual display device can combine the advertisement content to display images to the user's human eyes at different depths, thereby avoiding eye fatigue. As another example, during the loading process of display resources such as videos or images, the virtual display device can display images of different depths to the user in combination with the content of the display resources to be displayed (or other preset content), so as to exercise the ciliary muscles, Avoid video features. In some embodiments of the present application, the virtual display device can display different options for each control with different depths when presenting different options to the user, so that the user can also see objects at different depths to avoid eye strain.

In a possible design, the first optical display module is also used to display the first detection image to the first human eye under the control of the processor. When displaying the first detection image, the number of pixels of the opening for displaying the first detection image is the first number. The processor is also used to receive first recognition feedback, which is an indication input by the user when using the first human eyes to observe the first detection image, and the first recognition feedback is used to indicate whether the user can The opening direction of the first detection image is identified. Based on this solution, an example of a solution for judging the image recognition ability of the user's human eyes is provided. It can be understood that the virtual display device can display the detection image in the virtual three-dimensional environment and guide the user to observe the detection image, so as to determine the user's ability to recognize the image according to the feedback input by the user. In some examples, the user's ability to recognize images may include the degree of myopia of the user's human eyes and the like. In some embodiments, it is taken as an example that the detected image is a letter with an opening. In order to accurately judge the degree of myopia of the human eye, the virtual display device can determine the number of pixels corresponding to the required visual score for displaying to the user according to the visual score of each pixel on the display screen, thereby determining the required The size of the displayed letters. In this way, the letter image shown to the user with the opening with the above number of pixels can accurately determine the degree of myopia of the human eye according to whether the user can recognize the direction of the opening. It can be understood that the smaller the number of opening pixels that the user can identify, the better the user's eyesight and the smaller the corresponding degree of myopia.

In a possible design, when the first number of pixels displays the opening of the first detection image to the user, the visual score of the first human eye observing the opening of the first detection image is the first visual score. When the first recognition feedback indicates that the user cannot recognize the opening direction of the first detection image, the processor is configured to determine the degree of myopia of the first human eye as the degree of myopia corresponding to the first vision score. Based on this solution, a specific example of judging the degree of myopia of a user is provided. It can be understood that, when measuring the degree of myopia, the size of the smallest letter that can be seen clearly may correspond to the degree of myopia of the user. For different letter sizes, for the human eyes whose relative positions are fixed, their visual scores are also corresponding. Therefore, in this example, the virtual display device can determine the degree of myopia of the user according to the vision score corresponding to the opening with the smallest number of pixels that the user can identify. For example, when the user determines that the opening direction of the current letter cannot be recognized, an unrecognizable feedback may be input. Based on this feedback, the virtual display device can know that the user cannot recognize the direction of the opening with the current number of pixels. The opening direction corresponds to a certain visual score, so the degree of myopia of the user can be determined by referring to the visual score according to the virtual display device. It should be noted that, in this example, the first recognition feedback may be input to the virtual display device by the user through a remote control, voice, gesture, and the like. In some other embodiments, the virtual display device may also determine that the user cannot recognize the opening direction of the letter of the current size when the user fails to input correct recognition feedback in the preset market. In addition, in some embodiments, in order to make the measurement of the degree of myopia of the user's human eyes more accurate, the virtual display device may display another detection image that is close to or the same size as the current image to the user after the user inputs feedback that the current image cannot be recognized. Image, provided to the user for identification. When the user fails to recognize the detection image of the current size for many times, the virtual display device can accurately determine that the user cannot recognize the current detection image.

In a possible design, when the first identification feedback indicates that the user can identify the opening direction of the first detection image, the first optical display module is further configured to, under the control of the processor, display The first human eye presents a second detection image. When displaying the second detection image, the number of pixels of the opening for displaying the second detection image is the second number. The second quantity is less than the first quantity. The processor is also used to receive second recognition feedback, which is an indication input by the user when using the first human eyes to observe the second detection image, and the second recognition feedback is used to indicate whether the user can The opening direction of the second detection image is identified. Based on this solution, another example of determining a user's ability to recognize graphics is provided. In this example, the virtual display device may present another detection image (such as a second detection image) of a size different from the first detection image to the user if the user can recognize the first detection image. In some embodiments, the second detection image may be smaller than the first detection image. In combination with the foregoing solution, the virtual display device may display the second detection image with the number of pixels corresponding to a smaller visual score. In this way, human eyes can recognize letter images with smaller openings, thereby enabling the virtual display device to more accurately measure the recognition ability of human eyes.

In a possible design, when the second number of pixels displays the opening of the second detection image to the user, the visual score of the opening of the second detection image observed by the first human eyes is the second visual score. When the second recognition feedback indicates that the user cannot recognize the opening direction of the second detected image, the processor is configured to determine the degree of myopia of the first human eye as the degree of myopia corresponding to the second vision score. Based on this solution, an example of a solution for determining the degree of myopia according to the second detection image is provided. In this example, the virtual display device may determine the degree of myopia of the user after determining the vision score corresponding to the smallest detected image that the user can recognize.

In a possible design, the first optical display module is used to adjust the light of the first optical display module under the control of the processor before displaying the first detected image to the first human eye. The focal power is the initial focal power, so that the first optical display module forms an image at the furthest distance. Based on this solution, an example in which a virtual display device displays initial settings in the process of detecting images to a user is provided. In this example, the virtual display device can adjust the optical power so that the virtual image that can be seen by human eyes is located at the farthest. In this way, the light incident on the human eye can be close to parallel light. It can be understood that, for human eyes without myopia, the user's human eyes can accurately focus parallel light on the retina. In this way, based on the solution provided in this example, the virtual display device can provide normal display for human eyes with normal vision. At the same time, it can also realize the power measurement of the human eyes with myopia.

In a possible design, when the opening of the third inspection image displayed by the first optical display module to the first human eye is the third number of pixels, when the received recognition feedback indicates that the user cannot recognize When the opening direction of the fourth detection image is detected, the first optical display module displays the optical power of the virtual image to the first human eye as the second optical power. The second optical power corresponds to a third visual score, which is the visual score corresponding to the third number of pixels. Based on this solution, an example of a solution for providing naked-eye virtual display experience to myopic users is provided. It is understandable that the human eyes of myopic users cannot smoothly converge the normally displayed images on the retina, but converge the image plane between the retina and the pupil, so that if the images are displayed normally, the myopic users cannot see clearly with the naked eye displayed content. Through the solution provided in this example, the virtual image plane in the process of displaying the image to the user's eyes can be adjusted according to the pixel number of the smallest opening that the user can identify (such as adjusting the optical power to realize the adjustment of the virtual image plane), so that It can enable myopic users to image the adjusted virtual image plane on the retina of their human eyes, thereby achieving the effect that myopic users can clearly see the display content with naked eyes.

In a possible design, the first optical display module includes a rotating mechanism for rotating the first optical display module in a direction perpendicular to the optical axis under the control of the processor. When the first optical display module presents the first image to the user, the focal power direction of the first optical display module coincides with the astigmatism axis of the first human eye. Based on this solution, a display solution that can match the human eyes of users with astigmatism is provided. It can be understood that for users with astigmatism, their human eyes cannot clearly see the displayed image at a certain incident angle. That is to say, the human eye with astigmatism can only clearly see the image where the incident light coincides with the astigmatism axis of the human eye. In this example, the virtual display device can flexibly adjust the focal power direction of the first optical display module, so as to achieve the effect that the angle of light incident on the virtual display device coincides with the astigmatism axis of the human eye. In this way, the user with astigmatism can also clearly see the effect of the image displayed by the virtual display device. In this way, even if the user's human eyes have astigmatism, the virtual display device can provide the user with naked-eye viewing experience according to the solution.

In a possible design, the processor is also used to determine the second The degree of astigmatism in a human eye. Based on this solution, an example of a solution for measuring the degree of astigmatism of a user is provided. It can be understood that the degree of astigmatism of the user's human eyes is related to the axis of the user's astigmatism. In this example, the axial position of astigmatism when the user can see the image clearly can be determined by adjusting the optical power direction of the system, and the degree of astigmatism of the user can be determined according to the axial position of astigmatism. In some embodiments, the virtual display device can display the astigmatism diopter measurement card to the user, and determine the user's astigmatism axis position according to the sign (such as the corresponding number) corresponding to the angle that cannot be seen clearly indicated by the user's feedback, so as to The astigmatism axis determines the degree of astigmatism in the user's eye.

In a possible design, the processor is also used to acquire the user characteristics of the current user before controlling the first optical display module to display the first image, and the processor is specifically used to control the first optical display module to display the first image. The group displays the first image corresponding to the user characteristics of the current user. Wherein, when displaying the first image, the optical power of the first optical display module is matched with the myopia degree and/or astigmatism degree of the first human eye of the current user, and the myopia degree of the first human eye of the current user is The degree of power and/or the degree of astigmatism is indicated by the virtual display information corresponding to the user characteristics of the current user. Based on this solution, an example of a solution for providing naked-eye virtual experience to corresponding users is provided. In this example, since different users have different degrees of myopia and/or astigmatism, the virtual display device can identify the current user according to the user characteristics of the current user, and use the stored current The degree of myopia and/or astigmatism of the user determines the user's ability to recognize the image. Furthermore, the virtual display device can be adjusted to the corresponding optical power to display images to the user, so that the optical power of the displayed image can match the current user's ability to recognize the image. In this way, even if the user has myopia or astigmatism, he can clearly see the image displayed by the virtual display device with naked eyes.

In a possible design, the user feature includes any one of the following features: fingerprint information of the current user. The iris characteristics of this current user. The current user's account information. The identifier of the current user is different for different users. Based on this solution, an example of user characteristics is provided. It can be understood that the virtual display device can distinguish different users through different user characteristics. In this example, some examples of user features are provided. For example, user features may include biometric information such as fingerprints, irises, and voiceprints. As another example, the user feature may be the user's account information, such as user name, user nickname, and so on. For another example, the user feature may also be an identifier of a user preset in advance, and the identifier may be used to identify different users.

In a possible design, the virtual display device stores correspondences between different user characteristics and corresponding virtual display information. The processor is used to search for a matching entry from the corresponding relationship according to the user characteristics of the current user, and if there is a matching entry, determine that the virtual display information corresponding to the current user is in the matching entry Stored virtual display information. The virtual display information includes the degree of myopia and/or the degree of astigmatism of the corresponding user. Based on this solution, an example of a solution in which a virtual display device determines a corresponding display strategy according to a user is provided. In this example, the virtual display device can identify the current user according to the user characteristics of the user. Next, the virtual display device can search the virtual display information corresponding to the user characteristics of the user from the correspondence stored locally (or stored in the cloud) according to the characteristics of the current user. If it can be found, it indicates that it can be displayed according to the virtual display information. Exemplarily, the virtual display device may adjust the optical power according to the virtual display information, so as to provide naked-eye display experience to both eyes of the user at corresponding virtual image plane positions.

In a possible design, when there is no matching item corresponding to the user characteristics of the current user in the corresponding relationship, the first optical display module is also used to display the first image, the focal power when displaying the first image is the focal power that matches the myopia and/or astigmatism of the first human eye of the current user, and the myopia and/or astigmatism of the first human eye is determined automatically by the processor, or is determined at the direction of a user, or is manually entered by the user. Based on this solution, another solution for determining the display strategy according to the characteristics of the current user is provided. In this example, the virtual display device may measure the current user's image recognition ability according to the solution provided in the above example, for example, determine the degree of myopia of the user and/or the degree of astigmatism of the user. In this way, the corresponding naked-eye display experience can be presented to the user according to the user's image recognition ability. In some other embodiments of the present application, the determination of the degree of myopia and/or the degree of astigmatism of the user may be performed under the authorization or instruction of the user. In some other embodiments of the present application, the degree of myopia and/or the degree of astigmatism of the user may also be input by the user. It should be noted that, before the virtual display device obtains the user's image recognition ability, it can search whether the virtual display information corresponding to the user characteristics of the current user is stored in the corresponding relationship stored locally or in the cloud. If it exists, it can be executed according to the aforementioned solution, that is, display is performed according to the virtual display information. If it does not exist, the identification cannot directly obtain the current user's myopia and/or astigmatism, so the virtual display device can obtain the user's myopia and/or astigmatism according to the solution in this example.

In a possible design, the processor is further configured to store a correspondence between the degree of myopia and/or the degree of astigmatism of the first human eye and the user characteristics of the current user. Based on this solution, an example of a solution for updating the storage of the user's image recognition ability is provided. In this example, the virtual display device may determine that the current user is a new user when the corresponding relationship stored locally or from the cloud cannot find an entry corresponding to the current user. Then, the obtained corresponding relationship between the image recognition ability of the current user and the user characteristics of the current user can be stored, so that the image recognition ability of the user does not need to be measured again when the virtual display is provided to the user later.

In a second aspect, a virtual display device is provided, and the virtual display device may have the composition of the virtual display device provided in the first aspect. In this example, the first optical display module is further used to display a third object at the first virtual position and the second virtual position in the virtual three-dimensional environment under the control of the processor. The first virtual position and the second virtual position have different distances to the user's eyes in the three-dimensional environment. It can be understood that, in the description of the first aspect, the effect of exercising the ciliary muscle can be achieved by combining the technical means for solving the VAC problem. In this example, the virtual display device can also exercise the ciliary muscle only by displaying objects at different depths.

In a possible design, the virtual display device displays the third object at the first virtual position and the second virtual position respectively when the currently displayed scene is a preset scene. Wherein, the preset scene includes at least one of the following scenes: an advertisement playing scene, and a display resource loading scene.

In a third aspect, a virtual display device is provided, and the virtual display device may have the composition of the virtual display device provided in the first aspect. In this example, the first optical display module is also used to display the first detection image to the first human eye under the control of the processor. When displaying the first detection image, the number of pixels of the opening for displaying the first detection image is the first number. The processor is also used to receive first recognition feedback, which is an indication input by the user when using the first human eyes to observe the first detection image, and the first recognition feedback is used to indicate whether the user can The opening direction of the first detection image is identified.

In this example, the virtual display device can display the detection image in the virtual three-dimensional environment and guide the user to observe the detection image, so as to determine the user's ability to recognize the image according to the feedback input by the user. In some examples, the user's ability to recognize images may include the degree of myopia of the user's human eyes and the like. In this way, the optometry of the user is realized. It should be noted that this solution can support the independent optometry of the virtual display device to determine the degree of myopia of the user's eyes. In other implementations, this scheme can also be used to support remote vision.

In a possible design, when the first number of pixels displays the opening of the first detection image to the user, the visual score of the first human eye observing the opening of the first detection image is the first visual score. When the first recognition feedback indicates that the user cannot recognize the opening direction of the first detection image, the processor is configured to determine the degree of myopia of the first human eye as the degree of myopia corresponding to the first vision score.

In a possible design, when the first identification feedback indicates that the user can identify the opening direction of the first detection image, the first optical display module is further configured to, under the control of the processor, display The first human eye presents a second detection image. When displaying the second detection image, the number of pixels of the opening for displaying the second detection image is the second number. The second quantity is less than the first quantity. The processor is also used to receive second recognition feedback, which is an indication input by the user when using the first human eyes to observe the second detection image, and the second recognition feedback is used to indicate whether the user can The opening direction of the second detection image is identified.

In a possible design, when the second number of pixels displays the opening of the second detection image to the user, the visual score of the opening of the second detection image observed by the first human eyes is the second visual score. When the second recognition feedback indicates that the user cannot recognize the opening direction of the second detected image, the processor is configured to determine the degree of myopia of the first human eye as the degree of myopia corresponding to the second vision score.

In a possible design, the first optical display module is used to adjust the light of the first optical display module under the control of the processor before displaying the first detected image to the first human eye. The focal power is the initial focal power, so that the first optical display module forms an image at the furthest distance.

In a possible design, when the opening of the third inspection image displayed by the first optical display module to the first human eye is the third number of pixels, when the received recognition feedback indicates that the user cannot recognize When the opening direction of the fourth detection image is detected, the first optical display module displays the optical power of the virtual image to the first human eye as the third optical power. The third optical power corresponds to a third visual score, which is the visual score corresponding to the third number of pixels.

In a fourth aspect, a virtual display device is provided, and the virtual display device may have the composition of the virtual display device provided in the first aspect. In this example, the first optical display module includes a rotating mechanism for rotating the first optical display module in a direction perpendicular to the optical axis under the control of the processor. When the first optical display module presents the first image to the user, the focal power direction of the first optical display module coincides with the astigmatism axis of the first human eye.

Based on this solution, a display solution that can match the human eyes of users with astigmatism is provided. It can be understood that for users with astigmatism, their human eyes cannot clearly see the displayed image at a certain incident angle. That is to say, the human eye with astigmatism can only clearly see the image where the incident light coincides with the astigmatism axis of the human eye. In this example, the virtual display device can flexibly adjust the focal power direction of the first optical display module, so as to achieve the effect that the angle of light incident on the virtual display device coincides with the astigmatism axis of the human eye. In this way, the user with astigmatism can also clearly see the effect of the image displayed by the virtual display device.

In a possible design, the processor is also used to determine the second The degree of astigmatism in a human eye.

In a fifth aspect, a virtual display device is provided, and the virtual display device may have the composition of the virtual display device provided in the first aspect. In this example, the processor is also used to acquire the user characteristics of the current user before controlling the first optical display module to display the first image, and the processor is specifically used to control the first optical display module to display and display the first image. The first image corresponding to the user characteristics of the current user. Wherein, when displaying the first image, the optical power of the first optical display module is matched with the myopia degree and/or astigmatism degree of the first human eye of the current user, and the myopia degree of the first human eye of the current user is The degree of power and/or the degree of astigmatism is indicated by the virtual display information corresponding to the user characteristics of the current user.

Based on this solution, an example of a solution for providing naked-eye virtual experience to corresponding users is provided. In this example, since different users have different degrees of myopia and/or astigmatism, the virtual display device can identify the current user according to the user characteristics of the current user, and use the stored current The degree of myopia and/or astigmatism of the user determines the user's ability to recognize the image. Furthermore, the virtual display device can be adjusted to the corresponding optical power to display images to the user, so that the optical power of the displayed image can match the current user's ability to recognize the image. In this way, even if the user has myopia or astigmatism, he can clearly see the image displayed by the virtual display device with naked eyes.

In a possible design, the user feature includes any one of the following features: fingerprint information of the current user. The iris characteristics of this current user. The current user's account information. The identifier of the current user is different for different users.

In a possible design, the virtual display device stores correspondences between different user characteristics and corresponding virtual display information. The processor is used to search for a matching entry from the corresponding relationship according to the user characteristics of the current user, and if there is a matching entry, determine that the virtual display information corresponding to the current user is in the matching entry Stored virtual display information. The virtual display information includes the degree of myopia and/or the degree of astigmatism of the corresponding user.

In a possible design, when there is no matching item corresponding to the user characteristics of the current user in the corresponding relationship, the first optical display module is also used to display the first image, the focal power when displaying the first image is the focal power that matches the myopia and/or astigmatism of the first human eye of the current user, and the myopia and/or astigmatism of the first human eye is determined automatically by the processor, or is determined at the direction of a user, or is manually entered by the user.

In a possible design, the processor is further configured to store a correspondence between the degree of myopia and/or the degree of astigmatism of the first human eye and the user characteristics of the current user.

It should be noted that the functions corresponding to the virtual display device provided by the second aspect, the third aspect, the fourth aspect, and the fifth aspect and any possible implementation thereof may be realized independently or in combination with each other. of. For example, the function corresponding to the virtual display device provided by the second aspect and any possible implementation thereof may be the same as the virtual display device provided by the third aspect and/or the fourth aspect and/or the fifth aspect and any possible implementation thereof. The functions corresponding to the devices are integrated in the same device. As another example, the functions corresponding to the virtual display device provided by the third aspect and any possible implementation thereof may be integrated with the functions corresponding to the virtual display device provided by the fourth aspect and/or the fifth aspect and any possible implementation thereof in the same device. As another example, the function corresponding to the virtual display device provided by the fourth aspect and any possible implementation thereof may be integrated into the same device as the function corresponding to the virtual display device provided by the fifth aspect and any possible implementation thereof.

In a sixth aspect, a virtual display method is provided, the virtual display method is applied to the virtual display device provided in the first aspect and any possible implementation thereof, and the virtual display method is used to provide a user with a display of a virtual three-dimensional environment Function. The virtual display method includes: the first optical display module displays the first object, the depth of convergence of the first object is the first depth of convergence, and the depth of the imaging surface of the first optical display module is the first depth. The first optical display module displays the second object, the depth of convergence of the second object is the second depth of convergence, and the depth of the imaging surface of the first optical display module is the second depth. Wherein, the first object and the second object, the first object and the second object are included in the virtual three-dimensional environment, when the first depth of convergence and the second depth of convergence are different, the optical display module adjusts the The first depth is different from the second depth. In a possible design, the virtual display method further includes: the processor controls a first eye-tracking module in the virtual display device to perform eye-tracking on the first human eye. When the eye tracking results of the first human eyes are different, the second depths are different.

In a possible design, when the first depth of convergence is greater than the second depth of convergence, the first depth is greater than the second depth. When the first depth of convergence is smaller than the second depth of convergence, the first depth is smaller than the second depth.

In a possible design, the processor controls the first eye-tracking module of the first optical display module to perform eye-tracking on the gaze point of the first human eye, and the gaze point of the first human eye When the depths of convergence are different, the first optical display module is configured to adjust the depth of the imaging surface to be different.

In a possible design, when the diopters of the users are different, the first optical display module is configured to adjust the depth of the imaging surface to be different.

In a possible design, the virtual display device further includes: a second optical display module, the second optical display module is used to display images to the second human eyes under the control of the processor, the second The human eye is the one of the user's eyes that is different from the first human eye. The method further includes: the first optical display module realizes the following functions under the control of the processor: displaying the first object, displaying that the depth of the imaging surface of the first object is a third depth, and the depth of the imaging surface of the first object in the The depth in the virtual three-dimensional environment is the second depth, and the first depth is similar to or the same as the third depth.

In a possible design, the first optical module includes: a first zoom module. The optical power of the first zoom module is adjustable. The method further includes: the processor controls the first zoom module, and when displaying the first object, adjusts the optical power of the first zoom module to the first optical power, so that the The first optical module is capable of imaging at the first depth. Or, the processor controls the first zoom module, and when displaying the second object, adjusts the optical power of the first zoom module to the second optical power, so that the first optical lens with the second optical power The module is capable of imaging at this second depth.

In a possible design, the method further includes: the first optical display module displays a third object at the first virtual position and the second virtual position in the virtual three-dimensional environment under the control of the processor. The first virtual position and the second virtual position have different distances to the user's eyes in the three-dimensional environment.

In a possible design, the method further includes: the first optical display module displays the first detection image to the first human eye under the control of the processor. When displaying the first detection image, the number of pixels of the opening displaying the first detection image is the first number. The processor is also used to receive first recognition feedback, which is an indication input by the user when using the first human eyes to observe the first detection image, and the first recognition feedback is used to indicate whether the user can The opening direction of the first detection image is identified.

In a possible design, when the first optical display module uses the first number of pixels to display the opening of the first detection image to the user, the first human eye observes the visual field of the opening of the first detection image. The score is the first-look score. In the case where the first recognition feedback indicates that the user cannot recognize the opening direction of the first detection image, the method further includes: the processor determines that the degree of myopia of the first human eye corresponds to the first vision score degree of myopia.

In a possible design, the method further includes: when the first identification feedback indicates that the user can identify the opening direction of the first detection image, the first optical display module under the control of the processor , showing the second detection image to the first human eye. When displaying the second detection image, the number of pixels of the opening for displaying the second detection image is the second number. The second quantity is less than the first quantity. The processor receives second recognition feedback, the second recognition feedback is an instruction input by the user when using the first human eyes to observe the second detection image, and the second recognition feedback is used to indicate whether the user can recognize the second detection image 2. Detect the opening direction of the image.

In a possible design, when the first optical display module uses the second number of pixels to display the opening of the second detection image to the user, the first human eye observes the visual field of the opening of the second detection image. The score is the second-view score. In the case where the second recognition feedback indicates that the user cannot recognize the opening direction of the second detection image, the method further includes: the processor determines that the degree of myopia of the first human eye corresponds to the second vision score degree of myopia.

In a possible design, the method further includes: before the first optical display module presents the first detected image to the first human eye, under the control of the processor, adjusting the first optical display module The optical power of the group is the initial optical power, so that the first optical display module forms an image at the furthest distance.

In a possible design, the first optical display module includes a rotating mechanism for rotating the first optical display module in a direction perpendicular to the optical axis under the control of the processor. The method further includes: when the first optical display module presents the first image to the user, the focal power direction of the first optical display module coincides with the astigmatism axis of the first human eye.

In a possible design, the processor determines that the first human eye degree of astigmatism.

In a possible design, the processor is also used to acquire the user characteristics of the current user before controlling the first optical display module to display the first image, and the processor is specifically used to control the first optical display module to display the first image. The group displays the first image corresponding to the user characteristics of the current user. Wherein, when displaying the first image, the optical power of the first optical display module is matched with the myopia degree and/or astigmatism degree of the first human eye of the current user, and the myopia degree of the first human eye of the current user is The degree of power and/or the degree of astigmatism is indicated by the virtual display information corresponding to the user characteristics of the current user.

In a possible design, the virtual display device stores correspondences between different user characteristics and corresponding virtual display information. The processor searches for a matching entry from the corresponding relationship according to the user characteristics of the current user, and if there is a matching entry, determines that the virtual display information corresponding to the current user is stored in the matching entry virtual display information. The virtual display information includes the degree of myopia and/or the degree of astigmatism of the corresponding user.

In a possible design, when there is no matching item corresponding to the user characteristics of the current user in the corresponding relationship, the first optical display module displays the first image under the control of the processor, and the The optical power when displaying the first image is the optical power matching the myopia degree and/or the astigmatism degree of the first human eye of the current user, and the myopia degree and/or the astigmatism degree of the first human eye is the processing automatically determined by the instrument, or determined under the instruction of the user, or manually input by the user.

In a possible design, the processor stores a correspondence between the degree of myopia and/or the degree of astigmatism of the first human eye and the user characteristics of the current user.

In a seventh aspect, a virtual display device is provided. The virtual display device includes one or more processors and one or more memories; one or more memories are coupled with one or more processors, and one or more memories store computer instructions; when one or more processors execute computer When instructing, the virtual display device is made to realize the function of the virtual display device in any one of the above-mentioned first aspect and various possible designs; or realize the function of the virtual display device in any one of the above-mentioned second aspect and various possible designs function; or realize the function of the virtual display device in any one of the third aspect and various possible designs; or realize the function of the virtual display device in the fourth aspect and any one of the various possible designs; or realize The function of the virtual display device in any one of the above fifth aspect and various possible designs.

In the eighth aspect, a chip system is provided, the chip system includes an interface circuit and a processor; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive signals from the memory and send signals to the processor, and the signals include the data stored in the memory Computer instruction; when the processor executes the computer instruction, the virtual display device provided with the system-on-a-chip realizes the function of the virtual display device in any one of the above-mentioned first aspect and various possible designs; or realizes the above-mentioned second aspect and each The function of any one of the virtual display device in the possible designs; or realize the function of the virtual display device in the third aspect and any of the various possible designs; or realize the fourth aspect and the various possible designs above any one of the functions of the virtual display device; or realize the function of the virtual display device of any one of the fifth aspect and various possible designs.

In the ninth aspect, a computer-readable storage medium is provided, the computer-readable storage medium includes computer instructions, and when the computer instructions are run, the functions of the virtual display device in the above-mentioned first aspect and any one of various possible designs are realized; Or realize the function of the virtual display device in any one of the above second aspect and various possible designs; or realize the function of the virtual display device in the above third aspect and any one of various possible designs; or realize the above first aspect The function of the virtual display device in any one of the four aspects and various possible designs; or realize the function of the virtual display device in the fifth aspect and any one of the various possible designs.

It should be understood that the technical features of the technical solutions provided in the second aspect to the ninth aspect above can all correspond to the shooting methods provided in the first aspect and its possible design, so the beneficial effects that can be achieved are similar, and here Let me repeat.

Description of drawings

Fig. 1 is a schematic diagram of a convergence angle;

Fig. 2 is a schematic diagram of the composition of a human eye;

3 is a schematic diagram of an imaging mechanism of a human eye;

4 is a schematic diagram of an imaging mechanism of VR glasses;

FIG. 5A is a schematic diagram of another imaging mechanism of VR glasses;

5B is a schematic diagram of another imaging mechanism of VR glasses;

FIG. 6A is a schematic diagram of the composition of a wearable device provided by an embodiment of the present application;

FIG. 6B is a schematic diagram of the composition of an optical display module provided by the embodiment of the present application;

FIG. 7 is a schematic diagram of the composition of a folded optical mirror group provided by the embodiment of the present application;

Fig. 8 is a working schematic diagram of a folded optical mirror group provided by the embodiment of the present application;

FIG. 9 is a schematic diagram of the composition of a VR glasses provided by the embodiment of the present application;

Fig. 10 is a schematic diagram of imaging of VR glasses provided by the embodiment of the present application;

FIG. 11 is a schematic flowchart of a virtual display method provided by an embodiment of the present application;

Fig. 12 is a schematic diagram of imaging of another kind of VR glasses provided by the embodiment of the present application;

Fig. 13 is a schematic diagram of a display effect provided by the embodiment of the present application;

Fig. 14 is a schematic diagram of yet another imaging mechanism of the human eye;

Fig. 15 is a schematic diagram of yet another imaging mechanism of the human eye;

Fig. 16 is a schematic diagram of an eye chart provided in an embodiment of the present application;

Fig. 17 is a schematic diagram of a visual score provided by the embodiment of the present application;

FIG. 18 is a schematic flowchart of another virtual display method provided by the embodiment of the present application;

FIG. 19 is a schematic diagram of an imaging mechanism provided by an embodiment of the present application;

FIG. 20 is a schematic diagram of an imaging mechanism provided by an embodiment of the present application;

FIG. 21 is a schematic diagram of an imaging mechanism provided by an embodiment of the present application;

Fig. 22 is a schematic diagram of a detection image provided by the embodiment of the present application;

Fig. 23 is a schematic diagram of an input mode of recognition feedback provided by an embodiment of the present application;

Fig. 24 is a schematic diagram of a display mechanism provided by the embodiment of the present application;

Fig. 25 is a schematic diagram of an additional display mechanism provided by the embodiment of the present application;

FIG. 26 is a schematic diagram of an interaction scene of VR glasses provided by an embodiment of the present application;

Fig. 27 is a schematic flowchart of another virtual display method provided by the embodiment of the present application;

Fig. 28 is a schematic diagram of a red-green balance method provided in the embodiment of the present application;

Fig. 29 is a schematic diagram of another imaging mechanism provided by the embodiment of the present application;

FIG. 30 is a schematic diagram of another imaging mechanism provided by the embodiment of the present application;

Fig. 31 is a schematic diagram of an astigmatism detection chart provided by an embodiment of the present application;

Fig. 32 is a schematic flowchart of another virtual display method provided by the embodiment of the present application;

FIG. 33 is a schematic diagram of the composition of an electronic device provided in the embodiment of the present application;

FIG. 34 is a schematic diagram of the composition of a chip system provided by an embodiment of the present application.

Detailed ways

In the following, some terms used in the embodiments of the present application are explained, so as to facilitate the understanding of those skilled in the art.

(1) At least one of the embodiments of the present application involves one or more; wherein, a plurality means greater than or equal to two. In addition, it should be understood that in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as express or implied relative importance, nor can they be understood as express or imply order. For example, the first object and the second object do not represent the importance of the two, or represent the order of the two, but to distinguish the objects.

In the embodiment of this application, "and/or" is an association relationship describing associated objects, indicating that there may be three relationships, for example, A and/or B, which may mean: A exists alone, and A and B exist at the same time , there are three cases of B alone. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.

(2) Virtual Reality (VR) technology is a means of human-computer interaction created with the help of computer and sensor technology. VR technology integrates computer graphics technology, computer simulation technology, sensor technology, display technology and other science and technology to create a virtual environment. The virtual environment includes three-dimensional realistic images generated by computers and dynamically played in real time to bring visual perception to users; moreover, in addition to the visual perception generated by computer graphics technology, there are also perceptions such as hearing, touch, force, and movement. It even includes the sense of smell and taste, also known as multi-sensing; in addition, it can also detect the user's head rotation, eyes, gestures, or other human behaviors, and the computer will process the data that is suitable for the user's actions, and The user's actions respond in real time and are fed back to the user's five sense organs respectively to form a virtual environment. Exemplarily, the user can see the VR game interface by wearing the VR wearable device, and can interact with the VR game interface through operations such as gestures and handles, as if in a game.

(3) Augmented Reality (AR) technology refers to superimposing computer-generated virtual objects on real-world scenes to enhance the real world. In other words, AR technology needs to collect real-world scenes, and then add a virtual environment to the real world.

Therefore, the difference between VR technology and AR technology is that AR technology creates a complete virtual environment, and all users see are virtual objects; while AR technology superimposes virtual objects on the real world, that is, it includes objects in the real world. Also includes dummy objects. For example, the user wears transparent glasses, through which the real environment around can be seen, and virtual objects can also be displayed on the glasses, so that the user can see both real objects and virtual objects.

(4) Mixed reality technology (Mixed Reality, MR) is to build a bridge of interactive feedback information between the virtual environment, the real world and users by introducing real scene information (or called real scene information) into the virtual environment. , thereby enhancing the realism of the user experience. Specifically, the real object is virtualized (for example, using a camera to scan the real object for 3D reconstruction to generate a virtual object), and the virtualized real object is introduced into the virtual environment, so that the user can see in the virtual environment real object.

It should be noted that the technical solutions provided in the embodiments of the present application may be applicable to electronic devices of technologies such as VR, AR, or MR. Electronic devices can provide users with virtual display functions through AR, VR or MR technologies. In this way, the user can experience the stereoscopic vision in the virtual scene through the virtual display function without being in the actual environment.

In order to clearly explain the virtual display function provided to users through AR, VR or MR technology, the following first briefly describes the mechanism of human vision generation.

It can be understood that, in an actual scene, when a user views an object, the human eye can obtain a light signal in the actual scene and process the light signal in the brain to realize visual experience. Wherein, the optical signal in the actual scene may include reflected light from different objects, and/or an optical signal directly emitted by a light source. Since the light signal of the actual scene can carry the relevant information (such as size, position, color, etc.) of each object in the actual scene, the brain can obtain the information of the object in the actual scene by processing the light signal , that is, to obtain visual experience.

It should be noted that when human eyes (such as left and right eyes) watch the same object, their sight angles are slightly different. Therefore, the scene seen by the left eye and the right eye is actually different. For example, the left eye can acquire the light signal of the two-dimensional image (hereinafter referred to as the left eye image) on the plane where the focus of the human eye is perpendicular to the line of sight of the left eye. Similarly, the right eye can obtain the light signal of the two-dimensional image (hereinafter referred to as the right eye image) on the plane where the focus of the human eye is perpendicular to the line of sight of the right eye. The image for the left eye is slightly different from the image for the right eye. The brain can obtain information about different objects in the current scene by processing the light signals of the left-eye image and the right-eye image.

In addition, users can also obtain stereoscopic vision experience by obtaining the depth of different objects in the actual scene. Stereo vision experience can also be called binocular stereo vision.

Exemplarily, when the brain looks at an object in an actual scene, the brain can determine the depth of the object (that is, the depth of field) through the vergence distance (Vergence distance) and the zoom depth (Accommodation distance) of the eyes.

Among them, the brain can determine the depth of convergence according to the mechanism shown in Figure 1 . As shown in Figure 1, when the two eyes observe an object in an actual scene, the sight of the left eye and the right eye can be turned to the object by controlling the muscles near the human eye. The brain can judge the depth of the object by obtaining the convergence angle of the eyes, that is, the convergence depth. As an example, as shown in FIG. 1 , when observing the object shown in FIG. 1 , the convergence angle may be the included angle between the sight lines of the two eyes at the position where the object is observed as shown in the figure. It is understandable that the closer the observed object is to the human eye, the larger the convergence angle and the smaller the convergence depth. Correspondingly, the farther the observed object is from the human eye, the smaller the convergence angle and the greater the convergence depth.

In addition, the brain can also judge the depth of the object according to the zoom depth. Exemplarily, the zoom depth is described with reference to FIG. 2 and FIG. 3 . Figure 2 shows a schematic diagram of the composition of the human eye. As shown in FIG. 2 , the human eye may include a lens, a ciliary muscle, and a retina located in the fundus.

The crystalline lens acts as a zoom lens, converging the rays of light entering the eye. In order to converge the incident light onto the retina of the human eye fundus, so that the scene in the actual scene can form a clear image on the retina. The ciliary muscle can be used to adjust the shape of the lens. For example, the ciliary muscle can adjust the diopter of the lens by contracting or relaxing, so as to achieve the effect of adjusting the focal length of the lens. Therefore, objects at different distances in the actual scene can be clearly imaged on the retina through the lens. As an example, refer to FIG. 3 , which is a diagram illustrating the adjustment of the ciliary muscle to the lens when the human eye observes objects at different distances. As shown in (a) of FIG. 3 , when the human eye observes a distant object, it is taken as an example that the object is a non-light source. The reflected rays from the surface of this object can be close to parallel rays. At this time, the ciliary muscle can control the state of the lens to the state shown in Figure 3 (a), such as the ciliary muscle relaxes, and controls the lens to be flat and the diopter is small, so that parallel incident light can pass through the lens Converge on the retina of the fundus. When the human eye observes a relatively close object, in combination with (b) in FIG. 3 , take the object as a non-light source as an example. The reflected light from the surface of the object may enter the human eye according to the optical path shown in (b) in FIG. 3 . At this time, the ciliary muscle can keep the state of the lens in the state shown in (b) in Figure 3, as the ciliary muscle contracts, the lens protrudes, and the diopter becomes larger, so that the lens shown in (b) in Figure 3 Incident light can pass through the lens and then converge on the retina in the fundus of the eye.

That is to say, when the human eye observes objects at different distances, the contraction or relaxation state of the ciliary muscle is different. In this way, the brain can judge the depth of the object through the contraction or relaxation of the anterior ciliary muscle when the human eye clearly observes the object. This depth may be referred to as zoom depth.

At present, electronic devices can display virtual scenes to users through AR, VR or MR technologies in combination with the above-mentioned generation mechanism of human vision, so as to provide virtual display functions.

Exemplarily, an electronic device that provides a virtual display function through AR, VR or MR technology is VR glasses with a VR display function as an example. 4 is a schematic diagram of VR glasses. As shown in FIG. 4 , two display screens (such as a display screen 1 and a display screen 2 ) may be provided in the VR glasses, and each display screen has a display function. Each display screen can be used to display corresponding content to one eye (such as left eye or right eye) of the user through a corresponding eyepiece. For example, on the display screen 1, the corresponding left-eye image in the virtual scene can be displayed. The light of the left-eye image can pass through the eyepiece 1 and converge at the left eye, so that the left eye can see the left-eye image. Similarly, on the display screen 2, the corresponding right-eye image in the virtual scene can be displayed. The light of the right-eye image can pass through the eyepiece 2 and converge at the left eye, so that the right eye can see the right-eye image.

Thus, the brain can fuse the left-eye image and the right-eye image, so that the user can see objects in the virtual scene corresponding to the left-eye image and the right-eye image.

It should be noted that due to the converging effect of the eyepieces, as shown in FIG. 5A , the image seen by human eyes is actually an image on the virtual image plane shown in FIG. 5A corresponding to the image displayed on the display screen. For example, the left-eye image viewed by the left eye may be a virtual image corresponding to the left-eye image on the virtual image plane. For another example, the right-eye image viewed by the right eye may be a virtual image corresponding to a limited image on the virtual image plane.

It is understandable that when observing an object in an actual scene, the depth of the object judged by the brain based on the convergence depth and the zoom depth may be consistent. However, when the depth of the object indicated by the depth of convergence and the depth of zooming are inconsistent, visual fatigue will occur and affect the visual experience of the user. In this example, the depth inconsistency of the object indicated by the vergence depth and the zoom depth may also be referred to as vergence accommodation conflict (Vergence accommodation conflict, VAC).

However, when the VR glasses display the virtual scene to the user through the solution shown in FIG. 4 or FIG. 5A , the depth of the object indicated by the convergence depth and the zoom depth will be inconsistent.

For example, refer to FIG. 5B. When the human eye observes the corresponding display screen, in order to be able to see the images on the display screen clearly, the ciliary muscle will adjust the lenses of both eyes so that the images on the virtual image plane can be converged on the retina through the lenses. Therefore, the zoom distance may be the distance from the virtual image plane to the human eye (depth 1 as shown in FIG. 5B ). However, the objects in the virtual scene displayed by the VR glasses to the user are often not on the virtual image plane. For example, in the virtual scene, the object to be observed is a triangle as shown in FIG. 5B as an example. The user can focus the eyesight of both eyes on a triangle in the virtual scene (such as the dotted triangle 501 ) by rotating the eyeballs. The angle of convergence is indicated by the label in Fig. 5B. In this way, the depth of convergence should be the depth of the observed object (such as the dotted triangle 501 ) in the virtual scene. For example, the depth of convergence may be depth 2 as shown in FIG. 5B .

It can be seen that depth 1 and depth 2 are not consistent at this time. In this way, the brain will not be able to accurately judge the depth of the observed object, thereby causing visual fatigue and other situations that affect user experience. If it is like this for a long time, it will also have a serious impact on the user's vision.

In order to solve the above problems, the embodiment of the present application provides a virtual display device and a virtual display method, which can avoid VAC that occurs during the process of providing virtual display functions to users through AR, VR or MR technology, thereby avoiding the resulting Fatigue, improve the user's visual experience.

The solutions provided by the embodiments of the present application will be described in detail below with reference to examples and accompanying drawings.

For example, please refer to FIG. 6A , which shows a schematic structural diagram of a wearable device provided by an embodiment of the present application, taking the virtual display device as a wearable device as an example. As shown in Figure 6A, the wearable device 100 can include a processor 110, a memory 120, a sensor module 130 (which can be used to obtain the user's gesture), a microphone 140, a button 150, an input and output interface 160, a communication module 170, a camera 180, a battery 190. An optical display module 1100, an eye tracking module 1200, and the like.

It can be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the wearable device 100 . In other embodiments of the present application, the wearable device 100 may include more or fewer components than shown in the illustration, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 110 is generally used to control the overall operation of the wearable device 100, and may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor ( graphics processing unit, GPU), image signal processor (image signal processor, ISP), video processing unit (video processing unit, VPU) controller, memory, video codec, digital signal processor (digital signal processor, DSP) , a baseband processor, and/or a neural-network processing unit (NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.

In some embodiments of the present application, the processor 110 may be used to control the optical power of the wearable device 100 . Exemplarily, the processor 110 may be used to control the optical power of the optical display module 1100 to realize the function of adjusting the optical power of the wearable device 100 . For example, the processor 110 can adjust the relative positions of the various optical devices (such as lenses, etc.) When the human eye is imaging, the position of the corresponding virtual image plane can be adjusted. In this way, the effect of controlling the optical power of the wearable device 100 is achieved.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, a universal asynchronous receiver/transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general input and output (general -purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and/or universal serial bus (universal serial bus, USB) interface, serial peripheral interface (serial peripheral interface, SPI) interface etc.

In some embodiments, the processor 110 may render different objects based on different frame rates, for example, use a high frame rate for rendering for nearby objects, and use a low frame rate for rendering for distant objects.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 110 may include multiple sets of I2C buses.

The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 and the communication module 170 . For example: the processor 110 communicates with the Bluetooth module in the communication module 170 through the UART interface to realize the Bluetooth function.

The MIPI interface can be used to connect the processor 110 with the display screen in the optical display module 1100 , the camera 180 and other peripheral devices.

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 180 , the display screen in the optical display module 1100 , the communication module 170 , the sensor module 130 , the microphone 140 and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc. Optionally, the camera 180 can collect images including real objects, and the processor 110 can fuse the images collected by the camera with the virtual objects, and realize the fused images through the optical display module 1100. For this example, refer to the application shown in FIG. 9 The scene will not be repeated here.

The USB interface is an interface that conforms to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface can be used to connect a charger to charge the wearable device 100, and can also be used to transmit data between the wearable device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as mobile phones. The USB interface may be USB3.0, which is compatible with high-speed display port (DP) signal transmission, and can transmit video and audio high-speed data.

It can be understood that the interface connection relationship between the modules shown in the embodiment of the present application is only a schematic illustration, and does not constitute a structural limitation of the wearable device 100 . In other embodiments of the present application, the wearable device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

In addition, the wearable device 100 may include a wireless communication function. For example, the wearable device 100 may receive rendered images from other electronic devices (such as a VR host or a VR server) for display, or may receive an unrendered image and then the processor 110 may process the image Render and display. The communication module 170 may include a wireless communication module and a mobile communication module. The wireless communication function can be realized by an antenna (not shown), a mobile communication module (not shown), a modem processor (not shown), and a baseband processor (not shown).

Antennas are used to transmit and receive electromagnetic wave signals. Multiple antennas may be included in the wearable device 100, and each antenna may be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module can provide applications on the wearable device 100 including second generation (2th generation, 2G) network/third generation (3th generation, 3G) network/fourth generation (4th generation, 4G) network/fifth generation ( 5th generation, 5G) network and other wireless communication solutions. The mobile communication module may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module can receive electromagnetic waves through the antenna, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module can also amplify the signal modulated by the modem processor, and convert it into electromagnetic wave and radiate it through the antenna. In some embodiments, at least part of the functional modules of the mobile communication module may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module and at least part of the modules of the processor 110 may be set in the same device.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs sound signals through audio equipment (not limited to speakers, etc.), or displays images or videos through the display screen in the optical display module 1100 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module or other functional modules.

The wireless communication module can provide applications on the wearable device 100 including wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (wireless fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module may be one or more devices integrating at least one communication processing module. The wireless communication module receives electromagnetic waves through the antenna, frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module can also receive the signal to be sent from the processor 110, frequency-modulate it, amplify it, and convert it into electromagnetic wave through the antenna to radiate out.

In some embodiments, the antenna of the wearable device 100 is coupled to the mobile communication module, so that the wearable device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and/or IR technology, etc. GNSS can include global positioning system (global positioning system, GPS), global navigation satellite system (global navigation satellite system, GLONASS), Beidou satellite navigation system (beidou navigation satellite system, BDS), quasi-zenith satellite system (quasi- zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).

The wearable device 100 realizes the display function through the GPU, the optical display module 1100 , and the application processor. The GPU is a microprocessor for image processing, and is connected to the optical display module 1100 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The memory 120 may be used to store computer-executable program code, including instructions. The processor 110 executes various functional applications and data processing of the wearable device 100 by executing instructions stored in the memory 120 . The memory 120 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The data storage area can store data created during use of the wearable device 100 (such as audio data, phonebook, etc.) and the like. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.

The wearable device 100 can implement audio functions through an audio module, a speaker, a microphone 140, an earphone interface, and an application processor. Such as music playback, recording, etc.

The audio module is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module can also be used to encode and decode audio signals. In some embodiments, the audio module may be set in the processor 110 , or some functional modules of the audio module may be set in the processor 110 .

Loudspeakers, also called "horns", are used to convert audio electrical signals into sound signals. The wearable device 100 can listen to music through the speaker, or listen to hands-free calls.

The microphone 140, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. The wearable device 100 may be provided with at least one microphone 140 . In other embodiments, the wearable device 100 can be provided with two microphones 140, which can also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the wearable device 100 can also be provided with three, four or more microphones 140 to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions, etc.

The headphone jack is used to connect wired headphones. The headphone interface can be a USB interface, or a 3.5mm (mm) open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface .

In some embodiments, the wearable device 100 may include one or more buttons 150 , and these buttons may control the wearable device and provide users with access to functions on the wearable device 100 . Keys 150 may be in the form of buttons, switches, dials, and touch or near-touch sensing devices such as touch sensors. Specifically, for example, the user can turn on the optical display module 1100 of the wearable device 100 by pressing a button. The keys 150 include a power key, a volume key and the like. The key 150 may be a mechanical key. It can also be a touch button. The wearable device 100 can receive key input and generate key signal input related to user settings and function control of the wearable device 100 .

In some embodiments, the wearable device 100 may include an input-output interface 160, and the input-output interface 160 may connect other devices to the wearable device 100 through suitable components. Components may include, for example, audio/video jacks, data connectors, and the like.

The optical display module 1100 is used for presenting images to the user under the control of the processor. The optical display module 1100 can convert the real pixel image display into a near-eye projection virtual image display through one or several optical devices such as mirrors, transmission mirrors, or optical waveguides, so as to realize virtual interactive experience, or realize virtual and Interactive experience combined with reality. For example, the optical display module 1100 receives image data information sent by the processor, and presents corresponding images to the user.

In some embodiments, the wearable device 100 may further include an eye tracking module 1200, which is used to track the movement of human eyes, and then determine the point of gaze of the human eyes. For example, the position of the pupil can be located by image processing technology, the coordinates of the center of the pupil can be obtained, and then the gaze point of the person can be calculated.

With reference to the description of the wearable device 100 shown in FIG. 6A , the virtual display device provided by the embodiment of the present application has the function of automatically adjusting the optical power. In some embodiments, this function can be realized by the optical display module 1100 .

For example, please refer to FIG. 6B , which is a schematic composition diagram of an optical display module provided by an embodiment of the present application. The virtual display device can be used to support AR, VR or MR technologies to provide a virtual display function. In a specific implementation, the virtual display device may be a head-mounted display (Head-mounted display, HMD) device, such as AR, VR or MR glasses, an AR, VR or MR helmet, or an AR, VR or MR all-in-one machine. Alternatively, the virtual display device may also be included in the above example head-mounted virtual display device. It should be noted that, in some embodiments, the virtual display device can also be used to support the realization of mixed reality (Mixed Reality, MR) technology.

As shown in FIG. 6B , the optical display module may include an eyepiece 601 , a zoom module 602 , and a display screen 603 .

Wherein, the eyepiece 601 may be a Fresnel lens, and/or an optical device or device group such as an aspherical lens. Eyepiece 601 may be used to project light from display screen 603 into the user's eyes. In some embodiments, eyepiece 601 may have positive optical power. As a result, the light from the display screen 603 can be converged into the human eye through the eyepiece 601 in a small space. Exemplarily, in different implementations, when the distance from the human eye to the display screen 603 is different, the display screen 603 can be projected to different virtual image distance positions by adjusting the distance between the eyepiece 601 and the display screen 603 . The virtual image distance position may be the position of the user's eyes.

As an example, FIG. 7 shows a schematic diagram of a pancake folding optical lens assembly provided in an embodiment of the present application. The Pancake folding optical lens group (hereinafter referred to as the Pancake lens group) can be used to realize the function of the eyepiece as shown in FIG. 6B . As shown in FIG. 7 , the pancake lens set may include at least 5 optical components (such as 701-705). Wherein, the specific implementation of any one of the optical components in 701-705 may be an optical lens with a corresponding function, or the corresponding function of the component may be realized by an optical coating on an adjacent lens or other optical components.

In the example shown in FIG. 7 , the pancake lens group is from the object side to the image side, and the order of the optical components can be 701-702-703-704-705. It can be seen that in FIG. 7 , a specific example of each optical component is also shown. In this example, 701 may be a polarizer (Polarizer, P). 702 may be a quarter wave plate (Quarter wave-plate, QWP). 703 can be realized by a semi-transverse and semi-permeable membrane (Beam splitter, BS). 704 may be a quarter wave plate. 705 can be realized by polarizing reflector (Polarization reflector, PR).

The following is an example to explain the light processing mechanism of the Pancake lens group when it is working. Please refer to FIG. 8 , which is an example in which the object-space light is emitted from the display screen of the optical display module, and the image-space light enters the human eye.

As shown in the figure, after the incident light is incident 701, it can be transmitted sequentially in the order of 702-703-704-705.

Exemplarily, the light may be modulated into linearly polarized light after passing through 701 . In some embodiments, the modulation direction of 701 may be set as the y-axis direction. Thus, after passing through 701 , the incident light can be modulated into linearly polarized light in the y-axis direction. Next, the linearly polarized light may pass through 702, thereby being adjusted to rotated polarized light. Exemplarily, if the fast axis direction of 702 is 45° to the y-axis, then the linearly polarized light can be adjusted to right-handed polarized light after passing through 702 . The right-handed polarized light can be incident 703 . Due to the transflective property of the 703 , part of the right-handed polarized light can be transmitted through the 703 , while the other part of the light can be reflected by the 703 . Right-handed polarized light transmitted 703 may be incident 704 . In the case that the fast axis direction of 704 is the same as that of 702 , the right-handed polarized light transmitted through 703 can directly pass through 704 and enter 705 . The right-handed polarized light incident 705 can be modulated into linearly polarized light along the x direction, and reflected on the surface of 705 .

Light reflected by 705 can pass through 704-703 and be reflected at 703.

Exemplarily, the light reflected on the surface of 705 can pass through 704 and be modulated into right-handed polarized light, and a part of the light of the right-handed polarized light can be reflected on the surface of 703 . It should be noted that after reflection from the 703 surface, the right-handed polarized light can be modulated into left-handed polarized light.

The left-handed polarized light reflected by 703 can exit the Pancake lens group through 704-705, and finally enter the human eye.

Exemplarily, after step 704, the left-handed polarized light may be modulated into linearly polarized light whose polarization direction is along the y-axis direction. Then, the linearly polarized light along the y-axis direction can exit the Pancake lens group through 705 and enter human eyes. It can be understood that, in some embodiments, the polarized transmission characteristic of 705 can be set to transmit linearly polarized light in the y-axis direction, thereby ensuring that the linearly polarized light in the y-axis direction can be emitted from 705 smoothly.

In this way, the light can be folded and transmitted in the Pancake mirror group through the order of 701-702-703-704-705-704-703-704-705, thereby achieving the effect of folding the optical path. Therefore, light transmission with a longer optical path can be realized in a small space (such as inside an optical display module such as VR glasses).

In some embodiments, the zoom module 602 can adjust the distance between one or more optical components (such as lenses) in the eyepiece 601 to change the optical power, thereby adjusting the depth of the virtual plane. For example, the zoom module 602 can change the relative position of one or more optical components in the pancake lens group to change the optical power, thereby adjusting the depth of the virtual surface.

It should be noted that the above composition in FIG. 7 or FIG. 8 is only an example, and in other embodiments of the present application, the pancake lens group may also include more or less optical components. In addition, the functions implemented by the optical lens in the above examples may also be implemented by other optical components, such as by performing corresponding optical coatings on adjacent optical lenses.

It can be understood that, in combination with the aforementioned description of the eyepiece in the optical display module, through the description of the above-mentioned Figure 7 or Figure 8, based on the Pancake mirror group, the optical path of the mirror group can be folded to increase the optical power of the mirror group, thereby shortening the The effect of focal length. In this way, the length of the lens barrel (that is, the optical display module) along the optical axis can be shortened. Therefore, the optical display module can meet the miniaturization requirement of the virtual display device.

In different embodiments of the present application, the specific implementation of the eyepiece as shown in FIG. 6B may be different. For example, the eyepiece can be realized by using the pancake lens group shown in FIG. 7 or FIG. 8 in the above example. As another example, the eyepiece can also use other optical lenses or mirror groups to project the light from the display screen to the human eye.

In the optical display module shown in Figure 6B, the display screen 603 may include, but not limited to, a liquid crystal display (Liquid Crystal Display, LCD), an organic light-emitting semiconductor display (Organic Light-Emitting Diode, OLED), a micro-luminescence semiconductor Display components such as display screen (Micro Light-Emitting Diode, Micro-LED), quantum dot light-emitting semiconductor display (Quantum Dots Light-Emitting Diode, QLED). In the embodiment of the present application, the display screen is used as an image source of the optical display module and can be used to display images to be displayed.

In the optical display module, the function of the zoom module 602 can be realized by components with automatic zoom functions such as a mechanical zoom mechanism, a liquid crystal zoom device, a liquid zoom device, and an Alvarez lens. In some embodiments of the present application, the zoom module 602 can achieve the purpose of zooming the optical display module by changing its own optical power.

Exemplarily, in some embodiments, with reference to the foregoing description of FIG. 6A , based on the composition of FIG. 6B , the processor in the virtual display device can be used to control the zoom module to adjust the optical power. By adjusting the optical power, the optical display module can adjust the distance from the virtual image plane to the user's eyes during the process of providing virtual display to the user, thereby realizing the matching of the depth of convergence and the depth of focus. In some other embodiments, the processor can also realize the matching between the optical display module and the degree of myopia of the user during the display process. In some other embodiments, the processor may also be used to control the rotation of the zoom module so as to match the axis position corresponding to the user's astigmatism degree, thereby matching the user's astigmatism degree.

In the following examples of the present application, the operations controlled and executed by the optical display module/VR glasses can all be completed by the processor, which will not be described in detail below.

It should be noted that the composition of the optical display module shown in FIG. 6B is only a logical illustration. In a specific implementation, the number of eyepieces, zoom modules and display screens can be flexibly set according to different requirements. For example, in some embodiments, the optical display module may include two eyepieces, two zoom modules, and two display screens. Wherein, an eyepiece, a zoom module and a display screen can constitute a sub-optical display module for providing a virtual display function to one of the user's eyes (such as the left eye or the right eye). As another example, in other embodiments, the eyepieces that provide virtual display functions for the left eye and the right eye can also be integrated on the same component, so that only one eyepiece can be included in the optical display module. Similarly, in some other embodiments, the zoom module and/or the display screen may also be integrated on one or more components, instead of being set independently. For example, the zoom component can be integrated with the eyepiece in the same component (eg, eyepiece lens set). As another example, the eyepiece can be integrated in the zoom module.

In addition, in different embodiments of the present application, the specific positions of the eyepiece, the zoom module and the display screen in the optical display module may also be different. For example, in some embodiments, in the optical display module, according to the line of sight of the human eye, the position sequence of the above components on the optical path may be: eyepiece, zoom module, and display screen. In some other embodiments, the zoom module can be arranged between the eyepiece and the human eye. That is, according to the line-of-sight direction of the human eye, the position sequence of each component may be: a zoom module, an eyepiece, and a display screen.

As an example, FIG. 9 shows a specific implementation example of a virtual display device having functions related to the optical display module described above. In this example, the virtual display device may be VR glasses. Wherein, the VR glasses may include two groups of optical display modules having the composition as shown in FIG. 6B . In some embodiments, the composition of the VR glasses as shown in FIG. 9 can also be another division of the wearable device as shown in FIG. 6A , and related modules can also implement corresponding functions.

For example, the VR glasses may include a display screen L, a zoom module L, and a first display component composed of an eyepiece L. The first display component may be used to provide a virtual display function to the user's left eye. The VR glasses may also include a display screen R, a zoom module R, and a second display component composed of an eyepiece R. The second display component may be used to provide a virtual display function to the user's right eye. It should be noted that, in the embodiment of the present application, the display component may also be referred to as an optical display module. For example, the first display component may be referred to as a first optical display module. As another example, the second display component may also be referred to as a second optical display module.

It should be noted that, in some embodiments of the present application, the VR glasses may also include other components (such as an eye-tracking system) for realizing eye-tracking of the user. The eye-tracking system can determine the position of the user's fixation point (or determine the direction of the user's line of sight) through methods such as video eye diagram method, photodiode response method, or pupil-corneal reflection method, thereby realizing user's eye-tracking.

In some embodiments, it is taken as an example to determine a user's line of sight direction by using a pupil-cornea reflection method. The eye tracking system may include one or more near-infrared light-emitting diodes (Light-Emitting Diode, LED) and one or more near-infrared cameras. The near-infrared LED is not shown in FIG. 9 . In a different example, the near-infrared LEDs can be positioned around the eyepiece so as to fully illuminate the human eye. In some embodiments, the near-infrared LED may have a center wavelength of 850 nm or 940 nm. The eye tracking system can obtain the user's line of sight direction through the following methods: the human eye is illuminated by near-infrared LEDs, and the near-infrared camera captures the image of the eyeball, and then according to the position of the reflection point of the near-infrared LED on the cornea and the pupil in the eyeball image The center of the eyeball is determined to determine the direction of the optical axis of the eyeball, and finally the direction of the user's line of sight is obtained through the calibration procedure that the user participates in.

It should be noted that, in some embodiments of the present application, respective eye-tracking systems (infrared camera R and infrared camera L as shown in FIG. Eye tracking with both eyes. In some other embodiments of the present application, an eye-tracking system can also be set only near one human eye, and the line-of-sight direction of the corresponding human eye can be obtained through the eye-tracking system, and according to the relationship between the gaze points of the two eyes (for example, when the user passes through When observing an object with both eyes, the fixation point positions of the two eyes are generally similar or the same), combined with the user's binocular distance, the line of sight direction or fixation point position of the other eye can be determined.

In the embodiment of this application, the wearable device with the composition shown in Figure 6A, or the virtual display device (such as VR glasses) with the composition shown in Figure 9 can control the optical The zoom module in the display module adjusts the position of the virtual image plane (such as the virtual image plane shown in FIG. 5A or FIG. 5B ). This in turn adjusts the zoom depth of the human eye. The adjusted zoom depth may be close to or equal to the depth of convergence corresponding to the observation of objects in the virtual scene by human eyes (or referred to as the matching of the zoom depth and the depth of convergence). In this way, visual fatigue caused by different convergence depths and zoom depths can be avoided, and user experience can be improved.

As an example, referring to FIG. 10 , in the embodiment of the present application, a virtual display device (such as VR glasses) controls the zoom module, and then adjusts the zoom depth to match the convergence depth. Wherein, a display component provided in VR glasses is taken as an example. For a specific implementation manner of another display component, reference may be made to this example, and details are not repeated here.

Exemplarily, in some embodiments, when the user observes a nearby object in the virtual scene (for example, when observing the object, the corresponding depth of convergence may be at the position of the virtual image plane 1 as shown in (a) in FIG. 10 ) , the VR glasses can control the price-changing module to adjust its own optical power (for example, adjusted to optical power A), so that the position of the virtual image surface corresponding to the display component falls on the virtual image surface 1 as shown in (a) in Figure 10 Location. In this way, the matching between the zoom depth and the convergence depth when the user observes close objects in the virtual scene is realized.

In some other embodiments, when the user observes a distant object in the virtual scene (for example, when observing the object, the corresponding depth of convergence may be at the position of the virtual image plane 1 as shown in (b) in FIG. 10 ), the VR glasses The zoom module can be controlled to adjust its own optical power (for example, adjusted to optical power B), so that the position of the virtual image plane corresponding to the display component falls on the position of the virtual image plane 2 as shown in (b) in FIG. 10 . In this way, the matching between the zoom depth and the convergence depth when the user observes distant objects in the virtual scene is realized.

As a possible implementation, the optical power B may be smaller than the optical power A. For example, the optical power A can be -1D, and the optical power B can be -3D.

In order to enable those skilled in the art to more clearly understand the realization of the virtual display method provided by the embodiment of the present application, the virtual display device that executes the virtual display method is the VR glasses with the composition shown in Figure 9 as an example, and this The implementation flow of the virtual display method in the example is used for this description.

Exemplarily, refer to FIG. 11 , which is a schematic flowchart of a virtual display method provided by an embodiment of the present application. As shown in Figure 11, the method may include:

S1101. When presenting the virtual three-dimensional environment to the user, determine the user's line of sight direction through an eye-tracking system.

In this example, the virtual three-dimensional environment may include objects closer to the user and objects farther away from the user. For example, refer to Figure 12. The object closer to the user may be a virtual object 1 as shown in FIG. 12 , and the object farther from the user may be a virtual object 2 as shown in FIG. 12 . It should be noted that, in order to show the user the objects in the virtual three-dimensional environment (such as the virtual object 1 or the virtual object 2), the VR glasses can display different images on the display screen. Referring to FIG. 12 in conjunction with the foregoing description, when displaying virtual objects of different depths to the user, the display screen needs to display different content. For example, take the display screen L as an example. When the virtual object 1 is displayed, the display screen L may display an image including the virtual object 1 . The position of the virtual object 1 in the image may be near the intersection point between the line of sight of the left eye when observing the virtual object 1 and the display screen L as shown in the figure (for example, it is called position 1 ). When the virtual object 2 is displayed, the display screen L may display an image including the virtual object 2 . The position of the virtual object 2 in the image may be near the intersection point between the line of sight of the left eye when observing the virtual object 2 and the display screen L as shown in the figure (for example, it is referred to as position 2 ). Obviously, when virtual objects with different depths are displayed, the positions of objects displayed on the display screen are different. For example, in the scene shown in Figure 12, when a virtual object 2 with a large depth is displayed, the position of the object on the display screen L (that is, position 2) is compared to when a virtual object 1 with a smaller depth is displayed The position of (namely position 1) is closer to the middle position of the display screen.

In the embodiment of the present application, the VR glasses can realize eye movement tracking through methods such as the pupil cornea reflection method in the above description. Exemplarily, in the case that the VR glasses are respectively equipped with eye-tracking systems for both eyes, the line-of-sight directions of the human eyes can be determined according to the eye-tracking systems of the corresponding human eyes. For example, taking the point P1 on the virtual object 1 observed by the human eye as an example, when the human eye observes the virtual object 1, the VR glasses can determine that the line of sight of the human eye can be the line of sight when observing P1 as shown in FIG. 12 . As another example, taking the point P2 on the virtual object 2 observed by the human eye as an example, when the human eye observes the virtual object 2, the VR glasses can determine that the line of sight of the human eye can be the line of sight when observing P2 as shown in FIG. 12 .

S1102. Determine the current depth of convergence according to the user's line of sight direction.

With reference to the foregoing description, when the user observes an object in the virtual scene, the user essentially observes the projection of the corresponding object on the virtual image plane through the left eye and the right eye respectively.

For example, when the user observes the virtual object 1, the left eye can obtain the projection of the virtual object on the corresponding virtual image plane A, and the projection observed by the left eye can be a projection perpendicular to the line of sight of the left eye. The right eye can obtain the projection of the virtual object on the corresponding virtual image plane A, and the projection observed by the right eye can be a projection perpendicular to the line of sight of the right eye. However, if the depth of the virtual image plane A (that is, the zoom depth) is inconsistent with the depth of the virtual object in the virtual scene (that is, the convergence depth), VAC will be generated.

In this example, the VR glasses can determine the depth (that is, the depth of convergence) of the object currently observed by the user in the virtual scene according to the viewing directions of the two eyes determined in S1101. As an example, take the user observing point P1 on the virtual object 1 as an example. The VR glasses can determine the angle of convergence corresponding to the user observing the point P1 as a1 according to the line of sight when the user observes the virtual object 1 with the left eye and the right eye. Similarly, when the user observes the point P2 on the virtual object 2, the VR glasses can determine that the current convergence angle is a2.

In this way, according to the convergence angle, the VR glasses can determine the convergence depth of the current user observation position. Exemplarily, continue to take the example that the user is observing point P1. Since the virtual three-dimensional environment is constructed by the VR glasses, the three-dimensional coordinates of each object in the virtual three-dimensional environment in the virtual three-dimensional environment are known by the VR glasses. In some embodiments, according to the angle between the line of sight (a1 shown in Figure 12) and the distance between the left and right eyes when the left and right eyes look at P1 in the virtual scene, the VR glasses can calculate and obtain The distance between the point P1 currently observed by the user and the user, that is, the depth of convergence.

In some other embodiments, the VR glasses may determine the position of the intersection point of the sight lines of the two eyes in the virtual three-dimensional environment according to the user's sight direction acquired in S1101. Then the position may be the gaze point of the user in the virtual three-dimensional environment. According to the known 3D coordinates of the gaze point in the virtual 3D environment, combined with the 3D coordinates of the user in the virtual 3D environment, the VR glasses can calculate the distance between the user and the gaze point, and thus obtain the convergence depth.

It should be noted that in the virtual three-dimensional environment, the user's line of sight may not exactly intersect at one point. Therefore, in some embodiments of the present application, the VR glasses can determine the three-dimensional coordinates of the gaze point in the virtual three-dimensional environment by the following method:

The VR glasses can use the X coordinate and Y coordinate (such as (X1, Y1)) of the intersection point of the projection of the user's eyes on the XOY plane as the projection coordinates of the user's current gaze point on the XOY plane as (X1, Y1). The VR glasses can also be in the virtual three-dimensional environment according to the line of sight of the left eye, the Z coordinate (such as Z _L ) at the (X1, Y1) position, and the Z coordinate (such as Z L ) at the position (X1, Y1) of the line of sight of the right eye (such as Z _R ), to determine the height of the fixation point. For example, the Z coordinate of the gaze point can be

In this way, the coordinates of the gaze point in the virtual 3D environment can be determined as

In some other embodiments of the present application, when the VR glasses determine the user's dominant eye through eye movement tracking, the fixation point corresponding to the dominant eye may be used as the fixation point during binocular observation.

S1103. Adjust the position of the virtual image plane according to the depth of convergence, so that the zoom depth matches the depth of convergence.

Wherein, the matching between the zoom depth and the convergence depth may include that the zoom depth and the convergence depth are the same, or the difference between the adjustment depth and the convergence depth is smaller than a preset threshold. In the case that the zoom depth matches the vergence depth, VAC generated during long-term use of a virtual display device (such as VR glasses) can be avoided.

Exemplarily, the VR glasses can adjust the position of the virtual image plane by adjusting the optical power of the zoom module, so as to achieve the effect of adjusting the zoom depth. In the embodiment of the present application, the VR glasses can adjust the optical power of the zoom module according to the adjustment manner shown in FIG. 10 . I won't repeat them here.

Referring to FIG. 12 , take the user observing the virtual object 1 as an example. The VR glasses can adjust the virtual image plane to the position of virtual image plane A as shown in FIG. 12 . In this way, in order to see the objects on the virtual image plane A clearly, the human eye can control the ciliary muscle to adjust the state of the lens, so that the objects on the virtual image plane A can be clearly imaged on the retina. At this time, the zoom depth determined by the human eye is the distance from the human eye to the virtual image plane A. Correspondingly, the user can control the rotation of the eyes so as to focus the line of sight on the point P1 in the virtual three-dimensional environment, and because the point P1 is on the virtual image plane A, or the point P1 is close to the virtual image plane A. Therefore, the depth of convergence determined by the user according to the rotation of the eyes may also be (or be close to) the distance from the human eye to the virtual virtual image plane A. In this way, the matching of the zoom depth and the convergence depth is realized.

Similarly, when the user observes the virtual object 2, the VR glasses can adjust the virtual image plane to the position of the virtual image plane B as shown in FIG. 12 , so as to realize the matching between the zoom depth and the convergence depth.

In this way, when the user observes objects in the virtual scene through the VR glasses, there will be no visual fatigue caused by the inconsistency between the zoom depth and the convergence depth. In this way, while providing the virtual display function to the user, the visual experience of the user can be improved and the damage to eyesight can be avoided.

It should be noted that, in some embodiments of the present application, in order to provide users with a better visual experience, the virtual display device can also complete the matching of the zoom depth and the convergence depth according to the method shown in Figure 11 (that is, execute S1103 ), different objects in the currently displayed virtual 3D environment are blurred according to the zoom depth (or convergence depth), so as to highlight that the user can obtain a more intuitive three-dimensional experience when observing objects in the virtual 3D environment.

For example, refer to FIG. 13 in combination with FIG. 12 . In some embodiments, the virtual object 1 that is being observed near is used as an example. The VR glasses can blur the distant view according to the depth of the virtual object 1 . Thus, the user can see an image as shown in (a) in FIG. 13 . It can be seen that in this image, due to the blurring of the distant view, the user can have a clearer observation experience of the currently observed virtual object 1, thereby achieving the simulation of the defocused blurring experience when the user observes the real scene . Similarly, when the user observes the virtual object 1 in the distance, the VR glasses can blur the near view, so as to show the user the image shown in (b) in Figure 13, thereby achieving the effect of protruding the distant view.

It should be noted that when the virtual display device blurs the distant view/near view, it can be determined according to the difference between the depth corresponding to the object to be processed (such as zoom depth or convergence depth) and the depth at which the current user is observing the object. The degree of blurring. For example, take VR glasses as an example to blur the distant view. When the depth corresponding to the object in the foreground is greatly different from the depth of the object in the foreground currently observed by the user (for example, greater than the depth threshold), the object in the foreground is highly blurred. Correspondingly, when the difference between the depth corresponding to the object in the foreground and the depth of the object in the foreground currently observed by the user is small (for example, less than the depth threshold), the VR glasses can appropriately reduce the effect of blurring the object in the foreground. degree, thereby increasing the layering of the image observed by the user. In this example, the degree of blurring is determined by the depth threshold as an example. In other embodiments of the present application, multiple depth thresholds can be preset in the VR glasses, and each depth threshold corresponds to a virtual blurring. The degree of blurring can be adjusted to achieve the effect of multi-layer blurring according to the depth difference.

In combination with the foregoing description, when the user observes a nearby object for a long time, the ciliary muscle will be contracted for a long time. This will seriously affect the eyesight of the user. Such as lead to false myopia, or even true myopia.

For example, refer to FIG. 14 in combination with FIG. 3 . As illustrated in FIG. 3 , when a user observes a distant object with the human eye, the light entering the human eye is close to parallel light, and the light path of the light in the human eye is shown in (a) in FIG. 3 . When the user uses the human eye to observe nearby objects, the ciliary muscle can be contracted to adjust the diopter of the lens. The light path of the light in the human eye is shown in (b) in Figure 3 . When observing nearby objects for a long time, the ciliary muscle is in a state of contraction for a long time. At this time, if the human eye observes objects at other distances (such as distant objects), the ciliary muscle may not be able to relax in time, resulting in loss of the lens. The diopter is still maintained at a relatively high level. At this time, after the light enters the human eye, the light path is shown in Figure 15. It can be seen that after the light passes through the lens, it cannot be focused on the retina. In this way, a clear image cannot be formed on the retina. The user cannot see the corresponding object clearly. After a period of time, the ciliary muscle gradually relaxes, the diopter of the lens is adjusted accordingly, and the optical path of the light after passing through the lens gradually returns to the state shown in (a) in Figure 3, and the user can gradually see objects clearly. This state of being unable to see objects clearly in a short period of time is called pseudomyopia. If the user's human eyes are in the state of pseudomyopia for a long time, it may evolve into true myopia, that is, the ciliary muscle cannot adjust the diopter of the lens to return to the state shown in (a) in Figure 3, when parallel light enters the human eye It then focuses between the lens and the retina.

The virtual display device provided by the embodiment of the present application can flexibly and clearly display near objects and distant objects to the user without VAC because the zoom module can be controlled to adjust the depth of the virtual image plane. In some embodiments of the present application, the virtual display device can also provide the user with functions of relaxing and exercising the ciliary muscle.

Exemplarily, when the virtual display device detects that the user observes a nearby virtual object through the virtual display device for a long time (for example, longer than a preset duration), it may prompt the user to take a break, or switch to display a virtual object with a greater depth. virtual objects. For example, when the user observes the scene shown in (a) in FIG. 13 for longer than the preset duration, the VR glasses may prompt the user to use the eyes for too long. In different implementations, the VR glasses can prompt the user to use the eyes for too long by displaying prompt information (such as text information, etc.) on the display screen, or by vibrating, or by voice. Next, the VR glasses can display a virtual object with a greater depth under the control of the user, or spontaneously, such as displaying the scene shown in (b) in FIG. 13 . In some other embodiments, the virtual display device can also display the same object on the virtual image plane, and change the depth of the virtual image plane displayed by the object in the virtual environment through the focusing module, so that the user can view the virtual image at different depths. The effect of focusing on the surface. In this way, the virtual display device can guide the ciliary muscle of the human eye to relax, thereby avoiding pseudomyopia. In addition, by repeatedly guiding the ciliary muscle of the human eye to relax and contract, the ciliary muscle can also be exercised to avoid muscle stiffness, thereby improving resistance to vision problems.

As a possible implementation, take the virtual display device as VR glasses as an example. VR glasses can provide the display of the above-mentioned objects with different depths during the user's use, so as to protect the human eyes. Exemplarily, the VR glasses can display virtual objects of different depths to the user by controlling the zoom module to adjust the optical power of the optical display module in the VR glasses during the advertisement playing process or the waiting process of the video loading. In some embodiments, during the process of showing virtual objects of different depths to the user, the VR glasses can guide the user to control the human eyes to observe the virtual objects of different depths through voice or text prompts or other human-computer interaction methods, so as to realize adjustment The effect of the ciliary muscle in the human eye.

Combined with the above description, when the ciliary muscle of the human eye is in a contracted state for a long time, visual fatigue or even myopia may occur. For virtual display devices (such as VR glasses) that provide virtual display functions, the proportion of myopic users in the user group is also increasing.

Generally speaking, in normal life, myopic users can achieve vision correction by wearing glasses and other methods. For example, refer to FIG. 14 . For myopic users, the focal plane of the light entering the human eye may be located between the lens and the retina, so that the light (such as parallel light) cannot be clearly imaged on the retina after entering the human eye. Users can wear myopia glasses to adjust the optical path of light before it enters the human eye, so that the light can be clearly imaged on the retina after entering the human eye. For example, referring to FIG. 15 , take the myopia mirror as a concave lens as an example. Before the parallel light enters the human eye, it can be refracted by the concave lens, so that the light can converge on the retina after passing through the lens, so that the object emitting the light can be clearly imaged on the retina.

However, when a myopic user uses a virtual display device, it is obviously not convenient enough to wear glasses. Therefore, in order to enable a myopic user to wear a virtual display device, the picture displayed on the display screen can be clearly seen by the user (that is, the virtual image corresponding to the display screen can be clearly imaged on the retina of the human eye). In the implementation of some virtual display devices, the virtual display device will provide the user with the manual adjustment of the focal length of the optical system between the human eye and the display screen, such as the composition diagram of VR glasses shown in FIG. 4 . In this implementation, the VR glasses can provide the user with a control method to adjust the optical power of the eyepiece. For example, a mechanical rotation mechanism can be provided on the VR glasses, so that the user can achieve the effect of adjusting the optical power of the eyepiece when rotating the mechanical rotation mechanism. By adjusting the optical power of the eyepiece, the user can clearly see the virtual image corresponding to the display screen through the human eyes through manual control, that is, the virtual image of the display screen is clearly formed on the retina of the human eye. In this example, the adjusted eyepiece acts both to simulate myopia. Thus, the myopic user can use the virtual display function provided by the virtual display device without wearing myopia glasses.

It should be noted that the human eye has a certain automatic adjustment ability when observing objects. For example, when the image of the observed object in the human eye falls in front or behind the retina, the human eye will automatically control the ciliary muscle to adjust the concave-convex state of the lens, so as to adjust the imaging position as much as possible, so as to adjust the imaging position of the observed object in the human eye Adjusted to the retina.

Based on this, if the above-mentioned manual adjustment solution is adopted, even if the user adjusts the optical power of the eyepiece through manual control, the virtual image of the display screen can be clearly imaged on the retina. Due to the automatic adjustment mechanism of the human eye, the virtual image of the display screen in this state is clearly imaged on the retina because the ciliary muscle excessively squeezes the lens. That is to say, within the range where the ciliary muscle normally controls the lens, the current optical power of the eyepiece cannot guarantee that the virtual image on the display screen can be clearly imaged on the retina. Therefore, if the reality device provides the user with a virtual display function based on the focal power of the current eyepiece, the ciliary muscle of the user's eye will over-extrude the lens for a long time, resulting in eye discomfort, which may affect the user's eye health in severe cases .

The virtual display device provided by the embodiment of the present application (such as any wearable device or virtual display device in Fig. 6A-Fig. 10) can combine the automatic adjustment function of the zoom device to realize accurate To accurately measure the nearsightedness of the user's eyes. In this solution, the myopia of the human eye is measured and obtained, which can avoid the influence of the automatic adjustment ability of the human eye, so that according to the real situation of the human eye (such as myopia), the eye will be avoided when the user uses the virtual display function of the virtual display device. Discomfort, thereby improving the user experience.

In the following, in conjunction with the above-mentioned description of the wearable device or virtual display device provided by the embodiment of the application in Figure 6A-Figure 9, the scheme of providing virtual display according to the real situation of the user's human eyes provided by the embodiment of the application will be described in detail through the accompanying drawings .

For the convenience of description, the following assumes that the user is a myopic user and the virtual display device is VR glasses with the composition shown in FIG. 9 as an example.

In some embodiments of this example, before the VR glasses provide the virtual display function to the user, the image recognition ability of each of the user's eyes may be determined. In some embodiments, the ability of human eyes to recognize images can be identified by the power of human eyes.

It is understandable that at present, the degree of the human eye can be determined through an internationally accepted eye chart. For example, Figure 16 shows an eye chart. It can include the letters used to judge the recognition ability of the human eye, and the corresponding visual acuity of the letters of this size. For example, the visual acuity corresponding to the letters in the second row is 4.2° (logarithmic visual acuity), or 0.15° (international visual acuity). In this example, the process of determining the power of the human eye may also be referred to as a process of optometry. When using the eye chart to perform optometry on the user, the eye chart can be hung at a position 5m away from the user. By guiding the user to use the left eye and the right eye to determine the opening direction of the letters on the eye chart, the degree corresponding to the line where the letter is located is the minimum that can determine the opening direction, as the degree of the left eye or the right eye. For example, taking the use of the left eye to observe the eye chart as an example, the minimum ability to see the direction of the letter opening corresponding to 5.0 (1.0) on the eye chart indicates that the degree of the left eye can be 1.0. In this case, 5.0 (1.0) corresponds to a row of letter openings with an imaging width of about 5um on the retina, so the resolution of the left eye can reach a minimum width of 5um. At this time, it is generally believed that the vision of the left eye is normal, and there is no myopia.

It is understandable that if the human eye wants to be able to judge the opening direction of a letter, it needs to be able to distinguish the width corresponding to the opening direction based on the imaging of the letter on the retina. For example, if the human eye can clearly see the direction of the letter opening corresponding to a row of 5.0 (1.0), it means that the human eye can distinguish the width of the 5um image on the retina. The imaging size of letters on the retina of the human eye is obviously related to the distance between the letter and the human eye. This also makes it necessary to ensure the distance between the human eye and the letter when performing optometry through the eye chart.

Through this traditional optometry method, the respective diopters of both eyes of the user can be obtained, but the eye chart shown in Figure 16 needs to be used, and at the same time, there are relatively large requirements for the venue (for example, the venue needs to provide a 5m distance between the optometrist and the eye chart. distance). This is obviously not convenient enough. In addition, the traditional way of determining the recognition ability of the user's eyes to the image cannot be applied in the scene of virtual display.

For this, using the VR glasses provided in the embodiment of the present application, the following solution may be adopted to determine the image recognition ability of the user's human eyes.

Exemplarily, the VR glasses can sequentially show the user inspection images with openings of different sizes on the display screen (for example, the inspection images can include letters on the eye chart as shown in FIG. 16 ), combined with user feedback, determine The user's ability to recognize images.

In some embodiments of the present application, the VR glasses can determine the user's ability to recognize the image through the visual score of the image recognized by human eyes.

Wherein, the visual score may refer to a minimum angle of an image that can be distinguished by human eyes when observing the image. For example, referring to FIG. 17 , it is a schematic diagram of a visual score. Openings of letters shown as 1701 at the minimum that the user can distinguish. Then, the angle between the opening of the letter and the human eye (the angle of sight as shown in the figure) can be used as the visual score of the human eye.

Since the visual score is an indication of an angle, the distance between the human eye and the inspection image can be freely adjusted without restriction.

After obtaining the visual score of the human eye, the VR glasses can determine the visual acuity of the human eye based on the corresponding relationship between the visual score and the human eye example, combined with the visual score of the human eye.

As a possible implementation, the correspondence between the vision score and the human vision can be preset in the VR eye, or can be obtained from the cloud when the VR glasses need to be used. In some embodiments, Table 1 shows a correspondence between vision scores and human visual acuity.

Table 1

对数视力表logarithmic vision chart	国际视力表International Eye Chart	视分值Score
4.04.0	0.10.1	10′10′
4.14.1	0.120.12	7.947′7.947′
4.24.2	0.150.15	6.312′6.312′
4.34.3	0.200.20	5.013′5.013′
4.44.4	0.250.25	3.982′3.982′
4.54.5	0.30.3	3.163′3.163′
4.64.6	0.40.4	2.512′2.512′
4.74.7	0.50.5	1.996′1.996′
4.84.8	0.60.6	1.585′1.585′
4.94.9	0.80.8	1.259′1.259′
5.05.0	1.01.0	1′1'

As shown in Table 1, VR glasses can determine the corresponding visual acuity to be 4.0 (0.1) when the visual score that the human eye can distinguish is 10'. Similarly, VR glasses can determine the corresponding visual acuity to be 4.1 (0.12) when the visual score that the human eye can distinguish is 7.947'. VR glasses can determine the corresponding visual acuity to be 4.9 (0.8) when the visual score that the human eye can distinguish is 1.259'. and so on.

Please refer to FIG. 18 , which is a schematic flowchart of a virtual display method provided by an embodiment of the present application. Based on this process, the VR glasses can obtain the corresponding image recognition capabilities of the user's eyes. Among them, take the image recognition ability as an example through vision identification. As illustrated, the program can include:

S1801. When the user uses the VR glasses, display the first detected image corresponding to the first visual score to the first human eye of the user.

Wherein, the first human eye may be any one of the user's left eye or right eye. The first visual score may be any visual score in the corresponding relationship shown in Table 1. In order to accurately measure the vision of the human eye, in some embodiments of the present application, when the VR glasses start to detect the user's vision, the first visual score may be the visual score with the largest corresponding viewing angle, for example, as shown in Table 1 10' shown.

In this example, the VR glasses can display the first detection image on the display screen corresponding to the first human eye (such as the first display screen), so that the first human eye can observe the to the first detected image. For example, refer to FIG. 19 . Take the first human eye as the left eye, and the component in the VR eye that provides the virtual display function to the left eye as the first display component as an example. As shown in FIG. 19 , the VR glasses can control the display screen of the first display component to display the first detection image. In this way, the user can see the first detection image displayed at the position of the virtual image plane as shown in the figure through the left eye of the first detection image.

The user can observe the first detection image with human eyes (such as the first human eye), and judge whether the first detection image corresponding to the first visual score can be seen clearly. For example, the user can judge the opening direction of the letters or graphics included in the first detection image, and input the judgment result to the VR glasses, so that the VR glasses can determine whether the first human eye can clearly see the current first image with the first visual score. Detect images.

Exemplarily, it is taken that the detection image displayed by the VR glasses includes a C-shaped letter as an example. As shown in FIG. 20, in this S1801, the opening size of the C-shaped letter may correspond to the first visual score.

That is to say, the VR glasses need to ensure that the opening size of the C-shaped letter seen by the user corresponds to the first visual score of the first human eye.

In the embodiment of the present application, the VR glasses can determine, according to the current field of view and the number of pixels displayed for the graphics in the field of view, when displaying the C-shaped letter with the first visual score, the The size that the C-shaped letter needs to have.

It can be understood that, in the process of displaying images on the display screen, the display is performed in units of pixels. VR glasses can determine the display size of the first detection image on the first display screen by Pixel Per Degree (PPD), so as to ensure that the opening of the first detection image seen by the first human eye is the first visual score .

For example, in conjunction with Figure 21. Take the angle A as an example where the viewing angle of the left eye when observing the first display screen is taken as an example. As shown in FIG. 21 , the number of pixels participating in providing image display to the user's left eye on the display screen may be N pixels. N pixels can be seen. In this way, the PPD in this scenario can be obtained according to the following formula (1).

In this way, according to the PPD, the VR glasses can combine the distance between the display screen and the user's human eyes (such as the first human eye) to determine that when the C-shaped letter corresponding to the first visual score is to be displayed, the opening of the C-shaped letter The corresponding number of pixels. Further, the pixel corresponding to the C-shaped letter is determined. Accordingly, the VR glasses can ensure that the opening size of the C-shaped letter seen by the user corresponds to the first visual score of the first human eye. For example, the display effect shown in FIG. 20 is realized. It can be understood that, through the above solution, the VR glasses can determine the size of the first detection image to be displayed according to the visual score. Compared with directly calculating the size of the first detection image, the number of pixels required to display the first detection image can be more accurately determined, thereby presenting a more accurate first detection image with the first visual score to the user.

It should be noted that, in some embodiments of the present application, the first detection image may include a letter or a figure for detection of human eyesight. For example, refer to FIG. 22 . (a) in FIG. 22 shows an example in which a letter for detecting eyesight is included in the first detection image. As shown in (a) of FIG. 22 , the letter in the first detection image may have an opening (such as a notch of letter C). The VR glasses can determine the display size of the C-shaped letter on the first display screen according to the foregoing description, so as to display the C-shaped letter whose opening corresponds to the first visual score to the user.

In some other embodiments of the present application, the first detection image may include a plurality of letters or graphics used to detect human eyesight. Exemplarily, continue to take the example that the first detection image includes a C-shaped letter. The VR glasses can display multiple C-shaped letters with openings corresponding to the first visual score to the user at one time on the first display screen. Wherein, the opening directions of two adjacent C-shaped letters may be different. In this way, the user can sequentially recognize the opening directions of the multiple C-shaped letters, so that the VR glasses can more accurately judge the user's vision according to the recognition results input by the user. Exemplarily, (b) in FIG. 22 shows a display example in which the first detection image includes a plurality of C-shaped letters. Based on the first detection image, the user can sequentially judge the opening direction of the corresponding letter from left to right through the first human eye, so that the VR glasses can judge more accurately based on this whether the user can see the visual score (as shown in No. A detection image corresponding to a visual score).

In addition, in order to avoid mutual interference between the detection process of the first human eye and the detection process of the second human eye, in some embodiments of the present application, the VR glasses can display the first detection image on the first display screen, and the other A black screen is displayed on one display (such as the second display), or the second display is turned off. After the detection of the first human eye is completed according to the scheme provided in the embodiment of the present application, the VR glasses can control the second display screen to display according to a similar scheme, so as to realize the vision of another human eye (such as the second human eye) detection.

S1802. Receive first identification feedback from the user.

The user's first human eyes can see the first detection image corresponding to the first visual score with the opening through the first display component.

In this example, the user may input the first recognition feedback to the VR glasses according to the opening direction of the first detected image seen. In some embodiments, the first identification feedback may be used to indicate the opening direction of the first detection image judged by the user.

Exemplarily, take a C-shaped letter included in the first detection image as an example. After seeing the C-shaped letter, the user can input the opening direction of the C-shaped letter into the VR glasses. That is to say, in this example, the first recognition feedback may be the opening direction of the C-shaped letter.

Combined with Figure 23. Take the user inputting the opening direction through the remote control as an example. When the user recognizes that the opening direction of the C-shaped letter is upward through the first human eyes. In some embodiments, the user may input the first recognition feedback indicating the upward direction of the opening by touching a button corresponding to "up" on the remote control (button 2301 shown in (a) in FIG. 23 ). In some other embodiments, the user may input an upward sliding operation on the touch screen of the remote control (such as the screen 2302 shown in (b) in FIG. 23 ), so as to input the first recognition feedback indicating that the direction of the opening is upward.

It should be noted that, in the above example, the user inputs the first identification feedback through the remote controller as an example for illustration. In some other embodiments of the present application, the user may also implement the input of the first recognition feedback through voice commands and/or gesture commands. The embodiment of the present application does not limit the specific manner in which the user inputs the first identification feedback.

In this way, the VR glasses can receive the recognition of the opening direction of the current first detection image by the user's first human eyes.

In some embodiments of the present application, in order to enable the user to correctly input the first recognition feedback, the VR glasses may guide the user to recognize the first detection image and input the first recognition feedback before receiving the user's first recognition feedback. For example, the VR glasses can guide the user to recognize the first detection image and input the first recognition feedback through voice prompts, text prompts in the displayed virtual scene, or other methods (such as vibration, etc.).

S1803. Determine whether the user can recognize the first detection image according to the first recognition feedback.

In the embodiment of the present application, the VR glasses can determine whether the subsequent action is performed according to whether the first recognition feedback is consistent with the actual opening direction of the first detection image. Exemplarily, when the first recognition feedback is inconsistent with the actual opening direction of the first detection image, it indicates that the first human eye cannot clearly see the first display image currently having the first visual score. Then the VR glasses can continue to execute the following S1804. Correspondingly, when the first recognition feedback is consistent with the actual opening direction of the first detection image, the VR glasses indicate that the first human eye can clearly see the first display image currently having the first visual score. Then the VR glasses can continue to perform the following S1805.

It should be noted that, in some embodiments of the present application, when the VR glasses determine that the first identification feedback is inconsistent with the actual opening direction of the first detection image, the detection image with the first visual score is displayed to the user again, and the detection The opening direction of the image may be different from the first detection image. In order to facilitate the user to judge again the opening direction of the detection image of the first visual score. After repeating the above actions for a preset number of times (such as three times), if the recognition feedback input by the user is still inconsistent with the opening direction of the detected image, then the VR glasses can make it clear that the user cannot clearly see the image corresponding to the first visual score, and then Execute S1804. In this way, the accuracy of the VR glasses in judging the user's vision can be improved.

S1804. Determine the visual acuity of the first human eye according to the first visual acuity score.

It can be understood that if the first human eye cannot identify the opening direction of the first detection image, then the VR glasses can determine the visual acuity of the first human eye according to the first visual score.

For example, combine Table 1. In some embodiments, the VR glasses can determine the visual acuity corresponding to the first visual score according to the correspondence in Table 1. The vision may then be that of the first human eye. For example, as shown in Table 1, when the first visual score is 7.947', the VR glasses can determine that the visual acuity of the first human eye is 4.1 (0.12). For another example, when the first visual score is 1.259', the VR glasses can determine that the visual acuity of the first human eye is 4.9 (0.8).

In this way, the VR glasses can realize the determination of the image recognition ability of the first human eye.

S1805. Display the second detection image corresponding to the second visual score.

Wherein, the size of the second detection image corresponding to the second visual score may be smaller than the size of the first detection image corresponding to the first visual score. So as to achieve the purpose of gradient reduction to detect user examples.

Exemplarily, referring to Table 1, the first visual score is 2.512' as an example. In S1803, when the VR glasses determine that the opening direction of the letter corresponding to the user's first recognition feedback is consistent with the actual opening direction of the first detected image. The visual score of the second detected image displayed by the VR glasses on the first display screen may be 1.996′. That is to say, the second detection image is more stringent in the detection of eyesight than the first detection image, so it can be further determined whether the user can clearly see the smaller image when the first detection image can be clearly seen (such as the second detection image).

In this step, the method for the VR glasses to determine the second detection image is similar to the method for determining the first detection image. For example, VR can calculate and determine the size of the second detection image according to the PPD and the second visual score, and then display the second detection image with the second visual score on the first display screen according to the changed size. The specific implementation thereof is similar and will not be repeated here.

S1806. Receive second identification feedback from the user.

Wherein, the second recognition feedback may be an instruction input by the user into the VR glasses for indicating the opening direction of the second detection image after observing the second detection image with the first human eyes. For the specific execution process and implementation manner, please refer to the first recognition feedback.

S1807. In the case that the second identification feedback is different from the actual opening direction of the second detection image, determine the visual acuity of the first human eye according to the second visual score.

The specific implementation of this step is similar to the above S1804, and will not be repeated here.

It should be noted that, in the example shown in FIG. 18 , the user can recognize the opening direction of the first detection image but cannot recognize the opening direction of the second detection image as an example. That is to say, the VR glasses can provide the user with a detection image corresponding to a gradient drop of the visual score for the user to judge, and determine the visual acuity of the user's human eye based on the corresponding maximum visual score when the opening direction of the detection image cannot be clearly identified.

As another possible implementation manner, continue to refer to FIG. 18 . After performing step 1806, the VR glasses may perform S1801 in a loop according to the dotted path in the case that the second identification feedback is different from the actual opening direction of the second detection image. That is, continue to display other detection images to the user. It can be understood that, in this example, the opening size corresponding to the detection image displayed by the VR glasses to the user again may be smaller than the opening size corresponding to the second detection image, thereby further increasing the difficulty for the user to recognize the detection image until the detection image cannot be obtained by the human eye. up to the maximum visual score identified.

Therefore, the VR glasses can determine the degree of the human eye by determining the maximum resolvable visual score of the user's human eye, and then clarify the human eye's ability to recognize images.

In the specific implementation of the embodiment of the present application, the VR glasses can control the number of detected images displayed on the display screen, so that while obtaining the recognition ability of the user's human eyes for the image, the interest is improved, and then the user experience is improved. Effect.

Exemplarily, in combination with the solution shown in FIG. 18 , refer to FIG. 24 . The first display screen may display an interface as shown at 2410 in FIG. 24 when performing S1801, that is, displaying the first detection image. It can be seen that, in this example, multiple (eg, 5) detection images (eg, C-shaped letters) corresponding to the first visual score may be displayed on the interface. Different C-shaped letters may have different opening directions, so that users can make judgments from left to right in turn. For example, in the case that the user cannot accurately identify the opening direction of the leftmost C-shaped letter in the interface 2410, the VR glasses may display an interface as shown in 2420 on the first display screen. In order to facilitate the user to continue to judge the opening direction of the C-shaped letter (such as the letters in the area indicated by 2402) with the first visual score, and thus enable the VR glasses to more accurately judge whether the first human eye can recognize the first visual score . It can be seen that on the interface 2420, the letters 2401 that the user has recognized can be hidden or not displayed. A visual elimination effect is thereby achieved. In this way, the user can continue to judge the opening direction of the leftmost C-shaped letter on the interface. Correspondingly, when the opening direction indicated by the first recognition feedback input by the user is consistent with the actual opening direction of the C-shaped letter, the VR glasses may display an interface as shown at 2430 on the first display screen. In order to provide users with a smaller visual score for judgment. It can be seen that on the interface 2430, the position of the letter 2401 that the user has judged is empty, that is, the letter at this position is not displayed or the letter at this position is hidden, thereby achieving the visual effect of elimination. After that, the user can continue to judge the opening direction of the leftmost letter displayed in the area shown in 2403 through the first human eye, so that the VR glasses can determine whether the user can recognize images with smaller visual scores.

Therefore, through the specific implementation method provided by the example in Figure 24, the VR glasses can hide or not display the image of the position of the letter that has been judged in the process of displaying the detection image, so as to achieve the effect of visual elimination , so as to improve the interest of the whole detection process, thereby enhancing the user experience. At the same time, it is more convenient to locate unjudged letters and improve the efficiency of detection.

In addition, in the above example, the description is made by showing only one or more detection images with one visual score to the user at the same time on one interface (such as the interface of the first display screen). In other embodiments of the present application, the VR glasses can also display multiple detection images with different visual scores to the first human eye on the first display screen, and guide them through images, voices, or vibrations, so that The user can determine the opening direction of the image corresponding to each visual score according to a certain process. In this way, the VR glasses can also obtain the minimum visual score that can be recognized by the user.

For example, refer to Figure 25. The VR glasses can determine the size of letters corresponding to different visual scores according to the method for determining the letter size corresponding to each visual score in the scheme shown in FIG. 18 above. And on the first display screen, a complete image or a partial image including a plurality of detected images with different visual scores as shown in FIG. 25 is displayed to the user at one time. The VR glasses can prompt the user to judge the image of a specific row in the image, so as to guide the user to complete the vision test. For example, as shown in Figure 25, the VR glasses can display a prompt symbol (such as an arrow) as shown in 2501 near the line where the image that needs to be judged by the user is located, and guide the user to the line displayed on the line through voice or text prompts, etc. Detect the image for recognition, and input the corresponding recognition feedback. As an example, the VR glasses can guide the user through voice prompts: "Please judge the opening direction of the first letter on the left among the letters in the row pointed by the arrow". Thus, the user can judge the opening direction of the first letter on the left side of the line indicated by the arrow. After the user inputs the recognition feedback of the opening direction of the letter, in the case that the recognition feedback indicates that the user can correctly recognize the row of letters, the VR glasses can move the prompt symbol down one row (such as display 2502), and hide 2501 at the same time Or do not display 2501. Further, the user is guided to judge the opening direction of the letter with a smaller visual score. And so on, until the VR glasses determine the minimum visual score that the user can recognize.

It can be understood that, based on the above descriptions of FIGS. 18-25 , in the above embodiments, the VR glasses can provide detection images of different sizes corresponding to different visual scores on the same virtual image plane. That is to say, in this visual score-based detection scheme, the VR glasses can detect the user's vision without adjusting the position of the virtual image plane and the position of the imaging plane after the light enters the human eye. And because in the visual score-based display scheme, the display size of the detected image corresponds to the position of the current virtual image plane (eg, according to PDD), it is not necessary to control the distance between the virtual image plane and the human eye to be 5m.

In other embodiments of the present application, the VR glasses can also adjust the virtual image surface and the human eye to a specific position of 5m through the zoom module. The VR glasses can display a virtual image of the same size as the international eye chart on the first display screen, and guide the user to identify the smallest recognizable letter opening in the virtual scene, and then determine the user's vision condition accordingly. This process is equivalent to simulating the optometry process in the real environment in the virtual scene, so that the purpose of determining the user's vision can be realized without requiring the actual site.

After completing the detection of one human eye, the VR glasses can use a similar scheme to detect the other human eye, so as to obtain the image recognition ability of the user's eyes.

It should be noted that the solutions provided in the above-mentioned FIGS. 18-25 can realize the detection of the user's binocular vision (that is, complete the optometry). In this way, the degree of myopia of both eyes of the user is obtained. Obviously, this solution enables users to achieve optometry through VR glasses, without the need to go to a professional optometry institution to know the degree of myopia of both eyes.

In addition, in the above example, the user can realize the detection of the degree of myopia through the guidance of VR glasses. In other embodiments of the present application, as shown in FIG. 26 , the VR eyes can also interact with the cloud (such as a server, etc.), so that the cloud can provide users with more professional and scientific optometry guidance through the VR glasses. Among them, the cloud can send instructions to the VR glasses to control the VR glasses to perform optometry for the user. In some implementations, the instructions on the cloud can be uploaded based on other terminals that interact with the cloud. The instructions can be input by a professional optometrist or other professionals, or can be based on the cloud or other terminals. The execution plan is determined by itself.

In this embodiment of the application, the VR glasses can adjust the content displayed to the user after acquiring the user's ability to recognize images (such as the degree of myopia of the user's eyes), so that the nearsighted user can also You can experience clear virtual display functions through VR glasses.

Exemplarily, the VR glasses can obtain the degree of myopia of both eyes of the user through any of the schemes in Figure 18 to Figure 28 and any of the schemes in the foregoing description.

In the following, a specific scheme for adaptively adjusting the display content of the VR glasses according to the degree of myopia of the user will be described with reference to the accompanying drawings.

As a possible implementation, Table 2 below shows a schematic representation of the corresponding relationship between the degree of myopia of the human eye and the optical power of the VR glasses. The corresponding relationship may be preset in the VR glasses, or may be obtained from the cloud when the VR glasses need to be used.

Table 2

视力vision	近视度数degree of myopia	光焦度optical power
4.04.0	650650	-6.5D-6.5D
4.14.1	600600	-6D-6D
4.24.2	550550	-5.5D-5.5D
4.34.3	500500	-5D-5D
4.44.4	450450	-4.5D-4.5D
4.54.5	400400	-4D-4D
4.64.6	325325	-3.25D-3.25D
4.74.7	250250	-2.5D-2.5D
4.84.8	125125	-1.25D-1.25D
4.94.9	5050	-0.5D-0.5D

According to Table 2, the VR glasses can determine the corresponding optical power according to the user's human eyesight. For example, when the visual acuity of the user's first human eye is 4.6, the VR glasses can determine the focal power to be -3.25D. For another example, when the visual acuity of the user's first human eye is 4.8, the VR glasses can determine that the optical power is -1.25D. And so on. It should be noted that the correspondence shown in Table 2 is only an example, and does not list all possible correspondences between optical powers and degrees of myopia. In other implementations of the embodiments of the present application, more or fewer corresponding relationships may be included in Table 2.

In this example, the VR glasses can determine an adjustment mechanism corresponding to the optical power according to a preset policy. For example, take the realization of the function of the zoom module through the adjustment of the mechanical structure as an example. The corresponding relationship between different optical powers and the relative positions of each lens in the zoom module can be preset in the VR glasses. Thus, after determining the optical power, the VR glasses can control the zoom module to adjust the relative positions of the lenses to the positions corresponding to the optical power, thereby realizing the corresponding adjustment of the optical power.

In this way, the VR glasses can adjust the distance between the virtual image surface of the display screen and the human eye to an appropriate range by adjusting the optical power, so that the virtual image surface can form a clear image in the user's eyes. It should be noted that since the VR glasses control the zoom module to adjust the optical power before the virtual image plane is imaged in the human eye, the distance from the virtual image plane to the image plane imaged in the human eye (for example, called the virtual image distance) is the same as the user's current Myopia matching. As a result, the image on the virtual image plane can be clearly imaged on the retina of the human eye, so that the user can clearly see the image displayed by the VR glasses with the naked eye.

In order to enable those skilled in the art to understand the solutions provided in the embodiments of the present application more clearly. In the following, with reference to the accompanying drawings, it is taken as an example that the VR glasses obtain the degree of myopia of the user according to the scheme shown in FIG. 18 , and adjust the content displayed to the user according to the degree of myopia. It should be noted that, according to one of the eyes of the user, the VR glasses can execute the following scheme to realize the adjustment of the displayed image of the eye. The execution strategy of the other human eye is similar.

It can be understood that when the human eye has no myopia or the degree of myopia is very small, the parallel light entering the human eye can be smoothly converged on the retina of the human eye. When the virtual image plane is at the farthest point (ie 0D), the light incident on the human eye is closest to the parallel light. Therefore, in the embodiment of the present application, based on the virtual image distance corresponding to 0D, the detection images of different visual scores can be shown to the user, so as to adjust to the position of the virtual image distance matching the human eye according to the user's recognition situation. In this way, the effect of providing the human eye with a virtual image matching the degree of myopia of the human eye is achieved. For example, please refer to FIG. 27 , which is a schematic flowchart of a virtual display solution provided by the embodiment of the present application. As shown, the program can include:

S2701. Control the zoom module to adjust the virtual image plane to the first position. For example, the first position may be the 0D position.

Wherein, the 0D position may be the furthest distance that can be achieved by controlling the zoom module of the VR glasses to adjust the virtual image distance. In some embodiments, the processor may control the focal length of the zoom module, so that the virtual image plane is adjusted to the 0D position.

It is understandable that in the subsequent adjustment process, the VR glasses may need to control the zoom module to reduce the virtual image distance so that the virtual image distance can match the degree of myopia of the user.

S2702. Display the detection image 1 corresponding to the first visual score. For example, the first visual score may be 10'. Then the detection image 1 may be a detection image with a size corresponding to 10'.

In this example, the determination and display of the size of the detection image 1 corresponding to the visual score of 10' can refer to the determination and display of the first detection image in the solution shown in FIG. 18 above, which will not be repeated here.

It should be noted that the detection image 1 corresponding to 10' may be a detection image with a relatively large opening size in this solution, so for the user's human eyes, the recognition difficulty is the lowest. Through the following solution, the VR glasses can gradually reduce the opening size of the detection image, so as to achieve the effect of increasing the recognition difficulty step by step, and then enable the VR glasses to more accurately determine the degree of myopia of the user.

It should be noted that, in the embodiment of the present application, the execution of S2701 and S2702 does not have a strict sequence. For example, in some embodiments, S2701 may be performed before S2702, that is, the VR glasses may first adjust the position of the virtual image plane through the processor, and then display the detected image 1 on the virtual image plane. In some other embodiments, S2701 and S2702 can also be executed at the same time, that is, the VR glasses can display the detection image 1 at the position of the corresponding virtual image plane during the process of adjusting the position of the virtual image plane. In other embodiments, S2701 can also be performed after S2702, that is, the VR glasses can determine the size of the detected image 1 according to the first visual score, and display it on the current virtual image plane, and then adjust the virtual image plane through the processor position, so as to achieve the effect of displaying the detection image 1 at the 0D position.

S2703A. Determine whether the user can recognize the detection image 1.

When the user can recognize the opening direction of the detected image 1, S2704 is executed. When the user cannot recognize the opening direction of the detection image 1, execute S2703B.

It can be understood that, in conjunction with S1802 in FIG. 18 , the user can input recognition feedback corresponding to the detection image 1 to the VR glasses, so that the VR glasses know whether the user can recognize the opening direction of the detection image 1 .

S2703B. Control the zoom module to adjust the virtual image plane to the second position. For example, the second position may be the position corresponding to -7D.

When the user cannot identify the opening direction of the detected image 1, it indicates that the user has a high degree of myopia. The VR glasses can control the zoom module and adjust the virtual image distance so that the virtual image surface is at the position corresponding to -7D, thereby making the virtual image of the display screen The surface can be imaged in a relatively close position in the human eye as much as possible, so as to match the degree of myopia of the user, so that the user can clearly see the image on the virtual image surface through the naked eye of the human eye.

S2704. Display the detection image 2 corresponding to the second visual score. For example, the second visual score may be 7.947'. Then the detection image 2 can be a detection image with a size corresponding to 7.947′.

After the VR glasses determine that the user can recognize the detection image with a larger visual score (such as detection image 1), the visual score can be appropriately reduced and the detection image 2 can be displayed to determine the degree of myopia of the user.

S2705A. Determine whether the user can recognize the detection image 2.

When the user can recognize the opening direction of the detected image 2, S2706 is executed. When the user cannot recognize the opening direction of the detected image 2, S2705B is executed.

S2705B. Control the zoom module to adjust the virtual image plane to a third position. For example, the second position may be the position corresponding to -6D.

S2706. Display the detection image 3 corresponding to the third visual score. For example, the third look score may be 5'. Then the detection image 3 may be a detection image with a size corresponding to 5'.

S2707A. Determine whether the user can recognize the detection image 3.

When the user can recognize the opening direction of the detected image 3, S2708 is executed. When the user cannot recognize the opening direction of the detected image 3, S2707B is executed.

S2707B. Control the zoom module to adjust the virtual image plane to a fourth position. For example, the fourth position may be the position corresponding to -5D.

S2708. Display the detection image 4 corresponding to the fourth visual score. For example, the fourth look score may be 3.2'. Then the detection image 4 may be a detection image with a size corresponding to 3.2'.

S2709A. Determine whether the user can recognize the detection image 4.

When the user can recognize the opening direction of the detected image 4, S2710 is executed. When the user cannot recognize the opening direction of the detected image 4, S2709B is executed.

S2709B. Control the zoom module to adjust the virtual image plane to a fifth position. For example, the fifth position may be a position corresponding to -4D.

S2710. Display the detection image 5 corresponding to the fifth visual score. For example, the fifth look score may be 2'. Then the detection image 5 can be a detection image with a size corresponding to 2′.

S2711A. Determine whether the user can recognize the detection image 5 .

When the user can recognize the opening direction of the detected image 5, S2712 is executed. When the user cannot recognize the opening direction of the detected image 5, S2711B is executed.

S2711B. Control the zoom module to adjust the virtual image plane to the sixth position. For example, the sixth position may be a position corresponding to -2.5D.

S2712. Display the detection image 6 corresponding to the sixth visual score. For example, the sixth visual score may be 1.5'. Then the detection image 6 may be a detection image with a size corresponding to 1.5'.

S2713A. Determine whether the user can recognize the detection image 6 .

When the user can recognize the opening direction of the detected image 6, S2714 is executed. When the user cannot recognize the opening direction of the detected image 5, S2712B is executed.

S2713B. Control the zoom module to adjust the virtual image plane to the seventh position. For example, the seventh position may be the position corresponding to -1D.

S2714. Control the zoom module to adjust the virtual image plane to the first position. In combination with S2701, the first position may be a position corresponding to 0D.

After the above process, if the user can still recognize the opening direction of the detection image 6 with the minimum visual score, then it indicates that the user's human eyesight is good, and the system can be displayed normally without adjusting the optical power of the system. The image on the virtual image plane is clearly imaged on the retina of the human eye. Therefore, the VR glasses can execute S2714 to keep the virtual image noodle at the 0D position, that is, adjust to the position where the virtual image distance is the farthest, so as to provide a clear virtual display for human eyes.

It should be noted that, in the above description, the gradient reduction of the size of the detection image displayed to the user is taken as an example for description. In some other embodiments of the present application, the VR glasses can also display the detected images to the user in gradually increasing sizes. In some other embodiments of the present application, the VR glasses can also use the middle size as the starting point, and use the dichotomy method to show the user detection images that are larger than the starting middle size and smaller than the starting middle size, thereby determining The user's eyesight condition, and adjust the position of the virtual image plane accordingly.

It can be understood that, in the solution shown in FIG. 27 , the optical power positions of the virtual image planes corresponding to each visual score can be obtained through the above Table 2. In different implementations, the gradients and quantities of the corresponding relationships in Table 2 are different, and the solutions shown in FIG. 27 implemented accordingly may also be different.

In this way, the VR glasses can adaptively adjust the optical power according to the degree of myopia of the user's human eyes, so that even if the human eye is in a state of myopia, the virtual display function provided by the VR glasses can be used without wearing glasses.

Combined with the above description about the automatic adjustment function of the human eye, the human eye can recognize the opening direction of the image as much as possible through self-adjustment when recognizing images with different visual scores. In some embodiments of the present application, in order to avoid inaccurate visual acuity detection caused by excessive adjustment of the lens by using the ciliary muscle when the user recognizes and detects the opening direction of the image with human eyes, in some embodiments of the present application , the red-green balance scheme can be introduced on the basis of the above-mentioned scheme.

For example, refer to FIG. 28 . When different light rays pass through the human eye (such as the lens of the human eye) to form an image, the refraction paths of the light paths after passing through the lens are different due to the different wavelengths of the light rays. As shown in Fig. 28, when white light can be concentrated on the retina, the converging point of green light can be located between the lens and the retina, and the converging point of red light can be located behind the retina.

In this example, after the VR glasses determine the degree of myopia of the user, the virtual image distance can be further fine-tuned in combination with the red-green balance scheme, so that the virtual image distance can be displayed to the human eye at the position of the virtual image distance that matches the real myopia degree of the user's human eye. .

For example, referring to FIG. 27 , it is taken as an example that after the execution of S2703A, the VR glasses determine that the user cannot recognize the opening direction of the detection image 1 . As shown in Figure 27, the VR glasses can execute S2703B, that is, control the zoom module to adjust the virtual image plane to the -7D position. Combined with the red-green balance scheme, VR glasses can continue to control the zoom module after adjusting the virtual image surface to the position corresponding to -7D, and fine-tune around the corresponding position of -7D (such as fine-tuning between -6.5D and -7D) to find red The position where light and green light are imaged in the same sharpness on the retina. The VR glasses can use the position of the image plane as a fine-tuning result, so that in the process of displaying the virtual image to the user's current human eyes, the adjustment result can be used to show the user a clear virtual image.

Similarly, when executing S2705B, the VR glasses can control the zoom module to adjust the virtual image plane to the position of -6D, and fine-tune it between -6D and -6.5D to obtain the imaging clarity of red light and green light on the retina Consistent position as a result of tuning. Or, when executing S2707B, the VR glasses can control the zoom module to adjust the virtual image plane to the position of -5D, and then fine-tune it between -5D and -6D to obtain images with the same definition of red light and green light on the retina. position as a result of tuning. Or, when executing S2709B, the VR glasses can control the zoom module to adjust the virtual image plane to the position of -4D, and fine-tune it between -4D and -5D to obtain images with the same definition of red light and green light on the retina. position as a result of tuning. Or, when executing S2711B, the VR glasses can control the zoom module to adjust the virtual image plane to the position of -2.5D, and fine-tune it between -2.5D~-4D to obtain the imaging clarity of red light and green light on the retina Consistent position as a result of tuning. Or, when executing S2713B, the VR glasses can control the zoom module to adjust the virtual image plane to the position of -1D, and fine-tune it between -1D to -2.5D to obtain the same definition of red light and green light on the retina. The position of is used as the tuning result.

In this way, through the above coarse adjustment as shown in FIG. 27 , combined with the fine adjustment of the red-green balance scheme, a result matching the real degree of myopia of the user's human eyes can be obtained. The display based on this can prevent the user's human eyes from being in a state of over-adjustment, thereby avoiding the occurrence of problems such as visual fatigue caused by this.

It can be understood that, in the above description, the recognition ability of the user's human eyes for graphics is identified by the degree of myopia of the human eyes as an example for illustration. It is understandable that for some users, in addition to myopia, the human eye may also have problems with astigmatism. Different degrees of astigmatism can correspond to different axial directions of astigmatism.

Exemplarily, astigmatism will be described with reference to FIG. 29 and FIG. 30 . As shown in Figure 29, for users with astigmatism, at different angles, after the parallel light enters the human eye and is refracted by the lens, the converging position in the human eye may be different. For example, as shown in (a) of FIG. 29 , after the parallel rays in the XOZ plane enter the human eye, they can converge on the retina. At the same time, after the parallel rays in the YOZ plane are incident on the human eye, their converging point may not be on the retina (as shown in (b) in Figure 29, it is located between the lens and the retina). In this way, it will result in the situation that the images at some angles cannot be clearly imaged by human eyes. For example, in conjunction with Figure 30. As shown in (a) in Figure 30, when there is no astigmatism in the human eye, the incident light rays at various angles can be uniformly imaged in the human eye, that is, the imaging conditions of the light rays at various angles in the human eye are approximately of. For human eyes with astigmatism, there may be some angles of light that cannot be clearly imaged in the human eye. For example, referring to (b) in FIG. 30 , in the current coordinate system, light rays in the vertical direction can be imaged better, and the closer the angle of the light rays is to the horizontal, the poorer the imaging definition is.

However, in the actual scene or the virtual scene provided by VR glasses, the light entering the human eye generally does not come from only one angle, which will cause blur when the user observes the object and affect the user experience. In order to enable the user with astigmatism to use the virtual display function provided by the VR glasses with naked eyes, an embodiment of the present application further provides a solution that can determine the degree of astigmatism of the user with astigmatism. Furthermore, the image displayed to the user is corrected according to the degree of astigmatism, so that the user with astigmatism can also use the virtual display function of the VR glasses with naked eyes.

As an example, this solution may be implemented through the virtual display device provided in the embodiment of the present application. For example, take the virtual display device as VR glasses with the composition shown in FIG. 10 as an example. In this example, the zoom module in the VR glasses can produce cylindrical power and has a rotation function. When the zoom module rotates, the zoom module can be rotated around the center of the corresponding line of sight (such as around the center of the corresponding lens barrel), so that the direction of the cylindrical power of the optical system composed of the zoom module and the eyepiece coincides with the user's astigmatism axis. In some embodiments, if the human eye also has a degree of myopia, the optical power of the zoom module may match the optical power of the human eye with the degree of myopia. For specific matching and adjustment methods, reference may be made to the solutions provided in the above embodiments, which will not be repeated here.

Therefore, when the virtual display device provides a virtual display to the user, the light incident on the user's eye can be the light that coincides with the axis of astigmatism, so that the light can be clearly imaged in the human eye, avoiding the distortion caused by astigmatism. The image is not clear. This also allows users with astigmatism to use the virtual display solution provided by VR glasses with naked eyes.

Similar to the aforementioned process for myopia detection, in the embodiment of the present application, the virtual display device can adjust the axial position of the incident light according to the degree of astigmatism of the human eye so as to avoid unclear imaging caused by astigmatism. In some other embodiments of the present application, combined with the foregoing description of the remote optometry solution, the virtual display device provided in the embodiments of the present application can also be used to measure the degree of astigmatism.

Exemplarily, when the VR glasses measure the degree of astigmatism of the human eye, a corresponding measurement can be performed in combination with the degree of myopia of the current human eye. For example, take the degree of myopia as M (the unit is m ⁻¹ ) as an example. The VR glasses can control the zoom module so that the virtual image plane is 5m away from the human eye after myopia correction.

Exemplarily, the VR glasses can determine the 5m position after myopia correction according to the following formula (2).

Among them, M is the degree of myopia. V is the distance from the human eye in the virtual scene.

The VR glasses can display the astigmatism detection chart shown in FIG. 31 on the virtual image plane. VR glasses can guide users to input angles that they cannot see clearly. For example, the angle may be an angle corresponding to any one or more numbers in 1-12 as shown in FIG. 31 . In some embodiments, the user can input the relevant information of the unclear angle through voice input or text input of the remote control. According to the angle information input by the user, the VR glasses can determine the axial direction of astigmatism corresponding to the digital information input by the user. For example, if the number input by the user is 1, the VR glasses can determine that the axial direction of astigmatism is 60°. Similarly, if the number input by the user is 5, the VR glasses can determine that the axial direction of astigmatism is 120°. and so on.

In this example, the VR glasses can control the zoom mechanism to rotate according to the direction of the user's astigmatism axis, and then adjust the cylindrical power of the zoom mechanism until the user can see clearly the lines on the astigmatism detection chart shown in Figure 31 at the same time. . At this time, the cylindrical power of the zoom mechanism is the astigmatism power of the user. In this way, astigmatism detection of the user's eyes can be realized.

In some embodiments, combined with the implementation of the aforementioned remote optometry solution, the process of detecting astigmatism can also be implemented remotely. For example, a doctor performing remote detection can control VR glasses to guide users to perform astigmatism detection through the cloud.

It should be noted that, in the above descriptions of the solutions for the correction of myopia and the correction of astigmatism, an eye of the user is taken as an example for description. In the scene where the user's eyes need to be corrected, the VR glasses can respectively perform corresponding steps on the two human eyes according to the scheme in the above description, so that the image recognition capabilities (such as myopia and/or astigmatism) of the two eyes can be realized. ) to make corresponding adjustments, so as to ensure the effect of the naked-eye virtual display experience of the user.

It is understandable that the human eyes of different users have different recognition capabilities for images. In some embodiments of the present application, the VR glasses can generate a corresponding relationship between the acquired myopia and/or astigmatism degree of the user and the user information, so that when the user uses the VR glasses next time, it can avoid repeating the same user detection.

Exemplarily, in some embodiments, the VR glasses can store the corresponding relationship between the user's iris information and the degree of myopia and/or astigmatism locally or in the cloud, so that when the user uses the VR glasses next time, it can be based on the user's The iris information determines the degree of myopia and/or astigmatism of the user, and then controls the zoom module to adjust the optical power, and/or controls the zoom module to adjust the axial position of the incident light. In this way, a virtual display based on naked eyes is provided to the user.

Please refer to FIG. 32 , which is another virtual display method provided by the embodiment of the present application. As shown, the program can include:

S3201. When the user uses the VR glasses, extract the iris features of the user.

Exemplarily, the VR glasses can take pictures of the user's eyes through its eye-tracking module. The iris feature of the human eye is extracted by analyzing the captured image of the human eye.

In the implementation of this application, the VR glasses can only extract the iris feature of one human eye of the user. In this way, the confirmation of user identity can be realized under the premise of saving computing power.

S3202. Determine whether there is virtual display information matching the user according to the acquired iris features.

Wherein, the virtual display information may include degrees of myopia and/or degrees of astigmatism.

In some embodiments, in the VR glasses, the corresponding relationship between iris features and degrees of myopia and/or degrees of astigmatism of different users may be stored. In this way, after acquiring the iris feature of the current user, the VR glasses can search whether there is a matching item corresponding to the iris feature from the above correspondence relationship by way of query. If it exists, it means that there is virtual display information matching the user. Conversely, if no matching item corresponding to the iris feature is found, it means that there is no matching virtual display information for the user.

In some other embodiments, the corresponding relationship between iris features of different users and degrees of myopia and/or degrees of astigmatism may be stored in the cloud. The VR glasses can send the acquired iris feature of the current user to the cloud, so that the cloud can find out whether there is a matching item corresponding to the iris feature through query. If the cloud determines that there is a matching item corresponding to the iris feature, the matching item corresponding to the iris feature can be issued to the VR glasses with a prescription for ametropia corresponding to the above correspondence. Wherein, in some embodiments, the prescription for refractive error may include myopia and/or astigmatism. The VR glasses can confirm that there is virtual display information matching the user when receiving the degree of myopia and/or astigmatism, and the virtual display information matching the user can be the degree of myopia and/or astigmatism received from the cloud degree. Correspondingly, when it is determined that there is no matching item corresponding to the iris feature, the cloud may send a not found notification to the VR glasses, so that the VR glasses can determine that there is no matching item corresponding to the iris feature. Alternatively, the cloud may not send any information to the VR glasses when it is determined that there is no matching item corresponding to the iris feature. In this way, the VR glasses can determine that there is no matching item corresponding to the iris feature without receiving the degree of myopia and/or the degree of astigmatism within a preset time.

It should be noted that, in some other embodiments of the present application, the degree of myopia and/or the degree of astigmatism included in the virtual display information may be identified by corresponding optical power and/or axial position.

In addition, since the zoom module changes the optical power, the magnification of the entire display system will change. That is, the size of the VR content that the user sees changes. In addition, in the process of changing the optical power of the zoom device, the distortion parameters of the display system will also be changed. Therefore, in some embodiments of the present application, the virtual display information may also include distortion correction parameters and/or magnification parameters corresponding to the optical power in the process of adjusting the myopia degree.

If there is virtual display information matching the user, the VR glasses may perform the following S3203.

If there is no virtual display information matched by the user, the VR glasses may perform the following S3204.

S3203. Call the virtual display information, and provide a virtual display to the user according to the virtual display information.

When it is determined that there is a matching item, the VR glasses can call various parameters included in the virtual display information, and provide a virtual display to the human eyes of the current user according to these parameters.

S3204. Determine the virtual display information of the user.

S3205. According to the user's virtual display information, provide the user with a virtual display.

Wherein, the process of determining the virtual display information of the user may refer to the process of determining the detection process of the user's myopia degree and the detection process of the user's astigmatism degree in the above example. Thus, the virtual display information of the user can be determined. The process of providing virtual display to the user according to the virtual display information can refer to the solution of adjusting the optical power according to the degree of myopia and the solution of adjusting the axial position of the incident light according to the degree of astigmatism of the user in the above example. I won't repeat them here.

In some embodiments, since the iris characteristics of the current user do not find a match in the stored iris characteristics, it indicates that the user may be a new user. In this case, in order to ensure the information security of the iris characteristics of the user, before performing S3204, the VR glasses may obtain authorization from the user. In this way, the VR glasses can establish a corresponding relationship between the iris features of the current user and the corresponding virtual display information. In some implementations, the VR glasses can store the corresponding relationship, so that repeated detection can be avoided when the user uses the VR glasses next time.

It should be noted that, in the scheme shown in FIG. 32 , different users are identified by their iris characteristics as an example. In some other embodiments of the present application, the virtual display device may also identify different users in other ways. For example, the virtual display device may identify different users through user accounts of different users, and/or biometric information (such as distance between eyes, fingerprints, voiceprints, etc.) of different users. Then, similar to the above-mentioned correspondence between iris features and virtual display information, the virtual display device may store corresponding correspondence between feature information for identifying different users and virtual display information, so as to realize the above-mentioned function corresponding to FIG. 32 .

The foregoing mainly introduces the solutions provided by the embodiments of the present application from the perspective of electronic devices. In order to realize the above functions, it includes corresponding hardware structures and/or software modules for performing various functions. Those skilled in the art should easily realize that the present application can be implemented in the form of hardware or a combination of hardware and computer software in combination with the units and algorithm steps of each example described in the embodiments disclosed herein. Whether a certain function is executed by hardware or computer software drives hardware depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

The embodiments of the present application may divide the involved devices into functional modules according to the above method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. It should be noted that the division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.

Exemplarily, FIG. 33 shows a schematic composition diagram of an electronic device 3300 . The electronic device 3300 may correspond to any virtual display device in the foregoing embodiments. As shown in FIG. 33 , the electronic device 3300 may include: a processor 3301 and a memory 3302. The memory 3302 is used to store computer-executable instructions. Exemplarily, in some embodiments, when the processor 3301 executes the instructions stored in the memory 3302, the electronic device 3300 may be made to execute the virtual display method shown in any one of the above embodiments.

It should be noted that all relevant content of the steps involved in the above method embodiments can be referred to the function description of the corresponding function module, and will not be repeated here.

FIG. 34 shows a schematic composition diagram of a chip system 3400 . The chip system 3400 may be set in any virtual display device in the foregoing embodiments. The chip system 3400 may include: a processor 3401 and a communication interface 3402, configured to support related devices to implement the functions involved in the foregoing embodiments. In a possible design, the chip system further includes a memory for storing necessary program instructions and data of the virtual display device. The system-on-a-chip may consist of chips, or may include chips and other discrete devices. It should be noted that, in some implementation manners of the present application, the communication interface 3402 may also be called an interface circuit.

The functions or actions or operations or steps in the above-mentioned embodiments may be fully or partially implemented by software, hardware, firmware or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or may include one or more data storage devices such as servers and data centers that can be integrated with the medium. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)), etc.

Although the application has been described in conjunction with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made thereto without departing from the scope of the embodiments. Accordingly, the specification and drawings are merely illustrative of the application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of this application. Apparently, those skilled in the art can make various changes and modifications to this application without departing from the scope of this embodiment. In this way, if these modifications and variations of the application fall within the scope of the claims of the application and their equivalent technologies, the application also intends to include these modifications and variations.

Claims

A virtual display device, characterized in that the virtual display device is used to provide users with a display function of a virtual three-dimensional environment; the virtual display device includes:

A processor, and a first optical display module; the first optical display module is used to display images to the first human eyes under the control of the processor, and the first human eyes are both eyes of the user any of

The first optical display module is used to realize the following functions under the control of the processor:

Displaying a first object, the depth of convergence of the first object is a first depth of convergence, and the depth of an imaging plane of the first optical display module is a first depth;

Displaying a second object, the depth of convergence of the second object is a second depth of convergence, and the depth of the imaging surface of the first optical display module is a second depth;

Wherein, when the first depth of convergence is different from the second depth of convergence, the optical display module adjusts the difference between the first depth and the second depth.
The virtual display device according to claim 1, wherein:

When the first depth of convergence is greater than the second depth of convergence, the first depth is greater than the second depth; when the first depth of convergence is smaller than the second depth of convergence, the first depth is less than the second depth.
The virtual display device according to claim 1 or 2, characterized in that, the virtual display device further comprises: a first eye-tracking module, the first eye-tracking module is used in the processor Under control, performing eye movement tracking on the first human eye;

The processor is used to control the first eye-tracking module to perform eye-tracking on the fixation point of the first human eye,

When the depths of convergence of the gaze points of the first human eyes are different, the first optical display module is configured to adjust the depth of the imaging surface to be different.
The virtual display device according to any one of claims 1-3, wherein when the diopters of the users are different, the first optical display module is configured to adjust the depth of the imaging surface to be different.
The virtual display device according to any one of claims 1-4, wherein the virtual display device further comprises: a second optical display module, the second optical display module is used for processing displaying an image to a second eye, the second eye being the one of the user's eyes different from the first eye, under the control of the user;

The first optical display module is used to realize the following functions under the control of the processor:

displaying the first object, displaying that the depth of the imaging plane of the first object is a third depth, the depth of the first object in the virtual three-dimensional environment is a second depth, and the first depth and the The third depth is similar or the same.
The virtual display device according to any one of claims 1-5, wherein the first optical module comprises: a first zoom module; the optical power of the first zoom module is adjustable;

The first zoom module is configured to adjust the optical power of the first zoom module to a first optical power when displaying the first object under the control of the processor, so that the first optical power has the a first optical module of one optical power capable of imaging at said first depth; or,

The first zoom module is configured to adjust the optical power of the first zoom module to the second optical power when displaying the second object under the control of the processor, so that the first The two-power first optical module is capable of imaging at the second depth.
The virtual display device according to any one of claims 1-6, characterized in that,

The first optical display module is further configured to display a third object at a first virtual position and a second virtual position in the virtual three-dimensional environment under the control of the processor;

The first virtual position and the second virtual position have different distances from the user's eyes in the three-dimensional environment.
The virtual display device according to claim 7, wherein:

The virtual display device displays the third object at the first virtual position and the second virtual position respectively when the currently displayed scene is a preset scene;

Wherein, the preset scene includes at least one of the following scenes:

The ad playing scene shows the resource loading scene.
The virtual display device according to any one of claims 1-8, characterized in that,

The first optical display module is also used to display a first detection image to the first human eye under the control of the processor; when displaying the first detection image, it is used to display the first detection image. detecting that the number of pixels of the opening of the image is the first number;

The processor is further configured to receive a first recognition feedback, the first recognition feedback is an instruction input by the user when using the first human eyes to observe the first detection image, and the first recognition feedback uses to indicate whether the user can identify the opening direction of the first detection image.
The virtual display device according to claim 9, wherein:

When the first optical display module uses the first number of pixels to display the opening of the first detection image to the user, the visual score of the first human eye observing the opening of the first detection image is the second a score;

In the case where the first recognition feedback indicates that the user cannot recognize the opening direction of the first detection image,

The processor is configured to determine the degree of myopia of the first human eye as the degree of myopia corresponding to the first visual score.
The virtual display device according to claim 9 or 10, characterized in that,

In the case where the first recognition feedback indicates that the user can recognize the opening direction of the first detection image,

The first optical display module is also used to display a second detection image to the first human eye under the control of the processor; when displaying the second detection image, it is used to display the second detection image. The number of pixels of the opening of the detection image is a second number; the second number is smaller than the first number;

The processor is further configured to receive second recognition feedback, the second recognition feedback is an instruction input by the user when using the first human eyes to observe the second detection image, and the second recognition feedback uses to indicate whether the user can identify the opening direction of the second detection image.
The virtual display device according to claim 11, wherein:

When the first optical display module uses the second number of pixels to display the opening of the second detection image to the user, the visual score of the first human eye observing the opening of the second detection image is the first binocular score;

In the case where the second recognition feedback indicates that the user cannot recognize the opening direction of the second detection image,

The processor is used to determine the degree of myopia of the first human eye as the degree of myopia corresponding to the second visual score.
The virtual display device according to any one of claims 9-12, characterized in that,

The first optical display module is used to adjust the optical power of the first optical display module under the control of the processor before displaying the first detection image to the first human eye. The initial optical power enables the first optical display module to image at the furthest distance.
The virtual display device according to any one of claims 9-13, characterized in that,

In the case where the opening of the third inspection image displayed by the first optical display module to the first human eye is a third number of pixels, the received recognition feedback indicates that the user cannot recognize the fourth inspection image When the opening direction of

The optical power of the first optical display module to display the virtual image to the first human eye is the third optical power; the third optical power corresponds to the third visual fraction, and the third visual fraction A value is a view score value corresponding to the third number of pixels.
The virtual display device according to any one of claims 1-14, characterized in that,

The first optical display module includes a rotation mechanism for rotating the first optical display module in a direction perpendicular to the optical axis under the control of the processor;

When the first optical display module presents the first image to the user, the focal power direction of the first optical display module coincides with the astigmatism axis of the first human eye.
The virtual display device according to claim 15, wherein:

The processor is also used to determine the rotation of the rotating mechanism when the focal power direction of the first optical display module coincides with the astigmatism axis of the first human eye, and determine the degree of astigmatism.
The virtual display device according to any one of claims 1-16, characterized in that,

The processor is further configured to obtain user characteristics of the current user before controlling the first optical display module to display the first image,

The processor is specifically configured to control the first optical display module to display a first image corresponding to the user characteristics of the current user;

Wherein, when displaying the first image, the optical power of the first optical display module matches the degree of myopia and/or the degree of astigmatism of the first human eye of the current user, and the first degree of astigmatism of the current user The degree of myopia and/or the degree of astigmatism of the human eye is indicated by the virtual display information corresponding to the user characteristics of the current user.
The virtual display device according to claim 17, wherein:

The user features include any one of the following features: the fingerprint information of the current user; the iris feature of the current user; the account information of the current user; the identification of the current user, which is different for different users.
The virtual display device according to claim 17 or 18, wherein the virtual display device stores correspondences between different user characteristics and corresponding virtual display information;

The processor is configured to search for a matching entry from the corresponding relationship according to the user characteristics of the current user, and if there is a matching entry, determine that the virtual display information corresponding to the current user is the The virtual display information stored in the matching entry; the virtual display information includes the degree of myopia and/or the degree of astigmatism of the corresponding user.
The virtual display device according to claim 19, wherein:

When there is no matching item corresponding to the user feature of the current user in the corresponding relationship,

The first optical display module is further configured to display a first image under the control of the processor,

The optical power when displaying the first image is the optical power matching the myopia degree and/or astigmatism degree of the first human eye of the current user, and the myopia degree and/or astigmatism degree of the first human eye The degree is determined automatically by the processor, or at the direction of the user, or manually entered by the user.
The virtual display device according to claim 20, wherein:

The processor is further configured to store the correspondence between the degree of myopia and/or the degree of astigmatism of the first human eye and the user characteristics of the current user.
A virtual display method, characterized in that the virtual display method is applied to the virtual display device according to any one of claims 1-21, and the virtual display method is used to provide users with a display function of a virtual three-dimensional environment ; the method comprising:

The first optical display module displays the first object, the depth of convergence of the first object is the first depth of convergence, and the imaging plane depth of the first optical display module is the first depth;

The first optical display module displays a second object, the depth of convergence of the second object is a second depth of convergence, and the imaging plane depth of the first optical display module is a second depth;

Wherein, when the first depth of convergence is different from the second depth of convergence, the optical display module adjusts the difference between the first depth and the second depth.
The method of claim 22, wherein,

When the first depth of convergence is greater than the second depth of convergence, the first depth is greater than the second depth; when the first depth of convergence is smaller than the second depth of convergence, the first depth is less than the second depth.
The method according to claim 22 or 23, further comprising:

The processor controls the first eye-tracking module of the first optical display module to perform eye-tracking on the fixation point of the first human eye,

When the depths of convergence of the gaze points of the first human eyes are different, the first optical display module is configured to adjust the depth of the imaging surface to be different.
The method according to any one of claims 22-24, wherein the first optical module includes: a first zoom module; the optical power of the first zoom module is adjustable; and the method further include:

The processor controls the first zoom module, and when displaying the first object, adjusts the optical power of the first zoom module to the first optical power, so that the a first optical module capable of imaging at said first depth; or,

The processor controls the first zoom module, and when displaying the second object, adjusts the optical power of the first zoom module to the second optical power, so that the The first optical module is capable of imaging at the second depth.
The method according to any one of claims 22-25, further comprising:

The processor controls the first optical display module to display a third object at a first virtual position and a second virtual position in the virtual three-dimensional environment;

The first virtual position and the second virtual position have different distances to the user's eyes in the three-dimensional environment;

Wherein, when the currently displayed scene is a preset scene, the first optical display module displays the second object at the first virtual position and the second virtual position respectively;

Wherein, the preset scene includes at least one of the following scenes:

The ad playing scene shows the resource loading scene.
The method according to any one of claims 22-26, further comprising:

The processor controls the first optical display module to display a first detection image to the first human eye; when displaying the first detection image, the pixels used to display the opening of the first detection image the quantity is the first quantity;

The processor receives a first recognition feedback, the first recognition feedback is an instruction input by the user when using the first human eyes to observe the first detection image, and the first recognition feedback is used to indicate the Whether the user can identify the opening direction of the first detection image;

When the first optical display module uses the first number of pixels to display the opening of the first detection image to the user, the visual score of the first human eye observing the opening of the first detection image is the second a score;

In the case where the first recognition feedback indicates that the user cannot recognize the opening direction of the first detected image,

The processor determines that the degree of myopia of the first human eye is the degree of myopia corresponding to the first visual score.
The method of claim 27, wherein,

In the case where the first recognition feedback indicates that the user can recognize the opening direction of the first detection image, the method further includes:

The processor controls the first optical display module to display a second detection image to the first human eyes; when displaying the second detection image, the pixels used to display the opening of the second detection image the quantity is a second quantity; the second quantity is less than the first quantity;

The processor receives second recognition feedback, the second recognition feedback is an instruction input by the user when observing the second detection image with the first human eyes, the second recognition feedback is used to indicate the Whether the user can identify the opening direction of the second detection image;

When the first optical display module uses the second number of pixels to display the opening of the second detection image to the user, the visual score of the first human eye observing the opening of the second detection image is the first binocular score;

In the case where the second recognition feedback indicates that the user cannot recognize the opening direction of the second detection image,

The processor determines that the degree of myopia of the first human eye is the degree of myopia corresponding to the second visual score.
A method according to any one of claims 27 or 28, wherein,

In the case where the opening of the third inspection image displayed by the first optical display module to the first human eye is a third number of pixels, the received recognition feedback indicates that the user cannot recognize the fourth inspection image When the opening direction of

The processor controls the optical power of the first optical display module to display the virtual image to the first human eye to be a third optical power; the third optical power corresponds to a third visual score, so The third visual score is a visual score corresponding to the third number of pixels.
The method according to any one of claims 22-29, further comprising:

Before the processor controls the first optical display module to display the first image, acquire the user characteristics of the current user,

The processor controlling the first optical display module to display the first image includes:

The processor controls the first optical display module to display a first image corresponding to the user characteristics of the current user;

Wherein, when displaying the first image, the optical power of the first optical display module matches the degree of myopia and/or the degree of astigmatism of the first human eye of the current user, and the first degree of astigmatism of the current user The degree of myopia and/or astigmatism of the human eye is indicated by the virtual display information corresponding to the user characteristics of the current user;

The user features include any one of the following features: the fingerprint information of the current user; the iris feature of the current user; the account information of the current user; the identification of the current user, which is different for different users.
The method according to claim 30, wherein the method stores the correspondence between different user characteristics and corresponding virtual display information; the method further comprises:

The processor is configured to search for a matching entry from the corresponding relationship according to the user characteristics of the current user, and if there is a matching entry, determine that the virtual display information corresponding to the current user is the The virtual display information stored in the matching entry; the virtual display information includes the degree of myopia and/or the degree of astigmatism of the corresponding user.
The method according to claim 31, further comprising:

When there is no matching item corresponding to the user feature of the current user in the corresponding relationship,

The first optical display module displays a first image under the control of the processor,

The optical power when displaying the first image is the optical power matching the myopia degree and/or astigmatism degree of the first human eye of the current user, and the myopia degree and/or astigmatism degree of the first human eye The degree is determined automatically by the processor, or at the direction of the user, or manually entered by the user.
The method of claim 32, further comprising:

The processor stores the corresponding relationship between the degree of myopia and/or the degree of astigmatism of the first human eye and the user characteristics of the current user.
A virtual display device, characterized in that the virtual display device is used to provide users with a display function of a virtual three-dimensional environment;

The virtual display device includes one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories store computer instructions;

When the one or more processors execute the computer instructions, the electronic device is made to execute the virtual display method according to any one of claims 22-33.
A computer-readable storage medium, characterized in that the computer-readable storage medium includes computer instructions, and when the computer instructions are executed, the virtual display method according to any one of claims 22-33 is executed.
A virtual display device, characterized in that the virtual display device is used to provide users with a display function of a virtual three-dimensional environment; the virtual display device includes:

a processor, and a first optical display module;

The first optical display module is used to display a first image to the first human eyes under the control of the processor, the first image includes a first object, and the first object is the virtual three-dimensional objects in the environment; the first human eye is either of the user's eyes;

The first optical display module displays the first object, the depth of the imaging surface of the first object is displayed as a first depth, and the depth of the first object in the virtual three-dimensional environment is a second depth, The first depth is similar to or the same as the second depth.
The virtual display device according to claim 36, wherein when the first optical module is displaying the first object, the optical power of the first optical module is adjusted so that the first object is displayed. The virtual image plane depth of the object is the first depth.