TW201326894A - Methods and systems for creating free space reflective optical surfaces - Google Patents

Abstract

Computer-based methods and associated computer systems are disclosed for designing free space reflective optical surfaces (13) for use in head-mounted displays (HMDs). The reflective optical surface (13) produces a virtual image of a display surface (11) for viewing by a user's eye (15). The method includes using one or more computers to: (i) represent the display surface (11) by display objects (25); (ii) represent the free space reflective optical surface (13) by surface elements (23); and (iii) iteratively calculate spatial locations, normals, and radii of curvature for the surface elements (23) which will cause a virtual image of each display object (25) to be displayed to a nominal user's eye (15) in a desired direction of gaze of the eye (15).

Description

Method and system for creating free space reflecting optical surface

The present invention is directed to methods and systems for creating (i.e., designing or designing and producing) free-space reflective optical surfaces for use in head mounted displays. More generally, the present invention relates to methods and systems for creating a free-space optical surface for displaying images from a light-emitting display device that is held in close proximity to the user's eye.

A reflective optical surface is referred to herein as a "free space" surface because the local spatial position of the surface, the local surface curvature, and the local surface orientation are not related to a particular substrate, such as the xy plane, but during the design of the surface. It is determined using the basic optical principles applicable in three-dimensional space (for example, Fermat and Hero minimum time principle).

A head mounted display such as a head mounted display or a glasses mounted display (herein abbreviated as "HMD") is a display device worn on the head of a person having one eye or (more generally) located at the user's head. One or more small display devices near the two eyes. 1 shows a basic element of a type of HMD that includes a display 11, a reflective optical surface 13, and an eye 15 having a center of rotation 17. As shown in this figure, light 19 from display 11 is reflected by surface 13 and enters the user's eye 15.

Some HMDs only display analog (computer generated) images (as opposed to real world images) and are therefore often referred to as "virtual reality" or submerged HMDs. Other HMDs overlay (combine) analog images on non-simulated real world images. The combination of non-analog images and analog images allows the HMD user to view the world via, for example, goggles or eyepieces, and additional information related to the task to be performed is superimposed on the goggles or eyepieces to the user's front field of view (FOV) )on. This overlay is sometimes referred to as "Augmented Reality" or "Mixed Reality."

A partially reflective/partially transmissive optical surface ("beam splitter") can be used to combine a non-simulated real-world view with a simulated image, in which case the reflectivity of the surface is used to simulate the image as a virtual image (in In optical terms, the transmittance of the surface is used to allow the user to directly view the real world (referred to as the "see-through system"). The real world view and the analog image can also be electronically combined (referred to as a "video see-through system") by accepting video from the real world view of the camera and electronically mixing it with the analog image using a combiner. The combined image can then be presented to the user as a virtual image (in an optical sense) by means of a reflective optical surface (in this case it does not need to have a transmission property).

As can be seen from the foregoing, reflective optical surfaces can be used in HMDs, which provide the user with: (i) a combination of analog and non-analog real-world images, (ii) a combination of analog and real-world video images, or Iii) Pure analog imagery. (The last case is often referred to as the "immersion" system.) In each of these cases, the reflective optical surface produces a virtual image (in an optical sense) that is viewed by the user. Historically, such reflective optical surfaces have been part of an optical system that has an exit pupil that substantially limits not only the dynamic field of view available to the user but also the static field of view. In particular, in order to see the image produced by the optical system, the user needs to align his eye with the exit pupil of the optical system and keep it in such alignment, and even then, the image visible to the user will not be covered. The entire fully static field of view of the user, i.e., the prior optical system used in the HMD that has used the reflective optical surface, has been part of the pupil forming system and has therefore been limited by the exit pupil.

The reason why the system has been so limited is the fact that the human vision is significantly larger. Thus, the static field of view of the human eye (including both the visual and peripheral vision of the eye) is approximately ~150° in the horizontal direction and approximately ~130° in the vertical direction. (For the purposes of the present invention, 150 degrees will be used as a straight forward static field of view of the nominal human eye.) A well-corrected optical system with an exit pupil capable of accommodating this large static field of view is sparse, and when present, It is expensive and bulky.

In addition, since the eye can rotate around its center of rotation, that is, the human brain can aim at the visual field of the human eye + peripheral vision in different directions by changing the gaze direction of the eye, so the operational field of view (dynamic field of view) of the human eye is even Bigger. For nominal eyes, the vertical range of motion is approximately ~40° up and down ~60°, and the horizontal range of motion is approximately ±~50° from straight front. For an exit pupil of the size produced by an optical system of the type previously used in HMD, even small rotations of the eye will substantially reduce the overlap between the static field of view of the eye and the exit pupil, and Rotating will cause the image to completely disappear. Although theoretically possible, the exit pupil that will move in sync with the user's eyes is impractical and extremely expensive.

In view of these attributes of the human eye, there are three fields of view associated with an optical system that allows the user to view images produced by the image display system in the same manner as would be to view the natural world. The smallest of the three fields of view is the field of view defined by the user's ability to rotate their eyes and thus their ability to scan the field in the outside world. The maximum rotation is approximately ±50° from the straight front, so this field of view (visual field of view) is approximately 100°. The middle of the three fields of view is a straight front static view and includes both the user's visual and peripheral vision. As discussed above, this field of view (the visual field + peripheral static field of view) is approximately 150°. The largest of the three fields of view is the field of view defined by the user's ability to rotate their eyes and thus their ability to view their peripheral vision over the outside world. The maximum field of view (visual litter + peripheral dynamic field of view) is approximately 200° based on a maximum rotation of approximately ±50° and a visual field of about 150° + peripheral static field of view. The increased scale of the field of view from at least 100 degrees to at least 150 degrees and then to at least 200 degrees provides benefits to the user (in terms of their ability to visually and naturally view images produced by the image display system).

Therefore, there is a need for a reflective optical surface for use in HMDs having improved compatibility with the human eye's field of view (both static and dynamic). There is also a need for a reflective optical surface that can be used to provide a virtual image (in an optical sense) in the HMD to the human eye without the limitations imposed by the external exit pupil. The present invention provides methods and systems for creating such surfaces.

definition

In the rest of the invention and in the scope of the patent application, the phrase "virtual image" is used in its optical sense, that is, the virtual image is perceived as being from a particular image, and in fact, is being perceived. Not from one side.

Throughout the present invention, the following phrase/term should have the following meaning/scope:

(1) The phrase "reflective optical surface" (also referred to herein as "reflective surface") shall include only reflective surfaces and surfaces that are both reflective and transmissive. In either case, the reflectivity can be only partial, that is, a portion of the incident light can be transmitted through the surface. Likewise, when the surface is both reflective and transmissive, the reflectivity and/or transmittance can be partial.

(2) The phrase "field of view" and its abbreviation FOV refer to the "visual" field of view in the image (eye) space, as opposed to the "real" field of view in the object (ie, display) space.

According to one aspect, a computer-based method and associated computer system for designing a free-space reflective optical surface (13) for use in a head mounted display is disclosed, the free-space reflective optical surface producing a display A virtual image of the surface (11) for viewing by a user's eye (15) at a pre-selected spatial location, the method comprising performing the following steps using one or more computers:

(a) the display surface (11) is represented by a plurality of display objects (25);

(b) the free-space reflective optical surface (13) is represented by a plurality of surface elements (23), each surface element (23) being characterized by: (i) a nominal use relative to the display surface (11) a person's eye (15) and a spatial position of the pre-selected spatial location of the virtual image, (ii) a normal, and (iii) a radius of curvature;

(c) during the use of the head mounted display, a virtual image of each of the display objects (25) at the preselected spatial location will be displayed in the direction of the eye (15) of a nominal user such that Display object (25) associated with at least one surface element (23), each surface element (23) being associated with one and only one display object (25);

(d) For each surface element (23), perform the following steps:

(i) defining an initial spatial position of the component;

(ii) calculating the initial spatial position of the component, the position of the display object (25) associated with the component, and the position of a center of rotation (17) of a nominal user's eye (15). An initial direction of one of the normals such that light from the display object (25) reflected from the element will pass through the center of rotation;

(iii) calculating an initial radius of curvature of the component such that the virtual image of the display object (25) is at the pre-selected spatial location;

(e) for each surface element (23), by adjusting the spatial positions of the elements in an iterative manner until an error function satisfies a predetermined criterion, calculating a final spatial position of the element, the normal of the element A final direction and a final radius of curvature of one of the component and a set of surrounding components.

According to another aspect, a computer-based method and associated computer system for designing a free-space reflective optical surface (13) for use in a head mounted display is disclosed, the free-space reflective optical surface producing a Displaying a virtual image of one of the surfaces (11) for viewing by a user's eye (15), the method comprising performing the following steps using one or more computers:

(a) the display surface (11) is represented by a plurality of display objects (25);

(b) representing the free-space reflective optical surface (13) by a plurality of surface elements (23);

(c) calculating, in an iterative manner, at least one spatial position and at least one normal for each of the surface elements (23), the at least one spatial position and the at least one normal to cause each display object (25) One of the virtual images is displayed to the eye (15) in a desired gaze direction for the eye (15) of one of the nominal user objects.

In various embodiments, a reflective optical surface designed in accordance with the computer-based methods disclosed herein can provide a user with a full view fove dynamic field of view, a full view fossa + peripheral static field of view, or a full view fossa + peripheral dynamic field of view.

The reference numerals used in the above summary of the present invention are to be considered as illustrative and not intended to be The scope of the invention. Rather, the foregoing general description and the following detailed description of the invention,

The additional features and advantages of the invention are set forth in the description which follows, and in which Come to know. The accompanying drawings are included to provide a further understanding of the invention It is to be understood that the various features of the invention disclosed in this specification and in the drawings may be used in any or all combinations.

In order to focus humans on objects that are approximately 25 centimeters away, it is generally necessary to adjust the optical properties of the light being emitted from the object. The way in which the optical properties of the light are adjusted such that it can be focused by the human eye is to collimate the light to produce a light having a parallel beam and a flat wave front. The wavefront of the light exiting the point source has a spherical shape, and its curvature can be defined by an attribute called vergence (or V). The vergence is measured by diopter [D], where the amount of vergence is determined by the distance from the source (in meters). Therefore, if the observer is away from the point source by "s" [m], the vergence is:

Equation (1)

That is, the vergence is equal to the reciprocal of the distance "s" from the point source and has a unit [D] for diopter. The vergence is shown as negative because it is a standard representation indicating that the light is diverging.

In general, humans cannot adapt their eyes to focus on something close to 25 cm. This is sometimes called "near point." Therefore, the vergence Va at the adaptation limit is:

Equation (2)

Therefore, if the vergence divergence is greater than -4 D, such as when the non-optically corrected object is approximately 25 cm, the eye cannot focus on the object.

One of the objectives of certain embodiments of the free-space reflective optical surface disclosed herein is that the vergence of all of the light entering the eye after reflection from the surface has _a negative vergence closer to zero than V _a . Since the eye cannot focus on light having a vergence greater than zero, another object of the embodiments is that the distance to the virtual image of the object cannot exceed infinity, so the vergence must be greater than zero, or

This essentially means

V 0[ D ].

In order for the virtual image of the object to appear at a point farther than 25 cm, the degree of convergence to be achieved needs to be set to the reciprocal of the desired distance. For example, for a distance of 20 [m], the vergence of light waves entering the eye is:

V = -1/20 = -0.05 [D].

And, for a distance of 50 [m], the vergence is:

V = -1 / 50 = -0.02 [D].

If the display is 25 cm away from the eyes, the spread of the display is:

And the eye can focus on the display. If in Figure 1, the display is on the front of the user and the light is reflected off the reflective surface (mirror), the total distance s from the pixel on the display to the eye is:

s = s _P + s _R '

Wherein s _P and s _R indicate the lengths of the line segments P and R in Fig. 1, respectively. It will be assumed that the display does not perform any collimation of the illumination of its emission or any change in the optical properties of the illumination it emits. As discussed earlier, if there is no optical correction, the distance from the eye to the display must be greater than or equal to 25 cm.

Assuming that a virtual image with a seemingly 50 [m] away from the center of the eye is required, the spread of light entering the eye must be:

Equation (3)

In order to achieve this vergence, the reflective surface must condense the light emitted thereto as it directs light into the eye. The amount of focus P that the reflective surface must provide will depend on the distance from the display to the surface and to a lesser extent depending on the distance from the eye to the surface. Figure 2 shows the relevant parameters, where:

P = the power of the concave reflector [D]

W = the distance to the virtual image [m]

l = the distance to the object [m], in this case, the object is the display object

s _P = l = distance from the display as shown by line P in Figure 1 [m]

(Note that sp is negative due to optical conventions and reflection of the light path by the mirror)

s _R = distance from the reflector to the eye as shown by the line R in Figure 1 [m]

l' = distance from the surface of the reflector to the image 21 (in this case, the virtual image) [m]

As can be seen from Figure 2:

l' = W - s _R

Equation (4)

The vergence associated with distance l' is:

Equation (5)

From the Gaussian mirror equation,

L = L' - P

Equation (6)

Where L is the vergence associated with the distance l from the display to the reflector, the distance l being:

Equation (7)

For completeness, the horizontal magnification of the image

Equation (8)

=1+ P ( W - s _R )

An example of calculation is as follows, assuming a concave mirror with a focal length of 35 mm from the eye 30 mm, and a display distance of 34.976 mm is calculated as the distance l that will produce a virtual image that will appear 50 meters from the user's eye. This example uses the Mathcad nomenclature.

Fl:=35 mm

Fl=0.035 m

Radius: =2fl=0.07 m

P:=1/fl=28.571 m ^-1

W:=50 m to the desired distance of the virtual image

Sr:=30 mm distance from the eye to the reflector

Elp:=W-sr=49.97m

Lp=-1/elp=-0.0200120072 m ^-1

L:=Lp-P=-28.591 m ^-1

El:=1/L=-0.035m (note, el=sp)

El:=-34.976 mm

m:=L/Lp=1.42871429 x 10 ³

Instead of calculating the position of the display given the power of the reflector, the position of the eye, and the position of the virtual image, the above analytical calculation can be used given the distance to the display and the eye and the desired distance to the virtual image. The power of the reflector. It can be seen from equation (6):

P = L' - L

Equation (9)

Substituting equations (5) and (7) into L' and L gives:

Equation (10)

Since l = s _P , equation (10) becomes

Equation (11)

As an example, if the desired image distance W is 50 [m], the reflector is 40 mm from the eye, and the display is 40 mm from the reflector, the reflector power needs to be P = 24.98 [D], that is, [0.04- (-0.04)-50]/[-0.04(50-0.04)]. Note that s _p is negative.

Thus, for a given orientation of the display, the distance to the surface of the reflector, and the distance from the reflector to the eye, the correct reflector power can be determined. In a concave spherical reflector, the power is

Equation (12)

Where f is the focal length in meters [m]. In a spherical mirror, the focal length is related to the radius of curvature r, such as

r = 2 f

Equation (13)

And therefore

Equation (14)

Therefore, in order to obtain the desired power as calculated from equation (11), it is necessary to ensure that the surface has a curvature specified by the radius calculation of equation (14).

If the display is a simple point source, the spherical concave reflector can meet the reflector requirements, but the display is typically a planar device having a grid of illuminating pixels or pixels that are geometrically offset from the geometry that can be achieved by the sphere. Again, as discussed above, it is desirable to spread light over a larger area to obtain a wider field of view, for example, to take advantage of the wide field of view (static and/or static + dynamic) of the human eye.

In accordance with the present invention, these challenges are met by dividing the reflective surface into a plurality (e.g., thousands) of surface elements 23 and adapting (adjusting) their position, orientation, and curvature to obtain desired reflector properties. The surface elements of the triangle have been found to work successfully in the optimization, but other shapes may be used as desired. An example of a subset of surface elements 23 is shown in Figure 3, wherein the core surface elements have eight surrounding surface elements, and all of the surface elements must exist in a relationship to one another in order to meet the optical property requirements for the overall reflective surface. Demonstrating the weight from the core surface element to the surrounding surface elements, and such weights are intended to allow more than the others to be involved in certain surface elements, such as at the edges or corners (see below) ), where more impact is required to move the surface, because there are fewer contributing peripheral surface areas, so the surface area at it needs to contribute more adaptive motion effects.

A flat or curved display surface is also divided into a plurality of segments referred to herein as "display objects" or "virtual pixels." There may be only a few display objects (even if only one large virtual pixel is theoretically possible) or thousands of virtual pixels (typically) that are geographically arranged on the display surface.

In a computer system (see below), a display surface consisting of display objects is created, the center of the eye is created, and an initial grid of reflective surface elements is created. The reflective surface elements are then pointed (ie, their normals are pointed) to allow reflection into the eye according to the Fermat Hero rule (described later) so that when the user views the reflector surface on the reflector surface It is desirable to see that at the point in the direction of the display object, the derivative of the length of the path between the display object and the center of rotation of the eye will have zero.

The radius of curvature and spatial position of the surface element are then calculated as follows. First, for each core surface element corresponding to a particular display object of the display, the analysis stated above is used to calculate the surface elements required to place the virtual image of the displayed object at a desired distance from the front of the nominal user's eye. The radius of curvature. Next, the surrounding surface elements are inspected to determine if they are in a suitable position consistent with the stacked spheres, the center of which is located on the normal to the core surface elements (see below). If not, some or all of the surrounding surface elements are moved toward their correct positions for the display object (virtual pixels) and core surface elements being considered. The process then proceeds to other display object/core surface component combinations until all combinations have been updated. As discussed below, the error function is then calculated and a determination is made as to whether further iterations are needed.

Figure 4 shows a two-dimensional illustration of the process by which the position of the surrounding surface elements can be adjusted and the error function evaluated. In this figure, the optimum point (reference numeral 25) is at the center of the circle (reference numeral 27), and the radius Raxis of the circle is calculated according to the following equation 16. In the case of 3 dimensions, the circle 27 will be a sphere, and thus in the following discussion, the circle 27 will be referred to as a sphere 27. Again, as fully discussed below, the sphere 27 will preferably have its center along the normal to the core surface element, rather than having its center at, for example, a virtual pixel.

As shown in Figure 4, the upper and lower surface elements (as in Figure 3) labeled u and d are not aligned with the sphere 27. This is an error. Using these errors, the error function can be determined by summing the errors across the reflective surface.

Equation (15)

Wherein the individual error ε is calculated as the difference between: the center of the surface element under consideration (such as the surface element u in Figure 4); and the surface of the sphere 27 at the center of the sphere and the sphere in consideration The point of intersection between the lines between the centers of the surface elements (e.g., the point at which the sphere 27 intersects the line from reference numeral 25 to the line at the center of the surface element u in Fig. 4).

Note that the center of the radius of the sphere used to move the surface element and calculate the error should preferably be from the core surface element where the radius of curvature will be parallel to the normal required to provide Fermat/Hero reflection (or more specifically, along the method) Where the line is laid. For the current from the eye and the distance s _r from the current display object (virtual pixels) of the distance s _p, referred to at this point having a radius of optPoint:

Note that according to optical convention, s _p is a negative number because it is reflected light from the mirror. This radius is used by first locating the line from the centroid of the current surface element to (a) the vector of virtual pixels and the line from the centroid of the current surface element to (b) the center of rotation of the eye. The distance Raxis is then traversed on this line to place the optPoint for use as the center of the sphere for error checking and iterative correction for surface elements at each surface element. The method for determining and using Raxis is illustrated in Figure 5.

The desired final configuration of the reflective surface is obtained by slowly moving the surface elements towards the optimal surface (in this case in Figure 5, which is a spherical surface). Note that the spherical surface of Figure 5 is only optimal for the specified combination of virtual pixels, eye center, and long-range virtual images (not shown in Figure 5). The current surface component is the surface component used to calculate Raxis and optPoint. For each core surface component, there will be different Raxis and optPoint. The surrounding surface elements are adjusted to begin to derive the correct radius of curvature Raxis defined for each of the displayed object (virtual pixels) and the core surface elements. Next, consider the next display object (virtual pixel) and core surface elements. This next core surface element can affect surface elements that are only affected by previous operations, which is why only a small number of changes are performed at each iteration. The goal is to minimize errors across the display surface. Alternatively, certain sections of the display surface may have less error than other sections.

If there is only one display object (virtual pixel), the zero error surface will be a sphere with the correct radius to provide sufficient diopter correction so that the image of the displayed object appears at the desired distance W of equation (4). The viewpoint s _R may be included in the calculation, but when collimation causes the expected virtual image to appear 50 meters in front of the observer, little difference is produced. The viewpoint s _{R is} included in the actual calculation, but when W = 50 [m], only the fourth significant digit calculated by the reflective surface is affected.

Moreover, if there is only one or a few dummy pixels, and there is a pupil to be viewed through it, and it is reasonable to expect that the waveform of the light is contained in a region near or around the optical axis of the optical device, Axis techniques such as those available for telescopes or photographic lenses analyze this system. In this case, however, as discussed above, there is no true abiotic optical axis, and errors are detected, and the system is characterized by the techniques disclosed herein. While classical and other techniques for measuring performance (eg, the system's modulation transfer function (MTF)) can be included in the error function, the error function including the error of the type described in equation (15) will be used. To sum the performance errors across the field of view and reduce the error. Of course, the magnitude of the total error that can be tolerated will depend on the particular application of the HMD and can be readily set by those skilled in the art based on the invention and the specifications that the HMD image needs to meet.

It should be noted that the eye can handle a defocus of about 0.5D, and a defocus of about 0.5D can also be used as part of the error calculation, for example, when the optimization cycle has occurred, the reflective surface at each reflection point is determined. The average estimate of the radius of curvature. In order to achieve a smooth transition of the view across the field of view, the reflective surface elements can also transition smoothly with each other. For example, smoothing can be performed by applying a non-uniform rational B-spline (NURBS) technique to the splined surface, thus creating a smooth transition across the reflective optical surface.

The error surface discussed above is a measure on which the improvement in surface quality and the efficacy of surface quality are determined. In order to subsequently improve the surface reflection quality, the individual surface elements are moved with respect to the errors contributed by the individual surface elements. This is illustrated in Figure 4 by the two-way arrow identified by the word "correction." Individual surface elements are moved in this direction to reduce errors for a given core surface element sphere. In some embodiments, a rate variable β = [0..1] is used to determine the amount of adjustment made at each iteration to ensure that the surface elements move slowly enough. The spatial position of all of the surrounding surface elements in the vicinity of the core surface element is adjusted and then the same operation is performed at the next core surface element. As discussed above, various weightings of the effects of the core surface elements on nearby surface elements can be used such that surface elements, for example, near corners or edges, are also more fully enclosed by core surface elements that are incrementally changeable by supply. Similar amounts of surface elements are adjusted.

There are three types of surface elements: (1) surface elements with all nine triangles; (2) surface elements lacking a set of three triangles (such as at the edges); and (3) missing five A surface element of a triangle (as would occur at a corner). In each case, cs is present; the difference is the number of surrounding surface elements. In such cases, the influence weights are used to allow for an increase in the amount of adjustment of the surface elements to be more commensurate with the adaptation that can occur if the surface elements are surrounded by other surface elements.

In particular, at a corner, a given surface element is surrounded by only three surface elements rather than by eight surface elements, as shown in FIG. Only three of the eight potential influencers can be used to provide corrections, and therefore, the 8/3 impact weight is used to adjust the amount of correction that each of the core surface elements at position 1, 2, or 3 can provide. This influence weight is multiplied for the three-dimensional movement that has been reduced by the rate β. Similarly, the effect weight on the surface elements at the edges is 8/5.

When moving the surface element, the surface curvature is controlled to obtain the correct power in the field of view as the distance between the surface element, the display object, and the nominal user's eye changes. The normal to the surface elements is also adapted to ensure that the pointing angle of the area of the display (display object) is correct.

It is important to be able to stretch the field of view over a wide angle to allow the user to see more information with their perimeter vision and to scan the display in a more natural manner.

Since the era of Fermat and Hero of Alexandria, it is known that the image will appear on the mirror, and since additional follow-up work, the image has been shown to be at the point where the length of the light path has reached a stable point (maximum or minimum). This stable point can be found by finding the zero of the first derivative of the length of the optical path. For ease of presentation, assuming that the entire path of light is in the air, it will be appreciated that those skilled in the art can readily adapt the disclosed method to situations where all or part of the optical path is comprised of one or more different optical materials. For example, a circle having a radius r centered on [x, y] = [0, 0] has the following equation

x ² + y ² = r ²

Equation (17)

Solve x and get

Equation (18)

Assumed a point light source (S) having coordinates [x _S, y _S] of and having coordinates [x _V, y _V] of the viewport (V) in the air of the spherical reflector (e.g., in this assay expressed as in FIG. In the space around the circle 33), the optical path L is composed of the path length from the point source to the point Q on the surface (the image is seen here) and the path length from the Q to the inspection point. for

Equation (19)

For y to the equation (19), find the first-order partial differential (for the positive root) given

Equation (20)

And given a negative root term

Equation (21)

The concept can be tested with some representative values. Select a pair of points in Figure 7, and the pair is

V=[20,-50]

S=[40,40]

And the circle 33 of the radius 100 is centered on the origin 43.

A positive value is used for the x value associated with the y value of equation (20) and a negative value is used for the x value associated with the y value of equation (21), from equation (20) for y and (21) and the two points predicted for equation (18) of x are:

Q1=[-98.31,18.276]

Q2=[97.685,21.392]

These points are shown graphically in Figure 7, where the light from source S reflected at Q1 is shown by reference numeral 39, and the light from source S reflected from Q2 is shown by reference numeral 41. It can be seen that the line from point V and S to Q1 or Q2 can be equally divided by the line from the origin to the point Q1 or Q2 (i.e., lines 37 and 35). This is also the property of the core surface elements used herein, that is, the angle between the viewer's eye and the viewing object being viewed is halved by the normal to the core surface element. This property is used when the surface element is removed from a simple spherical surface to create a defined free space position by angling the surface normal such that it will bisect the vector between the viewer's eye and the display object to be inspected. . This orientation of the surface element can be performed, for example, using a quaternion method to rotate the surface element to a halfway between the rotation of the vector defining the viewer and the vector defining the vector of the display object being viewed. Orientation.

For example, in Figure 7, there is an additional line 31 from the center 43 of the circle 33 to the edge of the circle. It indicates that another point A2 of the image from point S may need to be displayed on the reflective surface. If the surface at the point has a surface normal to the vector of V and the vector to S, then when the observer at V looks at the new point on the surface, the image of S will appear at V. At the office.

This situation is further illustrated in Figure 8. In this figure, line 45 has been drawn at point A2. This point provides a normal 47 that bisects the vector from A2 to S and the vector from A2 to V. Another point A1 is made on line 45, and the point A1 is manually measured to allow the point-angled equation of line 45 to be derived, from which the equation according to y is obtained.

A ₁ =[45,-100]

A ₂ =[78,-62.85]

y = mx + b

b = y - mx

=-100- m (45)=-150.6591

therefore

Where the optical path length =

or

And

On the range of y=[-200..100], this partial derivative has only one zero, which is at y=-63.4828 (corresponding to the value of x.77.49), which is at the expected position where the line meets the circular curve. . Figure 9 is a graph of the partial length of the optical path length of line 45 with respect to the y coordinate of the line. As can be seen, the graph has only one zero.

Importantly, note that line 45 is not a tangent to circle 33. It has a different slope with a normal dividing the vector from A2 to viewpoint V and the vector of the displayed object from A2 to S. This is a way for the invention to place an image of an individual virtual pixel or an area of the display in different regions of the view area, and to adjust the slope of the core surface element and to check for errors in the surface element by iteratively Adjust its position until the error is optically acceptable to expand the field of view.

Returning to Figure 7, it can be seen in this figure that images from a single point can appear at multiple points on the reflector, in this case on both sides of the circular reflector. Therefore, an analysis such as the analysis performed in Figure 9 is required to detect spurious copying of the image. False images can also be detected by ray tracing. Ray tracing shows that only rays striking point Q1 or point Q2 will pass through V after leaving S. More generally, if any ray passes through V from a point such as S, the user can check where the ray is coming from. For example, another ray that will be apparent from this inspection is a ray that passes directly through the space from S to V. In the design of the HMD, this direct path can be physically blocked to reduce internal optical noise. Alternatively, the view of the available eye can naturally block the false image.

10 and 11 are flow diagrams outlining the above procedure for creating a reflective surface for use in an HMD. In particular, Figure 10 shows an overall strategy for creating an overall system including an initial reflective surface in one or more computers and then iteratively adjusting the initial reflective surface by adjusting the spatial position of the surface elements, adjusting them, etc. The radius of curvature of the component and the orientation of the surface component in the desired direction. The error is then calculated and used to determine if other iterations are needed or if the final surface configuration can be output. Figure 11 depicts an embodiment using equations (15) and (16) discussed above.

According to a particular embodiment of the procedure set forth in these figures, the iterative process uses a series of "on" surface elements. For a given "on" component, the surrounding components are only adjusted in one iteration, after which the system proceeds to the next component (the next "on" component) and adjusts its surrounding components, and so on. The components of the system "on" are unchanged, only the neighbors are changed to better match the surface of the sphere that is "on" the component and that is "on" to the "optPoint". Only one adjustment is made to each neighbor in one iteration, and then the process continues to the next "on" component, and all its neighbors are adjusted once until all surface elements have been "turned on" "element. The global error is then calculated, and if it is not low enough, the process repeats. The process does not repeatedly adjust adjacent components for an "on" component until proceeding. Rather, the process makes a small adjustment to each neighbor as needed at each iteration, and then proceeds to the next "on" component and its neighbor set. The result of this iterative adjustment of the spatial position of the surface elements is the final spatial position of each surface element, the final direction of the normal to each element, and the final radius of curvature of each element and a set of surrounding elements. When the error function satisfies a predetermined criterion, for example, when the error function is less than a predetermined value, the final position, the normal, and the radius of curvature are output, for example, stored in the memory.

Figures 12 and 13 show reflective surfaces created using the above techniques from two different perspective views. Figures 14 and 15 again show another modified version of the reflective surface of Figures 12 and 13 from two perspective views. As can be seen from these figures, the configuration of the reflective optical surface is quite complex and has little resemblance to a spherical or aspheric surface created by other optical design techniques. Individual surface elements can be splined together to create a smooth continuous surface, or many surface elements can be calculated such that the surface becomes smooth at fine grain scales.

The use of reflective optical surfaces designed in accordance with the methods disclosed herein is set forth at the same time as the name of G. Harrison, D. Smith and G. Wiese, respectively, entitled "Head-Mounted Display Apparatus Employing One or More Reflective Optical". United States Patent Application No. 13/211,372, entitled "Head-Mounted Display Apparatus Employing One or More Fresnel Lenses" and identified by the agent number IS-00267 and IS-00307, respectively. And the contents of the two applications are incorporated herein by reference.

The mathematical techniques discussed above (including the flowcharts of Figures 10 and 11) can be encoded in various programming environments now known or subsequently developed and/or in various programming languages now known or subsequently developed. The current preferred programming environment is the Java language implemented in the Eclipse Programmer interface. Other programming environments such as Microsoft Visual C# can also be used as needed. Calculations can also be performed using the Mathcad platform marketed by PTC (Needham, Mass.) and/or the Matlab platform from MathWorks, Inc., (Natick, Massachusetts). The resulting program can be stored on a hard drive, memory card, CD or similar device. Such programs can be executed using typical desktop computing devices available from multiple vendors (eg, DELL, HP, TOSHIBA, etc.). Alternatively, use a more powerful computing device that includes "cloud" computing as needed.

Once designed, the reflective optical surfaces disclosed herein can be produced (eg, mass produced) using a variety of techniques and materials that are now known or subsequently developed. For example, the surfaces can be made of a plastic material that has been metallized to have suitable reflectivity. Polished plastic or glass materials can also be used. For "Actual Reality" applications, the reflective optical surface can be constructed from a transmissive material having a small embedded reflector, thereby reflecting the portion of the incident wavefront while allowing light to pass through the material.

For prototype parts, acrylic plastic (for example, colloidal glass) can be used for parts that are being formed by diamond turning. For producing parts, acrylic or polycarbonate can be used, for example, for parts that are being formed by, for example, injection molding techniques. The reflective optical surface can be described as a detailed computer-aided drawing (CAD) description or as a non-uniform rational B-spline NURBS surface (which can be converted into a CAD description). Having a CAD file allows the device to be fabricated using 3D printing, in which case the CAD description directly produces 3D objects without machining.

Many modifications of the scope and spirit of the invention will be apparent to those skilled in the <RTIgt; For example, while providing a reflective optical surface that provides a user with a large field of view (eg, greater than or equal to 100°, or greater than or equal to 150°, or greater than or equal to 200°) constitutes an advantageous embodiment of the present invention, The methods and systems disclosed herein can be used to create reflective surfaces with smaller fields of view.

Similarly, although the invention has been described in terms of a system in which light emitted from a display has not been collimated before it reaches the reflective surface, it is equally applicable to partial or complete collimation (eg, by being located on a display and reflective surface) The light between the optical components). In such cases, the radius of curvature of the core surface elements will be adjusted to account for the collimation of light incident on the elements.

The following claims are intended to cover such and other modifications, variations and equivalents

11. . . Display/display surface

13. . . Free space reflective optical surface

15. . . Nominal user's eye

17. . . The center of rotation of the nominal user's eye

19. . . Light from the display

twenty one. . . image

twenty three. . . Surface component

25. . . Display object

27. . . Circle/sphere

31. . . Extra line

33. . . circle

35. . . line

37. . . line

39. . . Light

41. . . Light

43. . . Center of the circle

45. . . line

47. . . Normal

A1. . . point

A2. . . point

P. . . Line segment

Q1. . . point

Q2. . . point

R. . . Line segment

S. . . point Light

V. . . View point

Figure 1 is a schematic diagram showing the basic components of the HMD (i.e., the display, the reflective surface, and the eyes of the user).

2 is a schematic diagram showing the formation of a virtual image of an object (display) by the formation of a reflective surface and determining the position and size of the virtual image.

Figure 3 is a schematic diagram illustrating core surface elements and their adjacent surface elements.

Fig. 4 is a view for explaining the calculation of the error of the position of the surface element and the moving direction of the surface element to reduce the error.

Figure 5 is a schematic diagram illustrating the manner in which surface elements can be moved incrementally based on the optPoint / virtual pixel / radius of curvature / s _p / s _r set.

Figure 6 is a schematic view showing a corner surface element.

Figure 7 is a schematic diagram showing two optical paths between a source S and a viewer V for a circular reflector.

Figure 8 is a schematic diagram illustrating a single optical path between source S and viewer V for a flat surface having a normal that is not laid along the radius of the sphere.

Figure 9 is a graph showing that the first derivative of the optical path length between S and V of Fig. 8 has only one zero (point indicating the stability of the optical path).

Figure 10 is a flow chart illustrating an embodiment of the present invention.

Figure 11 is a flow chart illustrating another embodiment of the present invention.

Figures 12 and 13 illustrate reflective optical surfaces designed using the methods and systems disclosed herein from two perspective views.

Figures 14 and 15 illustrate, from two perspective views, another reflective optical surface designed using the methods and systems disclosed herein.

(no component symbol description)

Claims (23)

A computer-based method for designing a free-space reflective optical surface for use in a head mounted display that produces a virtual image of a display surface for use by a user's eye Detecting at a pre-selected spatial location, the method comprising performing the following steps using one or more computers: (a) representing the display surface by a plurality of display objects; (b) representing the free-space reflection by a plurality of surface elements An optical surface, each surface element characterized by: (i) a spatial position relative to the display surface, a nominal user's eye, and the pre-selected spatial location of the virtual image, (ii) a normal, and (iii) a radius of curvature; (c) wherein during the use of the head mounted display, a virtual image of each of the displayed objects at the preselected spatial location is displayed in the direction of the eye of a nominal user. The display object is associated with at least one surface element, each surface element being associated with one and only one display object; (d) for each surface element, performing the following steps: (i) defining One of the initial spatial positions of the component; (ii) calculating the normal of the component using the initial spatial position of the component, the position of the display object associated with the component, and the position of a center of rotation of a nominal user's eye An initial direction such that light from the display object reflected from the element will pass through the center of rotation; and (iii) calculating an initial radius of curvature of the element such that the virtual image of the display object is in the preselected And (e) for each surface element, by adjusting the spatial positions of the elements in an iterative manner until an error function satisfies a predetermined criterion, calculating a final spatial position of the element, the method of the element The final direction of one of the lines and the final radius of curvature of one of the elements and a set of surrounding elements. The method of claim 1, wherein in step (e), the iterative adjustment of the spatial position of the at least one first surface element is based at least in part on the radius of curvature and the normal of the at least one second surface element, the A surface element is the nearest neighbor of one of the second surface elements. The method of claim 2, wherein the iterative adjustment of the spatial position of the first surface element is based on a calculated deviation of a spatial position of the element from a sphere having a radius equal to a radius of curvature of the second element and Its center is along the normal of the second element. The method of claim 3, wherein the iterative adjustment comprises a deviation that is less than the entire calculation. The method of claim 1, wherein the iterative adjustment of the plurality of first surface elements is based at least in part on the radius of curvature and the normal of the at least one second surface element, each of the plurality of first surface elements being One of the second surface elements is nearest neighbor. The method of claim 5, wherein the plurality of first surface elements constitute all of the nearest neighbors of the second surface element. The method of claim 5, wherein the iterative adjustment of the spatial locations of the first surface elements is based on a deviation of a spatial position of the elements from a sphere calculated, the radius of the sphere being equal to the curvature of the second element Radius and its center along the normal of the second element. The method of claim 7, wherein the iterative adjustment comprises a deviation that is less than the entire calculation. The method of claim 7, wherein the error function is based on deviations from the calculations. The method of claim 9, wherein the error function is based on a sum of one of absolute values of the deviations of the calculations. The method of claim 10, wherein the predetermined criterion is a value of the sum of the absolute values. The method of claim 5, wherein the iterative adjustment of the spatial locations of the first surface elements is weighted such that the iterative adjustment of at least one of the first surface elements is greater or less than its weighting The adjustments that have been made in the case. The method of claim 5, wherein the second surface element is an edge surface element or a corner surface element, and the iterative adjustment of the spatial position of the at least one first surface element is weighted such that the iterative adjustment is greater than its unweighted In the case of the adjustments that have been made. The method of claim 1, comprising calculating a smoothing based on the final spatial positions of the plurality of surface elements calculated in step (e), the final directions of the normals, and the final radius of curvature The extra step of free space to reflect the optical surface. A computer-based method for designing a free-space reflective optical surface for use in a head mounted display that produces a virtual image of a display surface for use by a user's eye Inspecting, the method comprising performing the following steps using one or more computers: (a) representing the display surface by a plurality of display objects; (b) representing the free-space reflective optical surface by a plurality of surface elements; and (c) Computing at least one spatial position and at least one normal line in an iterative manner for each of the surface elements, the at least one spatial position and the at least one normal line causing a virtual image of each of the display objects to be displayed for the object One of the nominal user's eyes is displayed in the gaze direction to the eye. A computer-based method of claim 15, wherein a radius of curvature is calculated in an iterative manner for each of the surface elements in step (c). The method of claim 1 or 15, wherein an angle between at least two of the plurality of surface elements is greater than or equal to 100 degrees when measured from the center of rotation of the eye of the nominal user. The method of claim 1 or 15, wherein the angle between at least two of the plurality of surface elements is greater than or equal to 150 degrees when measured from the center of rotation of the eye of the nominal user. The method of claim 1 or 15, wherein the angle between at least two of the plurality of surface elements is greater than or equal to 200 degrees when measured from the center of rotation of the eye of the nominal user. The method of claim 1 or 15, further comprising producing the free-space reflective optical surface. A computer program embodied in a tangible computer readable medium for performing the method of claim 1 or 15. A computer system programmed to perform the method of claim 1 or 15. A system comprising: (a) a processor; and (b) a memory unit coupled to the processor, the memory unit storing including a program for performing the method of claim 1 or 15 A computer program of instructions.