CA3040218C - Apparatus and method for a bioptic real time video system - Google Patents
Apparatus and method for a bioptic real time video system Download PDFInfo
- Publication number
- CA3040218C CA3040218C CA3040218A CA3040218A CA3040218C CA 3040218 C CA3040218 C CA 3040218C CA 3040218 A CA3040218 A CA 3040218A CA 3040218 A CA3040218 A CA 3040218A CA 3040218 C CA3040218 C CA 3040218C
- Authority
- CA
- Canada
- Prior art keywords
- eye display
- head
- wearer
- electronic near
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title abstract description 8
- 230000006641 stabilisation Effects 0.000 claims abstract description 9
- 238000011105 stabilization Methods 0.000 claims abstract description 9
- 230000003287 optical effect Effects 0.000 claims abstract 2
- 210000003128 head Anatomy 0.000 claims description 31
- 239000013598 vector Substances 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims 16
- 210000001061 forehead Anatomy 0.000 claims 2
- 238000010348 incorporation Methods 0.000 abstract description 2
- 230000004438 eyesight Effects 0.000 description 6
- 230000005043 peripheral vision Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000002650 habitual effect Effects 0.000 description 4
- 208000013057 hereditary mucoepithelial dysplasia Diseases 0.000 description 4
- 230000004886 head movement Effects 0.000 description 3
- 230000036544 posture Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0127—Head-up displays characterised by optical features comprising devices increasing the depth of field
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
A method and apparatus of displaying an electronic video image using a head- worn near-to-eye display in a non-immersive fashion, such that the wearer can choose, through simple adjustments of their neck and eye angles, to either look at the displayed video image or their natural environment. The invention also relates to the incorporation of prescription lenses into the optical chain of the near-to-eye display. The invention also relates to the use of motion and position sensors incorporated into the head-worn device to enable automatic stabilization of the video image. The invention also relates to the use of motion and position sensors incorporated into the head-worn device to automatically adjust the vertical angle of either the camera or the electronic display or both, by sensing the vertical angular position of the user's head.
Description
AiARATUS Aril) IVIETIIIOD FORAinOFrIC REAL Tir VIDEOSYSTEM
CROSS-REFERENCE TO RELATED APPLICATIONS
[001] This application claims the benefit of priority as a divisional of CA
CROSS-REFERENCE TO RELATED APPLICATIONS
[001] This application claims the benefit of priority as a divisional of CA
2,875,261 which was filed December 1, 2014 entitled "Apparatus and Method for a Bioptic Real Time Video System", which itself claims priority as a national phase entry of World Intellectual Property Office PCT/CA2012/000532 filed June 1, 2012 entitled "Apparatus and Method for a Bioptic Real Time Video System."
FIELD OF THE INVENTION
[002] The invention relates generally to the field of wearable electronic displays and more specifically to the field of vision care.
BACKGROUND OF THE INVENTION
FIELD OF THE INVENTION
[002] The invention relates generally to the field of wearable electronic displays and more specifically to the field of vision care.
BACKGROUND OF THE INVENTION
[003] There are numerous applications for lightweight head-worn near-to-eye displays.
These are commonly called Head Mounted Displays (HMD). HMDs display to the eye an electronically rendered image such that the wearer perceives that they are watching a sizeable electronic display at some distance in front of them. The applications that use such HMDs are numerous, including but not limited to virtual reality, electronic gaming, simulation environments such as for military simulations or flight simulators, medical applications such as for the enhancement of sight, and consumer applications such as the ability to view videos in a mobile setting.
These are commonly called Head Mounted Displays (HMD). HMDs display to the eye an electronically rendered image such that the wearer perceives that they are watching a sizeable electronic display at some distance in front of them. The applications that use such HMDs are numerous, including but not limited to virtual reality, electronic gaming, simulation environments such as for military simulations or flight simulators, medical applications such as for the enhancement of sight, and consumer applications such as the ability to view videos in a mobile setting.
[004] One of the fundamental challenges of HMDs is the tradeoff between the display's Field of View (FOV), being the size of the virtual display as perceived by the wearer, and pixel size. FOV is normally defined as the number of angular degrees subtended within the viewer's overall field of vision, horizontally, vertically, or on the diagonal. Horizontal FOV
dimensions in the range of 20-30 degrees are typical, with larger dimensions being possible at significant expense. Pixel size is similarly expressed as the number of angular arc minutes (1/60th of a degree) subtended by a single, typically square pixel element. As one might - I -expect, to achieve a larger FOV with a given pixel resolution (number of pixels), results in a larger pixel size, and consequent loss of image detail.
1005] Peripheral vision is that portion of the human field of vision outside the center, say, 10-15 degrees FOV. Peripheral vision is extremely important in some HMD
applications, especially those in which the wearer must maintain a connection with their natural environment to contextualize their situation, and enable way finding, orientation, and mobility. To provide significant peripheral vision via the electronic display requires an extremely large (and expensive) HMD. Alternately HMDs which provide a significant natural peripheral vision external to the HMD housing, provide a very limited virtual electronic FOV.
[006] Many HMD applications can benefit from the incorporation of a live camera into the HMD, such that the wearer can not only view electronic data from a source, such as a video file, but also live video images of the world in front of them. Image processing can be used to enhance the live camera image before it is presented to the eye, providing magnification, enhancement of brightness, or improved contrast for example.
[007] In HMD systems that are to be used for multiple activities, different camera angles may be required for different tasks. For example, to observe an object a distance, the camera angle should be nearly horizontal relative to the horizon when the wearer's neck is straight, and their gaze angled at the horizon. On the other hand, to view hand-held objects at a close distance requires a camera that is angled downward, in order to avoid a highly exaggerated downward neck posture. In this manner, the angle of the camera mimics the angular movement of one's eyes in a non-HMD world.
[008] Finally, the angle of the display relative to the eyes is also dependent on the specific tasks of the wearer. In certain situations, the wearer would like to look into the electronic display only temporarily, and by looking up at an angle higher than their habitual gaze. In other situations, the wearer would prefer a more immersive usage model, where the electronic display is directly in front of their normal line of gaze.
[009] What is needed then is a general HMD device that is capable of providing significant unobstructed peripheral vision outside of the electronic display FOV, while simultaneously providing a high resolution video image. Further, the ability to adjust the angle of the display and the camera according to the wearer's specific activity would provide significant comfort and increased usability.
[0010] Other aspects and features of the present invention will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying figures.
SUMMARY OF THE INVENTION
[0011] The ability to quickly alternate as required by the specific task, between viewing an image presented in the electronic display and viewing the world without the electronic display, enables many possible usage models for an HMD. Furthermore, in an HMD
with an integrated camera, the ability to adjust the vertical camera angle for different tasks, viewing objects distant and close for example, significantly increases the usability of such a device.
Finally, an HMD whereby the user is able to select the vertical position of the electronic display, in order to tradeoff a comfortable immersive video experience versus maintaining a broad natural field of view enables the HMD to be used in a variety of user applications and postures.
[0012] The invention, in one aspect, relates to a method of orienting an electronic near-to-eye display such that the wearer views it slightly above their habitual line of sight for a given task. In this manner the wearer, through slight neck and eye angle adjustments can, with minimal effort, alternate between the electronic display and their natural vision.
[0013] In one embodiment, the electronic display is mounted slightly above the wearer's habitual line of sight, so that by angling the neck slightly forward and directing their gaze slightly upward, they can look into the display. Alternately by angling the neck slightly back and directing their gaze slightly down, they can view below the electronic display using their natural vision.
10014] In another embodiment, the apparatus incorporates the wearer's prescription ophthalmic lenses, so that whether they gaze up into the electronic HMD or down using their natural vision, they are able to do so through their prescription lenses. This embodiment of the invention alleviates the need to switch between the HMD device and the wearer's habitual spectacles.
[0015] In another embodiment the apparatus incorporates a video camera, which provides the video information to the electronic display, the angle of the camera being adjustable as required by the specific task.
[0016] In any of the above embodiments the source of the video may come from a device other than the camera, in any standard electronic video format such as MPEG
for example.
[0017] In another embodiment the apparatus may deploy motion sensing components in order to facilitate image stabilization for the electronic video image that is captured by the camera.
[0018] In another embodiment the motion sensing components could be used to determine the angular orientation of the apparatus, so that the vertical camera angle can be automatically adjusted based on head position.
BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The invention is pointed out with particularity in the appended claims.
The advantages of the invention described above, together with further advantages, may be better understood by referring to the following description taken in conjunction with the accompanying drawings. In the drawings, like reference characters generally refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.
[0020] FIGS. 1 through 4 are highly schematic diagrams of an embodiment of the system of the invention;
[0021] FIG. 5 is a more detailed view of an embodiment of the system for automatically adjusting the angle of the camera.
[0022] FIG. 6 is a more realistic rendering of a particular embodiment of the system of the invention.
[0023] FIGS. 7A through 7C depict three successive frames of simulated video, in order to show how motion vectors can be used for image stabilization.
[0024] FIGS. 8A and 8B show an alternate method of altering the camera angle through defining a window on the image sensor rather than by physically altering the camera angle.
DETAILED DESCRIPTION
[0025] In brief overview and referring to FIG. 1 and FIG. 6, the system in one embodiment includes prescription lenses 6 mounted to an eyeglasses frame 1. The Head Mounted Display portion of the system comprises a housing 2, which can move relative to the eyeglasses frame 1, about a pivot point 3. The HMD housing 2 incorporates HMD optics 5, a camera 9, and HMD electronics 8 (collectively, the "HMD").
[0026] In the orientation depicted in FIG. 1, the wearer's head/neck posture is angled slightly back 4, and he is viewing the world 7 through the prescription lens 6, without the use of the HMD optics 5 or camera 9.
[0027] In FIG. 2, the head/neck angle 4 is slightly forward, allowing the user to view the HMD optics 5 through the same prescription lens 6 by directing their gaze at a slight upward angle 7. In this mode, video information viewed through the HMD optics 5 can be provided from the video camera 9, oriented at an outward angle 10 such that objects at a distance can be viewed in the video image. As discussed, the video could also come from other sources.
[0028] In FIG. 3 the head/neck angle 4 is unchanged from FIG. 2, but the camera has been angled downward on a pivot point 11 so that the camera angle 10 is now aimed at a nearby object close at hand, perhaps in the wearer's hands.
[0029] In FIG. 4 the slightly forward head/neck angle 4 remains unchanged, but the HMD
angle has been significantly lowered by pivoting the HMD housing 2 relative to the eyeglasses frame 1, around a pivot point 3. In this orientation the wearer is able to adopt a more comfortable viewing angle 7 for near-in tasks. Furthermore, the camera angle 10 can be directed further downward because the camera pivot point 11 moves with the HMD
housing 2.
[0030] In FIG. 5, a method is shown whereby a linear motor 12 can be used to adjust the vertical angle 10 of the camera 9. The camera rotated around a pivot point 11 that is affixed to the HMD housing 2. With the adjustment of the camera angle automated thus, it is possible to use motion and position sensors embodied in the HMD electronics 8, to provide positional data that can be used to control the linear motor 12.
[0031] Alternately, the angle 10 of the camera 9 can be adjusted manually by the user.
[0032] FIG. 7 shows how the same motion and position sensors embodied in the HMD
electronics 8 can be used to enable an image stabilization function. FIGS. 7A, 7B, and 7C
show three consecutive frames of video as captured by the camera 9. Because of the normal movement of the wearer's head, successive frames of video have a translational (up/down, left/right) and rotational (angular) position relative to the previous frame.
The frame depicted in FIG. 7B for example has shifted to the right/down and rotated slightly counter-clockwise relative to the previous frame 7A. The translational vector Al and rotational angle 01 can be
dimensions in the range of 20-30 degrees are typical, with larger dimensions being possible at significant expense. Pixel size is similarly expressed as the number of angular arc minutes (1/60th of a degree) subtended by a single, typically square pixel element. As one might - I -expect, to achieve a larger FOV with a given pixel resolution (number of pixels), results in a larger pixel size, and consequent loss of image detail.
1005] Peripheral vision is that portion of the human field of vision outside the center, say, 10-15 degrees FOV. Peripheral vision is extremely important in some HMD
applications, especially those in which the wearer must maintain a connection with their natural environment to contextualize their situation, and enable way finding, orientation, and mobility. To provide significant peripheral vision via the electronic display requires an extremely large (and expensive) HMD. Alternately HMDs which provide a significant natural peripheral vision external to the HMD housing, provide a very limited virtual electronic FOV.
[006] Many HMD applications can benefit from the incorporation of a live camera into the HMD, such that the wearer can not only view electronic data from a source, such as a video file, but also live video images of the world in front of them. Image processing can be used to enhance the live camera image before it is presented to the eye, providing magnification, enhancement of brightness, or improved contrast for example.
[007] In HMD systems that are to be used for multiple activities, different camera angles may be required for different tasks. For example, to observe an object a distance, the camera angle should be nearly horizontal relative to the horizon when the wearer's neck is straight, and their gaze angled at the horizon. On the other hand, to view hand-held objects at a close distance requires a camera that is angled downward, in order to avoid a highly exaggerated downward neck posture. In this manner, the angle of the camera mimics the angular movement of one's eyes in a non-HMD world.
[008] Finally, the angle of the display relative to the eyes is also dependent on the specific tasks of the wearer. In certain situations, the wearer would like to look into the electronic display only temporarily, and by looking up at an angle higher than their habitual gaze. In other situations, the wearer would prefer a more immersive usage model, where the electronic display is directly in front of their normal line of gaze.
[009] What is needed then is a general HMD device that is capable of providing significant unobstructed peripheral vision outside of the electronic display FOV, while simultaneously providing a high resolution video image. Further, the ability to adjust the angle of the display and the camera according to the wearer's specific activity would provide significant comfort and increased usability.
[0010] Other aspects and features of the present invention will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying figures.
SUMMARY OF THE INVENTION
[0011] The ability to quickly alternate as required by the specific task, between viewing an image presented in the electronic display and viewing the world without the electronic display, enables many possible usage models for an HMD. Furthermore, in an HMD
with an integrated camera, the ability to adjust the vertical camera angle for different tasks, viewing objects distant and close for example, significantly increases the usability of such a device.
Finally, an HMD whereby the user is able to select the vertical position of the electronic display, in order to tradeoff a comfortable immersive video experience versus maintaining a broad natural field of view enables the HMD to be used in a variety of user applications and postures.
[0012] The invention, in one aspect, relates to a method of orienting an electronic near-to-eye display such that the wearer views it slightly above their habitual line of sight for a given task. In this manner the wearer, through slight neck and eye angle adjustments can, with minimal effort, alternate between the electronic display and their natural vision.
[0013] In one embodiment, the electronic display is mounted slightly above the wearer's habitual line of sight, so that by angling the neck slightly forward and directing their gaze slightly upward, they can look into the display. Alternately by angling the neck slightly back and directing their gaze slightly down, they can view below the electronic display using their natural vision.
10014] In another embodiment, the apparatus incorporates the wearer's prescription ophthalmic lenses, so that whether they gaze up into the electronic HMD or down using their natural vision, they are able to do so through their prescription lenses. This embodiment of the invention alleviates the need to switch between the HMD device and the wearer's habitual spectacles.
[0015] In another embodiment the apparatus incorporates a video camera, which provides the video information to the electronic display, the angle of the camera being adjustable as required by the specific task.
[0016] In any of the above embodiments the source of the video may come from a device other than the camera, in any standard electronic video format such as MPEG
for example.
[0017] In another embodiment the apparatus may deploy motion sensing components in order to facilitate image stabilization for the electronic video image that is captured by the camera.
[0018] In another embodiment the motion sensing components could be used to determine the angular orientation of the apparatus, so that the vertical camera angle can be automatically adjusted based on head position.
BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The invention is pointed out with particularity in the appended claims.
The advantages of the invention described above, together with further advantages, may be better understood by referring to the following description taken in conjunction with the accompanying drawings. In the drawings, like reference characters generally refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.
[0020] FIGS. 1 through 4 are highly schematic diagrams of an embodiment of the system of the invention;
[0021] FIG. 5 is a more detailed view of an embodiment of the system for automatically adjusting the angle of the camera.
[0022] FIG. 6 is a more realistic rendering of a particular embodiment of the system of the invention.
[0023] FIGS. 7A through 7C depict three successive frames of simulated video, in order to show how motion vectors can be used for image stabilization.
[0024] FIGS. 8A and 8B show an alternate method of altering the camera angle through defining a window on the image sensor rather than by physically altering the camera angle.
DETAILED DESCRIPTION
[0025] In brief overview and referring to FIG. 1 and FIG. 6, the system in one embodiment includes prescription lenses 6 mounted to an eyeglasses frame 1. The Head Mounted Display portion of the system comprises a housing 2, which can move relative to the eyeglasses frame 1, about a pivot point 3. The HMD housing 2 incorporates HMD optics 5, a camera 9, and HMD electronics 8 (collectively, the "HMD").
[0026] In the orientation depicted in FIG. 1, the wearer's head/neck posture is angled slightly back 4, and he is viewing the world 7 through the prescription lens 6, without the use of the HMD optics 5 or camera 9.
[0027] In FIG. 2, the head/neck angle 4 is slightly forward, allowing the user to view the HMD optics 5 through the same prescription lens 6 by directing their gaze at a slight upward angle 7. In this mode, video information viewed through the HMD optics 5 can be provided from the video camera 9, oriented at an outward angle 10 such that objects at a distance can be viewed in the video image. As discussed, the video could also come from other sources.
[0028] In FIG. 3 the head/neck angle 4 is unchanged from FIG. 2, but the camera has been angled downward on a pivot point 11 so that the camera angle 10 is now aimed at a nearby object close at hand, perhaps in the wearer's hands.
[0029] In FIG. 4 the slightly forward head/neck angle 4 remains unchanged, but the HMD
angle has been significantly lowered by pivoting the HMD housing 2 relative to the eyeglasses frame 1, around a pivot point 3. In this orientation the wearer is able to adopt a more comfortable viewing angle 7 for near-in tasks. Furthermore, the camera angle 10 can be directed further downward because the camera pivot point 11 moves with the HMD
housing 2.
[0030] In FIG. 5, a method is shown whereby a linear motor 12 can be used to adjust the vertical angle 10 of the camera 9. The camera rotated around a pivot point 11 that is affixed to the HMD housing 2. With the adjustment of the camera angle automated thus, it is possible to use motion and position sensors embodied in the HMD electronics 8, to provide positional data that can be used to control the linear motor 12.
[0031] Alternately, the angle 10 of the camera 9 can be adjusted manually by the user.
[0032] FIG. 7 shows how the same motion and position sensors embodied in the HMD
electronics 8 can be used to enable an image stabilization function. FIGS. 7A, 7B, and 7C
show three consecutive frames of video as captured by the camera 9. Because of the normal movement of the wearer's head, successive frames of video have a translational (up/down, left/right) and rotational (angular) position relative to the previous frame.
The frame depicted in FIG. 7B for example has shifted to the right/down and rotated slightly counter-clockwise relative to the previous frame 7A. The translational vector Al and rotational angle 01 can be
- 5 -determined from the motion and position sensors embodied in the HMD
electronics 8. By applying the opposite of the translational vector and the reverse rotational angle to the pixels in the video frame 7B, the differences between the two frames 7A and 7B can be removed.
FIG. 7C carries the example further, showing a frame of video that is now shifted left/down and clockwise relative to the previous frame 7B. A new vector A2 and rotational angle 02 can be applied, and so forth so that over time, small frame-to-frame variations caused by the wearer's head movement are removed from the displayed video. To distinguish between minor head movements, for which image stabilization should be applied, and gross head movements, which indicate the wearer's scene of interest has changed, requires that the image stabilization function only be applied to small translational and rotational vectors.
[0033] In FIGS. 8A and 8B, the angle of the camera 9 is adjusted not by physically rotating the camera as previously discussed. Rather, an area of pixels, or a window 13, 14 can be defined on the image sensor so that the wearer perceives that the camera angle is physically altered. This technique requires a camera system wherein the usable image sensor area is larger than the video that is to be displayed to the wearer. Motion and position sensors embodied in the HMD electronics 8 can be used to determine the wearer's head/neck angle 4 and, based on this information, define a capture window 13, 14 that gives the wearer the perception of a high or low camera angle 10.
[0034] While the present invention has been described in terms of certain exemplary preferred embodiments, it will be readily understood and appreciated by one of ordinary skill in the art that it is not so limited, and that many additions, deletions and modifications to the preferred embodiments may be made within the scope of the invention as hereinafter claimed.
Accordingly, the scope of the invention is limited only by the scope of the appended claims.
electronics 8. By applying the opposite of the translational vector and the reverse rotational angle to the pixels in the video frame 7B, the differences between the two frames 7A and 7B can be removed.
FIG. 7C carries the example further, showing a frame of video that is now shifted left/down and clockwise relative to the previous frame 7B. A new vector A2 and rotational angle 02 can be applied, and so forth so that over time, small frame-to-frame variations caused by the wearer's head movement are removed from the displayed video. To distinguish between minor head movements, for which image stabilization should be applied, and gross head movements, which indicate the wearer's scene of interest has changed, requires that the image stabilization function only be applied to small translational and rotational vectors.
[0033] In FIGS. 8A and 8B, the angle of the camera 9 is adjusted not by physically rotating the camera as previously discussed. Rather, an area of pixels, or a window 13, 14 can be defined on the image sensor so that the wearer perceives that the camera angle is physically altered. This technique requires a camera system wherein the usable image sensor area is larger than the video that is to be displayed to the wearer. Motion and position sensors embodied in the HMD electronics 8 can be used to determine the wearer's head/neck angle 4 and, based on this information, define a capture window 13, 14 that gives the wearer the perception of a high or low camera angle 10.
[0034] While the present invention has been described in terms of certain exemplary preferred embodiments, it will be readily understood and appreciated by one of ordinary skill in the art that it is not so limited, and that many additions, deletions and modifications to the preferred embodiments may be made within the scope of the invention as hereinafter claimed.
Accordingly, the scope of the invention is limited only by the scope of the appended claims.
- 6 -
Claims (22)
1. A head-worn device comprising:
a frame configured to mount onto a wearer's head including a first portion configured to be disposed on a first side of the wearer's head and a second portion configured to be disposed on a second distal side of the wearer's head; and a binocular electronic near-to-eye display assembly pivotally attached to the first portion of the frame and pivotally attached to the second portion of the frame such that the electronic near-to-eye display is restricted to pivot vertically across and in front of the wearer's face and pivots about an axis parallel to an axis defined between a left temple of the wearer's head and a right temple of the wearer's head; wherein the wearer views their environment directly with the electronic near-to-eye display assembly pivoted upwards in front of a forehead of the wearer's head above and out of their horizontal forward line of sight; views first visual content presented on the electronic near-to-eye display with the electronic near-to-eye display assembly pivoted down in front of their eyes into their line of sight; and the electronic near-to-eye display automatically pivots into different positions based upon a position of the wearer's head.
a frame configured to mount onto a wearer's head including a first portion configured to be disposed on a first side of the wearer's head and a second portion configured to be disposed on a second distal side of the wearer's head; and a binocular electronic near-to-eye display assembly pivotally attached to the first portion of the frame and pivotally attached to the second portion of the frame such that the electronic near-to-eye display is restricted to pivot vertically across and in front of the wearer's face and pivots about an axis parallel to an axis defined between a left temple of the wearer's head and a right temple of the wearer's head; wherein the wearer views their environment directly with the electronic near-to-eye display assembly pivoted upwards in front of a forehead of the wearer's head above and out of their horizontal forward line of sight; views first visual content presented on the electronic near-to-eye display with the electronic near-to-eye display assembly pivoted down in front of their eyes into their line of sight; and the electronic near-to-eye display automatically pivots into different positions based upon a position of the wearer's head.
2. The head-worn device according to claim 1, wherein when the electronic near-to-eye display is pivoted down in front of the wearer's eyes the viewer's direct view of their external environment is either blocked by the electronic near-to-eye display except peripherally; or through the electronic near-to-eye display.
3. The head-worn device according to claim 1, wherein the electronic near-to-eye display incorporates a camera for providing the first visual content;
and the vertical angle of the camera is at least one of manually adjustable or adjusts automatically according to a position of the wearer's head.
Date Recue/Date Received 2021-05-10
and the vertical angle of the camera is at least one of manually adjustable or adjusts automatically according to a position of the wearer's head.
Date Recue/Date Received 2021-05-10
4. The head-worn device according to claim 1, further comprising:
a camera associated with the wearer;
an orientation sensor; and a motion sensor, wherein image content acquired by the camera is stabilized using motion information provided by the orientation and motion sensors for presentation to the wearer as the first visual content.
a camera associated with the wearer;
an orientation sensor; and a motion sensor, wherein image content acquired by the camera is stabilized using motion information provided by the orientation and motion sensors for presentation to the wearer as the first visual content.
5. The head-worn device according to claim 4, wherein the vertical angle of the camera is automatically adjusted in dependence upon at least a task being performed by the wearer of the head-worn device and using information provided by at least one of the orientation sensor and the motion sensor.
6. The head-worn device according to claim 1, wherein the first visual content presented on the electronic near-to-eye display is second content received from an external source overlaid to third content established in dependence upon the wearer's field of view.
7. The head-worn device according to claim 1, wherein the electronic near-to-eye display assembly comprises a display and an optical element having a first surface adjacent the display and a second surface configured to be disposed towards a user's eye when the electronic near-to-eye display assembly is worn, wherein a normal perpendicular to the surface of the display is orthogonal to a line of sight of the user of the head-worn device when the user's head is in the second position.
8. The head-worn display according to claim 1, wherein the wearer views second visual content presented on the electronic near-to-eye display when the wearer's head is pivoted forward.
9. The head-worn display according to claim 8, wherein the first visual content and the second visual content are established by defining different areas of pixels within an image obtained from at least one of a camera and an external image source.
Date Recue/Date Received 2021-05-10
Date Recue/Date Received 2021-05-10
10. The head worn device according to claim 1, further comprising a camera for providing the first visual content forming part of the electronic near-to-eye display;
a motor coupled to the camera and to a controller forming part of the electronic near-to-eye display; and at least one of a position sensor providing position data to the controller relating to a position of the electronic near-to-eye display and a motion sensor providing motion data to the controller relating to motion of the electronic near-to-eye display; wherein the vertical angle of the camera adjusts automatically in dependence upon the controller processing the at least one of position data and motion data according to a position of the wearer's head.
a motor coupled to the camera and to a controller forming part of the electronic near-to-eye display; and at least one of a position sensor providing position data to the controller relating to a position of the electronic near-to-eye display and a motion sensor providing motion data to the controller relating to motion of the electronic near-to-eye display; wherein the vertical angle of the camera adjusts automatically in dependence upon the controller processing the at least one of position data and motion data according to a position of the wearer's head.
11. The head worn device according to claim 1, wherein the first visual content presented on the electronic near-to-eye display is stabilized using motion information provided by at least one of an orientation sensor forming part of the electronic near-to-eye display and a motion sensor forming part of the electronic near-to-eye display; and the stabilization is applied for translational and rotational vectors.
12. The head worn device according to claim 4, wherein the stabilization is applied for translational and rotational vectors.
13. A head mounted electronic near-to-eye display module, the electronic near-to-eye display module comprising:
a display electronics circuit in communication with a source of visual content;
a head mounted display optics having a first surface adjacent the display electronics circuit and a second surface disposed towards a wearer's head; and a first hinge for pivotally attaching the electronic near-to-eye display module to a first portion of a frame which is configured to be disposed on a first side of the wearer's head when worn by the wearer;
a second hinge for pivotally attaching the electronic near-to-eye display module to a second portion of the frame which is configured to be disposed on a second distal side of the wearer's head when worn by the wearer; wherein Date Recue/Date Received 2021-05-10 the first hinge and second hinge restrict the electronic near-to-eye display to pivoting vertically across and in front of the wearer's face when worn by the wearer;
the electronic near-to-eye display pivots about an axis parallel to an axis defined between a left temple of the wearer's head and a right temple of the wearer's head;
the wearer views their environment directly when the electronic near-to-eye display assembly is pivoted upwards in front of a forehead of the wearer's head above and out of their horizontal forward line of sight and views first visual content presented on the electronic near-to-eye display with the electronic near-to-eye display assembly pivoted down in front of their eyes into their line of sight; and the head mounted electronic near-to-eye display automatically pivots into different positions based upon a position of the wearer's head.
a display electronics circuit in communication with a source of visual content;
a head mounted display optics having a first surface adjacent the display electronics circuit and a second surface disposed towards a wearer's head; and a first hinge for pivotally attaching the electronic near-to-eye display module to a first portion of a frame which is configured to be disposed on a first side of the wearer's head when worn by the wearer;
a second hinge for pivotally attaching the electronic near-to-eye display module to a second portion of the frame which is configured to be disposed on a second distal side of the wearer's head when worn by the wearer; wherein Date Recue/Date Received 2021-05-10 the first hinge and second hinge restrict the electronic near-to-eye display to pivoting vertically across and in front of the wearer's face when worn by the wearer;
the electronic near-to-eye display pivots about an axis parallel to an axis defined between a left temple of the wearer's head and a right temple of the wearer's head;
the wearer views their environment directly when the electronic near-to-eye display assembly is pivoted upwards in front of a forehead of the wearer's head above and out of their horizontal forward line of sight and views first visual content presented on the electronic near-to-eye display with the electronic near-to-eye display assembly pivoted down in front of their eyes into their line of sight; and the head mounted electronic near-to-eye display automatically pivots into different positions based upon a position of the wearer's head.
14. The head mounted electronic near-to-eye display module according to claim 13, wherein when the electronic near-to-eye display is pivoted down in front of the wearer's eyes the viewer's direct view of their external environment is either blocked by the electronic near-to-eye display except peripherally; or through the electronic near-to-eye display.
15. The head mounted electronic near-to-eye display module according to claim 13, wherein the head mounted electronic near-to-eye display incorporates a camera for providing the first visual content; and the vertical angle of the camera is at least one of manually adjustable or adjusts automatically according to a position of the wearer's head.
16. The head mounted electronic near-to-eye display module according to claim 13, further comprising:
a camera associated with the wearer;
an orientation sensor; and a motion sensor, wherein image content acquired by the camera is stabilized using motion information provided by the orientation and motion sensors for presentation to the wearer as the first visual content.
Date Recue/Date Received 2021-05-10
a camera associated with the wearer;
an orientation sensor; and a motion sensor, wherein image content acquired by the camera is stabilized using motion information provided by the orientation and motion sensors for presentation to the wearer as the first visual content.
Date Recue/Date Received 2021-05-10
17. The head mounted electronic near-to-eye display module according to claim 16, wherein the vertical angle of the camera is automatically adjusted in dependence upon at least a task being performed by the wearer of the head mounted electronic near-to-eye display module and using information provided by at least one of the orientation sensor and the motion sensor.
18. The head mounted electronic near-to-eye display module according to claim 13, wherein a normal perpendicular to the surface of the display electronics circuit is orthogonal to a line of sight of a user of the head mounted electronic near-to-eye display module.
19. The head mounted electronic near-to-eye display module according to claim 13, wherein the first visual content presented on the electronic near-to-eye display is second content received from an external source overlaid to third content established in dependence upon the wearer's field of view.
20. The electronic near-to-eye display module according to claim 13, further comprising a camera for providing the first visual content forming part of the electronic near-to-eye display;
a motor coupled to the camera and to a controller forming part of the electronic near-to-eye display; and at least one of a position sensor providing position data to the controller relating to a position of the electronic near-to-eye display and a motion sensor providing motion data to the controller relating to motion of the electronic near-to-eye display; wherein the vertical angle of the camera adjusts automatically in dependence upon the controller processing the at least one of position data and motion data according to a position of the wearer's head.
a motor coupled to the camera and to a controller forming part of the electronic near-to-eye display; and at least one of a position sensor providing position data to the controller relating to a position of the electronic near-to-eye display and a motion sensor providing motion data to the controller relating to motion of the electronic near-to-eye display; wherein the vertical angle of the camera adjusts automatically in dependence upon the controller processing the at least one of position data and motion data according to a position of the wearer's head.
21. The electronic near-to-eye display module according to claim 13, further comprising a camera for providing the first visual content forming part of the electronic near-to-eye display;
a motor coupled to the camera and to a controller forming part of the electronic near-to-eye display; and at least one of a position sensor providing position data to the controller relating to a position of the electronic near-to-eye display and a motion sensor providing motion data to the controller relating to motion of the electronic near-to-eye display; wherein Date Recue/Date Received 2021-05-10 the vertical angle of the camera adjusts automatically in dependence upon the controller processing the at least one of position data and motion data according to a position of the wearer's head.
a motor coupled to the camera and to a controller forming part of the electronic near-to-eye display; and at least one of a position sensor providing position data to the controller relating to a position of the electronic near-to-eye display and a motion sensor providing motion data to the controller relating to motion of the electronic near-to-eye display; wherein Date Recue/Date Received 2021-05-10 the vertical angle of the camera adjusts automatically in dependence upon the controller processing the at least one of position data and motion data according to a position of the wearer's head.
22. The electronic near-to-eye display module according to claim 16, wherein the stabilization is applied for translational and rotational vectors.
Date Recue/Date Received 2021-05-10
Date Recue/Date Received 2021-05-10
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA3040218A CA3040218C (en) | 2012-06-01 | 2012-06-01 | Apparatus and method for a bioptic real time video system |
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CA2012/000532 WO2013177654A1 (en) | 2012-06-01 | 2012-06-01 | Apparatus and method for a bioptic real time video system |
| CA3040218A CA3040218C (en) | 2012-06-01 | 2012-06-01 | Apparatus and method for a bioptic real time video system |
| CA2875261A CA2875261C (en) | 2012-06-01 | 2012-06-01 | Apparatus and method for a bioptic real time video system |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2875261A Division CA2875261C (en) | 2012-06-01 | 2012-06-01 | Apparatus and method for a bioptic real time video system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA3040218A1 CA3040218A1 (en) | 2013-12-05 |
| CA3040218C true CA3040218C (en) | 2021-12-21 |
Family
ID=49672212
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2875261A Active CA2875261C (en) | 2012-06-01 | 2012-06-01 | Apparatus and method for a bioptic real time video system |
| CA3040218A Active CA3040218C (en) | 2012-06-01 | 2012-06-01 | Apparatus and method for a bioptic real time video system |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2875261A Active CA2875261C (en) | 2012-06-01 | 2012-06-01 | Apparatus and method for a bioptic real time video system |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP2859399A4 (en) |
| CA (2) | CA2875261C (en) |
| WO (1) | WO2013177654A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11282284B2 (en) | 2016-11-18 | 2022-03-22 | Eyedaptic, Inc. | Systems for augmented reality visual aids and tools |
| US11385468B2 (en) | 2018-05-29 | 2022-07-12 | Eyedaptic, Inc. | Hybrid see through augmented reality systems and methods for low vision users |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2985535A1 (en) * | 2015-05-15 | 2016-11-24 | Vertical Optics, LLC | Wearable vision redirecting devices |
| IL244255A (en) | 2016-02-23 | 2017-04-30 | Vertical Optics Llc | Wearable vision redirecting devices |
| US9690119B2 (en) | 2015-05-15 | 2017-06-27 | Vertical Optics, LLC | Wearable vision redirecting devices |
| US20190012841A1 (en) | 2017-07-09 | 2019-01-10 | Eyedaptic, Inc. | Artificial intelligence enhanced system for adaptive control driven ar/vr visual aids |
| US10984508B2 (en) | 2017-10-31 | 2021-04-20 | Eyedaptic, Inc. | Demonstration devices and methods for enhancement for low vision users and systems improvements |
| US11563885B2 (en) | 2018-03-06 | 2023-01-24 | Eyedaptic, Inc. | Adaptive system for autonomous machine learning and control in wearable augmented reality and virtual reality visual aids |
| WO2020068819A1 (en) | 2018-09-24 | 2020-04-02 | Eyedaptic, Inc. | Enhanced autonomous hands-free control in electronic visual aids |
| IT201900013164A1 (en) * | 2019-07-29 | 2021-01-29 | Eye Tech Lab S R L | AUGMENTED REALITY MAGNIFYING GLASSES |
Family Cites Families (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5642221A (en) * | 1994-03-09 | 1997-06-24 | Optics 1, Inc. | Head mounted display system |
| US5777715A (en) * | 1997-01-21 | 1998-07-07 | Allen Vision Systems, Inc. | Low vision rehabilitation system |
| US5971538A (en) * | 1998-10-30 | 1999-10-26 | Hewlett-Packard Company | Articulated nose bridge for head mounted display |
| JP2003046903A (en) * | 2001-07-31 | 2003-02-14 | Sanyo Electric Co Ltd | Display device |
| EP1300716A1 (en) * | 2001-10-08 | 2003-04-09 | Visys AG | Head mounted display apparatus |
| US20040004559A1 (en) * | 2002-07-01 | 2004-01-08 | Rast Rodger H. | Keyboard device with preselect feedback |
| US7145726B2 (en) * | 2002-08-12 | 2006-12-05 | Richard Geist | Head-mounted virtual display apparatus for mobile activities |
| US20080068557A1 (en) * | 2006-09-20 | 2008-03-20 | Gilbert Menduni | Lens holding frame |
| US7484847B2 (en) * | 2007-01-02 | 2009-02-03 | Hind-Sight Industries, Inc. | Eyeglasses having integrated telescoping video camera and video display |
| US9300949B2 (en) * | 2007-01-29 | 2016-03-29 | David J. Ahearn | Multi-view system |
| US9618748B2 (en) * | 2008-04-02 | 2017-04-11 | Esight Corp. | Apparatus and method for a dynamic “region of interest” in a display system |
| US20090040296A1 (en) * | 2007-08-06 | 2009-02-12 | Moscato Jonathan D | Head mounted display assembly |
| JP5871809B2 (en) * | 2009-11-21 | 2016-03-01 | マゲーリー, ダグラス, ピーターMAGYARI, Douglas, Peter | Head-mounted display device |
| US8665177B2 (en) * | 2010-02-05 | 2014-03-04 | Kopin Corporation | Touch sensor for controlling eyewear |
| US9560324B2 (en) * | 2010-02-12 | 2017-01-31 | Drew Incorporated | Tactical vision system |
| WO2011106798A1 (en) * | 2010-02-28 | 2011-09-01 | Osterhout Group, Inc. | Local advertising content on an interactive head-mounted eyepiece |
-
2012
- 2012-06-01 WO PCT/CA2012/000532 patent/WO2013177654A1/en unknown
- 2012-06-01 CA CA2875261A patent/CA2875261C/en active Active
- 2012-06-01 EP EP12877968.3A patent/EP2859399A4/en not_active Withdrawn
- 2012-06-01 CA CA3040218A patent/CA3040218C/en active Active
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11282284B2 (en) | 2016-11-18 | 2022-03-22 | Eyedaptic, Inc. | Systems for augmented reality visual aids and tools |
| US11385468B2 (en) | 2018-05-29 | 2022-07-12 | Eyedaptic, Inc. | Hybrid see through augmented reality systems and methods for low vision users |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2013177654A1 (en) | 2013-12-05 |
| CA2875261A1 (en) | 2013-12-05 |
| EP2859399A1 (en) | 2015-04-15 |
| CA3040218A1 (en) | 2013-12-05 |
| CA2875261C (en) | 2019-05-21 |
| EP2859399A4 (en) | 2016-01-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10495885B2 (en) | Apparatus and method for a bioptic real time video system | |
| CA3040218C (en) | Apparatus and method for a bioptic real time video system | |
| US10962788B2 (en) | Head-mounted device with active optical foveation | |
| KR102295452B1 (en) | Microdisplay based immersive headset | |
| KR102729054B1 (en) | Position tracking system for head-mounted displays containing sensor integrated circuits | |
| JP6573593B2 (en) | Wearable device having input / output structure | |
| JP6083880B2 (en) | Wearable device with input / output mechanism | |
| US9696552B1 (en) | System and method for providing an augmented reality lightweight clip-on wearable device | |
| US8619005B2 (en) | Switchable head-mounted display transition | |
| US20080174659A1 (en) | Wide field of view display device and method | |
| US20110234475A1 (en) | Head-mounted display device | |
| CN112655202B (en) | Reduced bandwidth stereoscopic distortion correction for fisheye lenses of head-mounted displays | |
| US11619814B1 (en) | Apparatus, system, and method for improving digital head-mounted displays | |
| US20240314281A1 (en) | Electronic Device that Displays Virtual Objects | |
| US20220146856A1 (en) | Head-mounted display apparatus | |
| US20230185090A1 (en) | Eyewear including a non-uniform push-pull lens set | |
| GB2562808A (en) | Reality viewer | |
| WO2020137088A1 (en) | Head-mounted display, display method, and display system | |
| CN214846063U (en) | Visual glasses | |
| CN117170602A (en) | Electronic device for displaying virtual object |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request |
Effective date: 20190415 |